feat(examples): run the OpenCode example on one GPU with Qwen3.8-27B-FP8

examples/README.md

@@ -20,7 +20,7 @@ to a separately maintained repository (clone and run there).
 | [Extract custom entities from primary sources](./named-entity-extraction) | Changing zero-shot labels across financial, healthcare, rail-safety and legal text, with the two labels that came back empty left visible | `extract` | SIE endpoint with GLiNER multi v2.1; `fetch.py` pulls the recorded run, `score.py` reproduces the counts with no key | Runnable recorded example |
 | [Extract the entity types you name, with no model to train](./custom-entity-types) | Asking for the 85 entity types of seven zero-shot NER sets and scoring exact spans against GPT-6 Luna, GPT-6 Sol, Claude Haiku 4.5, SIE Qwen3.8 27B, AWS Comprehend and spaCy, with a paired bootstrap and the price of each | `extract` | SIE server with GLiNER BioMed Large; `fetch.py` pulls the recorded run, `score.py` reproduces the figures with no key | Runnable recorded example |
 | [Answer a question from a passage, and quote the sentence you used](./chat) | Checking a generated answer for its citation, its length and whether it declines a question the passage cannot answer | `chat/completions` | SIE Cloud with Qwen3.8 27B; `fetch.py` pulls the recorded run, `score.py` reproduces the numbers with no key | Runnable recorded example |
-| [Run a coding agent on SIE with OpenCode](./coding-agent-opencode) | Pointing OpenCode at a laptop server, a self-hosted cluster or SIE Cloud, and checking the agent loop the way OpenCode drives it: streaming tool-call deltas, parallel calls, tool-result round trips, a 64K-plus prompt, finish reasons, usage and the reasoning field | `chat/completions` | An SIE Cloud endpoint that lists GLM-5.3-Flash via SIE Relay, or a local server; the laptop recipe was run on Apple Silicon with Qwen3.5-4B; `conformance.py` reports every case, `run_gate.sh` makes OpenCode fix a seeded failing test | Runnable check, laptop and SIE Cloud results recorded |
+| [Run a coding agent on SIE with OpenCode](./coding-agent-opencode) | Pointing OpenCode at a laptop server, one 80 GB GPU, a self-hosted cluster or SIE Cloud, and checking the agent loop the way OpenCode drives it: streaming tool-call deltas, parallel calls, tool-result round trips, a 64K-plus prompt, a 64 KiB tool call, finish reasons, usage and the reasoning field; `session_load.py` measures how many agent sessions one endpoint holds | `chat/completions` | An SIE Cloud endpoint that lists GLM-5.3-Flash via SIE Relay, or a local server; the laptop recipe was run on Apple Silicon with Qwen3.5-4B, the GPU and cluster recipes on one H100 with Qwen3.8-27B-FP8; `conformance.py` reports every case, `run_gate.sh` makes OpenCode fix a seeded failing test | Runnable check, laptop, one-GPU and SIE Cloud results recorded |
 | [Route requests while skipping most generation calls](./request-routing) | Assigning support, banking and assistant requests to routes with an embedding classifier and conditional generation | `encode`, `chat/completions` | SIE server with Qwen3 Embedding 4B and Qwen3.8 27B; recorded 3,000-case verification needs no API key | Runnable recorded example |
 | [Send each request to a cheap or a strong LLM](./request-routing/cost) | Cutting LLM spend with a bge-m3 nearest-neighbour router that picks GPT-6 Luna or GPT-6.1 Sol per request: 38.6% less than always using GPT-6.1 Sol at 95.8% of its quality on 728 held-out requests of a pre-registered study | `encode` | SIE Serverless with bge-m3; `fetch.py` pulls the recorded study, `score.py` reproduces the held-out table with no key | Runnable recorded example |
 | [Turn PDF pages into Markdown, scored on olmOCR-Bench](./document-to-markdown-olmocr) | Converting all 1,403 pages of Ai2's olmOCR-Bench with LightOnOCR-2-1B and scoring them with Ai2's own scorer, beside published Azure Document Intelligence and AWS Textract runs | `extract` | SIE Cloud with LightOnOCR-2-1B; `fetch.py` pulls the recorded run, `score.py` reproduces the scores with no key | Runnable recorded example |

examples/coding-agent-opencode/conformance.py

@@ -62,6 +62,18 @@
 NEEDLE = "The access code is 7741-ZEBRA."
 RECORDED_HEADERS = ("x-sie-served-by", "x-sie-model-revision")
 REASONING_QUESTION = "Which is larger, 9.11 or 9.9? Answer with just the larger number."
+LONG_CALL_MIN_BYTES = 65_536
+LONG_CALL_BUDGET = 32_000
+LONG_CALL_LINES = 1_320
+LONG_CALL_PATH = "notes/pangrams.txt"
+LONG_CALL_SENTENCE = "The quick brown fox jumps over the lazy dog."
+LONG_CALL_PROMPT = (
+    f"Use the write_file tool once to create the file {LONG_CALL_PATH}. Its content must be {LONG_CALL_LINES:,} "
+    f"lines numbered 0001 to {LONG_CALL_LINES:04d}: each line is its four-digit number, one space and the sentence "
+    f'"{LONG_CALL_SENTENCE}", about {LONG_CALL_LINES * (len(LONG_CALL_SENTENCE) + 6):,} characters in all. Write '
+    "every line out in full in that one call: no ellipsis, no placeholder, no summary and no code that generates "
+    "the lines."
+)
 MESSAGES_HEADERS = {
     "anthropic-version": "2023-06-01",
     "anthropic-beta": "claude-code-20250219,interleaved-thinking-2025-05-14",
@@ -86,6 +98,22 @@
     },
 }
 
+WRITE_TOOL = {
+    "type": "function",
+    "function": {
+        "name": "write_file",
+        "description": "Create or replace one text file.",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "path": {"type": "string", "description": "File path relative to the project root"},
+                "content": {"type": "string", "description": "The whole file content"},
+            },
+            "required": ["path", "content"],
+        },
+    },
+}
+
 FILLER_WORDS = [
     "the",
     "ledger",
@@ -156,6 +184,9 @@ class StreamResult:
     error: dict[str, Any] | None = None
     retries: list[dict[str, Any]] = field(default_factory=list)
     seconds: float = 0.0
+    first_chunk_s: float | None = None
+    stream_s: float = 0.0
+    longest_gap_s: float = 0.0
 
     def calls(self) -> list[ToolCall]:
         return [self.tool_calls[index] for index in sorted(self.tool_calls)]
@@ -305,6 +336,7 @@ def complete_chat(client: openai.OpenAI, model: str, messages: list[dict[str, An
 
 def _stream_once(client: openai.OpenAI, model: str, messages: list[dict[str, Any]], **params: Any) -> StreamResult:
     result = StreamResult()
+    started = previous = time.monotonic()
     try:
         raw = client.chat.completions.with_raw_response.create(
             model=model,
@@ -315,6 +347,13 @@ def _stream_once(client: openai.OpenAI, model: str, messages: list[dict[str, Any
         )
         _keep_headers(result, raw)
         for chunk in raw.parse():
+            now = time.monotonic()
+            if result.first_chunk_s is None:
+                result.first_chunk_s = now - started
+            else:
+                result.longest_gap_s = max(result.longest_gap_s, now - previous)
+                result.stream_s = now - started - result.first_chunk_s
+            previous = now
             result.chunks += 1
             result.usage_on_last_chunk = False
             usage = getattr(chunk, "usage", None)
@@ -372,6 +411,10 @@ def _complete_once(client: openai.OpenAI, model: str, messages: list[dict[str, A
     return result
 
 
+def _describe_names(calls: list[ToolCall]) -> str:
+    return ", ".join(str(call.name) for call in calls)
+
+
 def _describe_calls(result: StreamResult) -> str:
     return ", ".join(f"{call.name}({call.arguments})" for call in result.calls()) or "no tool call"
 
@@ -741,6 +784,76 @@ def case_tool_choice_required(client: openai.OpenAI, model: str) -> Case:
     )
 
 
+def long_call_line(number: int) -> str:
+    return f"{number:04d} {LONG_CALL_SENTENCE}"
+
+
+def case_long_tool_call(client: openai.OpenAI, model: str, tool_params: dict[str, Any]) -> Case:
+    name = "long_tool_call"
+    messages = [{"role": "user", "content": LONG_CALL_PROMPT}]
+    result = stream_chat(
+        client, model, messages, tools=[WRITE_TOOL], max_tokens=LONG_CALL_BUDGET, temperature=0, **tool_params
+    )
+    calls = result.calls()
+    size = sum(len(call.arguments.encode()) for call in calls)
+    observed = result.observed()
+    observed.update(
+        max_tokens=LONG_CALL_BUDGET,
+        argument_bytes=size,
+        first_chunk_s=None if result.first_chunk_s is None else round(result.first_chunk_s, 2),
+        stream_s=round(result.stream_s, 2),
+        longest_gap_s=round(result.longest_gap_s, 2),
+    )
+    timing = f"streamed over {result.stream_s:.0f} s, longest gap between chunks {result.longest_gap_s:.1f} s"
+    if result.error and _refuses_field(result.error, "max_tokens"):
+        error = result.error
+        detail = (
+            f"not run: max_tokens={LONG_CALL_BUDGET} refused: {error['status']} {error['code']}: {error['message']}"
+        )
+        return Case(name, "recorded", detail, observed)
+    failed = _fail_on_error(name, result)
+    if failed:
+        failed.observed = observed
+        failed.detail = f"{failed.detail} ({timing})"
+        return failed
+    if not calls:
+        detail = f"the model answered without a tool call: {result.content.strip()[:80]!r}"
+        return Case(name, "recorded", detail, observed)
+    arguments = [call.parsed_arguments() for call in calls]
+    if result.finish_reason == "length":
+        detail = f"max_tokens={LONG_CALL_BUDGET} ended the turn after {size:,} bytes of arguments ({timing})"
+        return Case(name, "recorded", detail, observed)
+    if None in arguments:
+        detail = f"{size:,} bytes of arguments, but not a JSON object: truncated or corrupted ({timing})"
+        return Case(name, "fail", detail, observed)
+    if result.finish_reason != "tool_calls":
+        detail = f"finish_reason is {result.finish_reason!r} after a complete call, expected 'tool_calls'"
+        return Case(name, "fail", detail, observed)
+    content = next((args.get("content") for args in arguments if args and isinstance(args.get("content"), str)), "")
+    observed["content_lines"] = len(content.splitlines())
+    observed["lines_as_asked"] = sum(
+        line.strip() == long_call_line(number) for number, line in enumerate(content.splitlines(), 1)
+    )
+    kib = size / 1024
+    if size < LONG_CALL_MIN_BYTES:
+        detail = f"the model wrote {kib:.1f} KiB of arguments, under the {LONG_CALL_MIN_BYTES // 1024} KiB asked for"
+        return Case(name, "recorded", detail, observed)
+    paths = [args.get("path") for args in arguments if args]
+    as_asked = (
+        [call.name for call in calls] == ["write_file"]
+        and paths == [LONG_CALL_PATH]
+        and observed["lines_as_asked"] == LONG_CALL_LINES == observed["content_lines"]
+    )
+    if not as_asked:
+        detail = (
+            f"{kib:.1f} KiB arrived as JSON, but not the file asked for: {_describe_names(calls)} to {paths}, "
+            f"{observed['lines_as_asked']} of {observed['content_lines']} lines as asked, {LONG_CALL_LINES} wanted"
+        )
+        return Case(name, "recorded", detail, observed)
+    detail = f"{kib:.1f} KiB of arguments in {len(calls)} call ({observed['content_lines']} lines), {timing}"
+    return Case(name, "pass", detail, observed)
+
+
 def model_entry(base_url: str, api_key: str, model: str) -> dict[str, Any] | None:
     headers = {"authorization": f"Bearer {api_key}"} if api_key else {}
     try:
@@ -816,6 +929,7 @@ def run(base_url: str, api_key: str, model: str) -> dict[str, Any]:
     cases.append(case_usage_on_final_chunk(stop_result))
     cases.append(reasoning_case)
     cases.append(case_tool_choice_required(client, model))
+    cases.append(case_long_tool_call(client, model, tool_params))
 
     statuses = ("pass", "fail", "recorded", "capacity")
     counts = {status: sum(1 for case in cases if case.status == status) for status in statuses}

examples/coding-agent-opencode/recipes/cluster/opencode.json

@@ -1,6 +1,6 @@
 {
   "$schema": "https://opencode.ai/config.json",
-  "model": "sie-cluster/zai-org/GLM-5.3-Flash",
+  "model": "sie-cluster/Qwen/Qwen3.8-27B-FP8:h100-256k-batch",
   "provider": {
     "sie-cluster": {
       "npm": "@ai-sdk/openai-compatible",
@@ -28,6 +28,17 @@
             "output": 4096
           },
           "tool_call": true
+        },
+        "Qwen/Qwen3.8-27B-FP8:h100-256k-batch": {
+          "name": "Qwen3.8-27B on the cluster, 256K window",
+          "limit": {
+            "context": 262144,
+            "output": 32768
+          },
+          "tool_call": true,
+          "options": {
+            "presence_penalty": 0
+          }
         }
       }
     }

examples/coding-agent-opencode/recipes/gpu/opencode.json

@@ -0,0 +1,30 @@
+{
+  "$schema": "https://opencode.ai/config.json",
+  "model": "sie-gpu/Qwen/Qwen3.8-27B-FP8:h100-256k-batch",
+  "provider": {
+    "sie-gpu": {
+      "npm": "@ai-sdk/openai-compatible",
+      "name": "SIE on one GPU",
+      "options": {
+        "baseURL": "http://localhost:8080/v1",
+        "apiKey": "local",
+        "timeout": 1800000,
+        "headerTimeout": 600000,
+        "chunkTimeout": 600000
+      },
+      "models": {
+        "Qwen/Qwen3.8-27B-FP8:h100-256k-batch": {
+          "name": "Qwen3.8-27B, 256K window, one 80 GB GPU",
+          "limit": {
+            "context": 262144,
+            "output": 32768
+          },
+          "tool_call": true,
+          "options": {
+            "presence_penalty": 0
+          }
+        }
+      }
+    }
+  }
+}

examples/coding-agent-opencode/run_gate.sh

@@ -17,7 +17,7 @@ HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 RECIPE="${1:-cloud}"
 CONFIG="$HERE/recipes/$RECIPE/opencode.json"
 if [[ ! -f "$CONFIG" ]]; then
-    echo "unknown recipe '$RECIPE'; pick laptop, cluster or cloud" >&2
+    echo "unknown recipe '$RECIPE'; pick laptop, gpu, cluster or cloud" >&2
     exit 2
 fi
 for tool in opencode uv; do
@@ -34,7 +34,7 @@ for var in GATE_STEPS GATE_DEADLINE_SECONDS; do
         exit 2
     fi
 done
-if [[ "$RECIPE" != "laptop" ]]; then
+if grep -q '{env:SIE_' "$CONFIG"; then
     for var in SIE_BASE_URL SIE_API_KEY; do
         if [[ -z "${!var:-}" ]]; then
             echo "$var is not set" >&2