English | 简体中文

Qwen3.6-35B-A3B on NVIDIA GB10

Release certification in progress. The current public reference candidate is PRIVATE_REFERENCE_VALIDATED; STABLE_RELEASE remains UNRELEASED until the signed image and independent Test15 reproduction are bound into v1.0.0.

Full-BF16 Qwen3.6-35B-A3B inference for NVIDIA GB10 and DGX Spark, powered by a tuned SGLang runtime, Qwen3.6-35B-A3B-DFlash speculative decoding, and FlashInfer attention.

Live Three-Engine Comparison

The same Qwen3.6-35B-A3B BF16 request is replayed side by side through vLLM, llama.cpp, and the GB10 specialized engine on one NVIDIA GB10. All three runs use the same prompt bytes, batch size 1, temperature 0, cross-request cache disabled, and real SSE arrival timing.

Watch the full-resolution MP4. This recording is a visual comparison; the versioned reference evidence and certification status below remain authoritative.

Reference Performance

Profile	Cache semantics	TTFT	Prefill	Decode
cache-on	cold-cache core workload	24.82 s	4,263 tok/s	74.6 tok/s
cache-on-hit	primed-prefix supplemental workload	7.90 s	8,295 tok/s	73.8 tok/s
cache-off	uncached cold requests	25.06 s	4,226 tok/s	74.1 tok/s

These are three-round medians from the public reference candidate on one GB10, not yet the final two-machine release claim. Inspect the candidate summary and the round-level cache-on cold-cache evidence, cache-on primed-prefix evidence, and cache-off cold-request evidence. The final README values will be checked against releases/v1.0.0/release-lock.yaml.

Quick Start

Future formal-tag commands cannot run while STABLE_RELEASE is UNRELEASED. Until then, Star or watch the repository and inspect the build and benchmark methodology.

git clone https://github.com/Approaching-AI/qwen36-gb10-highspeed.git
cd qwen36-gb10-highspeed
export QWEN36_POWER_SUPPLY_ATTESTED=1
./reproduce.sh cache-on --verify
./reproduce.sh cache-off --verify

Why This Exists

Running a model on GB10 is not the same as reproducing a tuned result. This project locks the runtime, model files, profile arguments, workload, host constraints, and raw evidence so another GB10 can test the same delivery.

What the Release Locks

The official reproduction target covers only NVIDIA GB10 on Linux ARM64, including a DGX Spark of the same specification. Other GPUs, drivers, kernels, firmware, or power conditions may run in compatibility mode but cannot produce a strict reproduction result.

Model weights are not included in the image or Git repository. The host first verifies existing main-model and DFlash files against the release manifest and downloads only missing or mismatched files from pinned revisions, then mounts both directories read-only. Keeping model weights outside the image allows the runtime to be audited, upgraded, and signed independently and avoids redistributing approximately 68 GiB of weights.

Locked runtime-required model files total 72,874,798,157 bytes (approximately 67.87 GiB):

• Qwen3.6-35B-A3B: 71,926,788,304 bytes.
• Qwen3.6-35B-A3B-DFlash: 948,009,853 bytes.

Strict preflight requires at least 160,000,000,000 bytes free on the filesystem containing the model root for weights, resumable-download temporary files, the runtime image, and results. A formal release-lock.yaml also binds the Git commit, model revisions and per-file hashes, profile and workload, reference results, immutable image digest, Cosign identity, SBOM, and build provenance. The public candidate does not yet supply that final signed release contract.

Reproduction Standard

Prerequisites

Install Python 3.11+, Docker Engine, Docker Compose v2, NVIDIA Container Toolkit (including nvidia-container-cli), nvidia-smi, and Cosign. The executing user must be able to access the Docker daemon. Before a strict test, stop other GPU workloads, use the NVIDIA-supplied 240W power adapter, and satisfy the release-locked driver, CUDA compatibility, firmware, and kernel constraints. DGX Spark has unified memory, so preflight validates /proc/meminfo and does not treat Memory-Usage: Not Supported from nvidia-smi as a fault; NVIDIA lists this as expected behavior.

The strict environment report contains a device identity commitment computed with a candidate-specific random salt. It distinguishes whether two certification runs came from different Linux machine identities without writing raw /etc/machine-id into the report. Formal two-machine certification requires both different commitments and a human attestation of independent physical machines because an ordinary software report is not hardware remote attestation.

Sources and execution

By default, --source auto tries Hugging Face first and falls back to ModelScope. You can also select a source explicitly:

./reproduce.sh cache-on --source auto --verify
./reproduce.sh cache-on --source huggingface --verify
./reproduce.sh cache-on --source modelscope --verify

The first run creates a local Python environment, verifies and reuses matching local models (downloading only files that are absent or invalid), verifies the image signature, pulls the image by digest, starts the service, and runs three benchmark rounds. See the Complete Reproduction Guide for the complete workflow and Troubleshooting for host checks and image-verification failures.

Profiles

cache-on enables Radix Cache for services that reuse prefixes; its baseline comes from R5ax. cache-off disables Radix Cache for uncached cold requests; its baseline comes from R5ee. Both profiles use Qwen3.6-35B-A3B BF16, DFlash speculative decoding, FlashInfer attention, and a customized Triton/SGLang runtime.

Setting	cache-on	cache-off
Radix Cache	enabled	disabled
static memory fraction	0.80	0.75
Mamba scheduler	extra_buffer	no_buffer
DFlash block/window	12 / 8192	12 / 8192
target/draft attention	FlashInfer / FlashInfer	FlashInfer / FlashInfer
chunk/max prefill tokens	32768 / 32768	32768 / 32768
CUDA Graph batch sizes	1, 2, 4	1, 2, 4

The complete auditable arguments are in the cache-on profile and cache-off profile. Runtime sources and patch hashes are recorded in container/source-manifest.txt.

Outcomes

Both profiles use a cold-cache core workload. TTFT, Prefill, and Decode are each reduced to a three-round median. Every across-round CV must be no greater than 3%, and all three metrics must be within ±5% of the signed reference.

A run has exactly one of these outcomes:

• REPRODUCED: all three metrics are within ±5% and stability is valid.
• PERFORMANCE_PASS_BASELINE_DIVERGENCE: performance did not regress, but at least one metric is more than 5% better than baseline and requires an environment or timing audit.
• NOT_REPRODUCED: at least one metric is below its acceptance threshold, or execution failed.
• INVALID_RUN: at least one across-round CV exceeds 3%, preventing noise from being mislabeled as a performance failure.
• COMPATIBILITY_RUN: the hardware or software fingerprint is not the strict target environment.

A formal release has two additional machine-certification levels:

• REFERENCE_VALIDATED: validated from a clean environment on the original optimized GB10.
• REPRODUCED_ON_SECOND_GB10: passed again on a second independent GB10 of the same specification. Only this level supports a claim of reproducibility on equivalent hardware.

The two-machine gate combines distinct salted device identity commitments, independent raw results, and human confirmation by the publisher. It is not presented as TPM-level physical-device proof. Candidate builds are explicitly labeled CANDIDATE and cannot impersonate either certification level. See Benchmark Methodology for formulas, cache semantics, and statistical rules.

Community

• Star or watch the repository for the certified v1.0.0.
• After a formal tag is published, reproduce both profiles on a GB10 and share sanitized evidence through the reproduction Issue template.
• Read CONTRIBUTING.md before proposing runtime changes.

Documentation

• Complete Reproduction Guide
• Benchmark Methodology
• Build from Source
• Candidate Build and Formal Release
• Private Reference-Machine Validation
• Troubleshooting
• Third-Party License Inventory
• Security Policy

Project-owned control-plane code is licensed under the Apache License 2.0. Models, the base image, and third-party components remain subject to their respective terms. Read the Third-Party License Inventory before downloading or using them.