[Sunghoon Choi (hanbitmyths)]

Summary

• Forward Mobius task-level prune_prefill_prefix, runtime_last_token_indices and disable_deepstack separately from model-specific options.
• Add recursive vision-processor overrides, validated against full-package export and actual processor generation; preserve existing GenAI overrides/defaults.
• Add case-sensitive RTN exact/glob node filters (*/?), with exclusions taking precedence.

Dependencies: Mobius #813, GenAI #2889; used by recipe #694. RTN globs are included here.

Reported validation: 111 combined RTN/MobiusBuilder tests; actual export, projection counts, processor/runtime contracts, external data and FP16 embedding parity verified. Format checks passed; no new lint diagnostics.

Performance: Qwen3-VL-2B

Combined package/runtime results, not isolated gains from this PR. Production uses decoder INT4/group32; optimized uses 66 INT8 + 131 INT4 projections, FP16 vision/embeddings/KV, pruned prefill and decoder capture.

A100-SXM4-80GB

Historical production: Oct 28 (b679913a2310); optimized: Nov 5 frozen clean gate.

Tokens	Production TTFT ms	Optimized TTFT ms	Production decode tok/s	Optimized decode tok/s	Change
128	21.25	10.73	232.16	349.41	+50.50%
512	29.52	17.66	230.47	340.51	+47.75%
2048	62.12	47.78	219.57	325.15	+48.08%
4096	122.50	100.42	212.16	304.94	+43.73%
7680	241.71	206.10	202.21	288.68	+42.76%

H200

Baseline: official qwen3-vl-2b-instruct-cuda-gpu:2 on published ORT 1.31.1 / GenAI 0.19.0, CUDA 13. Optimized: retained Nov 8 FPA1 results, not rerun.

Workload	Baseline TTFT ms	Optimized TTFT ms	Baseline decode tok/s	Optimized decode tok/s	Change (unpaired)
Text 128	9.06	8.04	220.0	514.9	+134.08%
Text 512	12.29	11.08	216.0	516.7	+139.25%
Text 2048	26.07	25.98	211.7	504.8	+138.42%
Text 4096	52.39	51.55	205.3	481.0	+134.26%
Text 7680	104.96	103.56	201.6	467.3	+131.85%
Image + text	30.55	40.90	216.9	514.4	+137.16%

Controls: batch 1, context 8192, FP16 KV, chunk 2048, 128 outputs, 8 inference threads, no prefix reuse. TTFT is HTTP first-token latency; decode uses 127 first-to-last intervals. A100: 7 production / 14 optimized samples. H200 baseline: 14 samples/row after warmups, GPU 1 / CPU cores 8-15; pre-rendered prompts, tokenization/image processing inside HTTP.

Limits: H200 is unpaired: prompts/harness differ; the new 448x448 image has 244 input tokens versus 356 historically. Ratios are descriptive, not causal speedups. A100 is not matched to the new H200 runtime; no cross-GPU hardware claim. Optimized ORT 5481f20b / GenAI 4e51307d + 41c96365 are frozen, not current PR-head benchmarks.

Review required: historical optimized export loses 9 AI2D answers versus the accepted native recipe; padded/batched capture limitations remain. No default promotion.

[Copilot]

🔵 Needs a closer look

The decoder-contract changes depend on companion Mobius/GenAI support, while mocked tests and pre-rebase end-to-end results leave rebased runtime compatibility and the reported AI2D quality regression requiring human assessment.

No open issues 🎉

What changed in this PR

Extends Olive’s Mobius export pass with opt-in pruning and compatibility controls plus recursive vision-processor configuration overrides, while preserving existing defaults and option forwarding.

Changes:

• Forward pruning, runtime final-token index, and DeepStack controls separately from model-specific extra_options.
• Merge processor overrides into generated JSON, requiring a full-package export and a generated vision processor.
• Add regression coverage for validation, option forwarding, and composite-package sidecar and quantization behavior using existing RTN exact-name selection.

File	Description
test/​passes/​onnx/​test_mobius_model_builder.py	Tests new controls, processor overrides, and composite RTN artifact preservation.
olive/​passes/​onnx/​mobius_model_builder.py	Adds optional controls, validation, and shared recursive JSON override handling.

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The pruning documentation misstates right-padding behavior, and the regression test can pass when expected package sidecars are missing.

No open issues 🎉

2 previously missed issues

In code that hasn't changed since last review

Clarify pruning selects the final sequence position, not the final valid token

olive/​passes/​onnx/​mobius_model_builder.py:158

With prune_prefill_prefix=True alone, the companion Mobius implementation selects sequence position -1, not the final non-padding token. For a right-padded batch this projects a padding position, so “final valid token” incorrectly implies padding-aware selection. Please describe this as the final sequence position and clarify that right-padded Qwen3-VL batches also require runtime_last_token_indices.

Assert both expected sidecar files are present

test/​passes/​onnx/​test_mobius_model_builder.py:444

These assertions accept an empty additional_files list, or a list containing only one sidecar, because they only inspect whichever paths remain. Consequently, the regression test still passes if either package configuration is dropped by the quantization pipeline. Assert that both expected output paths are present before checking their contents.

_{Review effort: Balanced}