Repository navigation
fix: pass ModelBuilder int4 options under their current names - #2719
Yuri Khrustalev (ykhrustalev) wants to merge 3 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
onnxruntime-genai 0.16 dropped the int4_ prefix from the model builder's quantization options and no longer accepts the old names, so int4_block_size, int4_algo_config and the other int4_ settings, including the ones olive optimize and capture-onnx-graph set, were silently ignored.
There was a problem hiding this comment.
🟡 Changes recommended
Documented extra_mb_options still bypass normalization and forward unsupported prefixed names.
1 open finding
What changed in this PR
Aligns ModelBuilder INT4 option forwarding with current ONNX Runtime GenAI names while preserving Olive’s pass configuration names.
Changes:
- Removes
int4_prefixes before forwarding typed options. - Updates list serialization and adds graph-level validation.
| File | Description |
|---|---|
olive/passes/onnx/model_builder.py |
Renames forwarded INT4 options. |
test/passes/onnx/test_model_builder.py |
Tests option forwarding and graph output. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
dc1872a to
f4c5ea8
Compare
Values in --extra_mb_options go to the model builder as written, and it no longer accepts the int4_ names the examples showed.
|
/azp run Olive CI |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Keep the help text reworded on main and show the model builder's current option names in its example.
|
Merged main to fix a conflict in the |
|
kunal-vaishnavi your approval was dropped when I merged main to fix a conflict in the |

Problem
onnxruntime-genai 0.16 dropped the
int4_prefix from the model builder's quantization options and no longer accepts the old names. Olive still forwards itsint4_block_size,int4_accuracy_level,int4_is_symmetric,int4_op_types_to_quantize,int4_nodes_to_excludeandint4_algo_configsettings under the old names, so the builder ignores them without a warning, including the valuesolive optimize,olive capture-onnx-graphandolive initset.Solution
int4_*ModelBuilder options under the builder's current names (block_size,algo_config, ...); Olive's option names stay the samecheck_extra_optionscall it makes)Testing
test_model_builder_applies_int4_optionsbuilds a tiny Llama with five of the options and checks each one in the graph; it fails on mainolive capture-onnx-graph --use_model_builder --precision int4 --int4_block_size 128on Qwen2.5-0.5B-Instruct builds with block size 32 on main and 128 with this change (onnxruntime-genai 0.17.1)Refs microsoft/onnxruntime-genai#2629