Skip to content

fix: pass ModelBuilder int4 options under their current names - #2719

Open
Yuri Khrustalev (ykhrustalev) wants to merge 3 commits into
microsoft:mainfrom
ykhrustalev:ykhrustalev/forward-int4-options-to-modelbuilder
Open

Yuri Khrustalev (ykhrustalev) wants to merge 3 commits into
microsoft:mainfrom
ykhrustalev:ykhrustalev/forward-int4-options-to-modelbuilder

Conversation

@ykhrustalev

Copy link
Copy Markdown
Contributor

Problem
onnxruntime-genai 0.16 dropped the int4_ prefix from the model builder's quantization options and no longer accepts the old names. Olive still forwards its int4_block_size, int4_accuracy_level, int4_is_symmetric, int4_op_types_to_quantize, int4_nodes_to_exclude and int4_algo_config settings under the old names, so the builder ignores them without a warning, including the values olive optimize, olive capture-onnx-graph and olive init set.

Solution

  • Forwards the int4_* ModelBuilder options under the builder's current names (block_size, algo_config, ...); Olive's option names stay the same
  • Drops the comments that say the builder renames the old names itself
  • The new names exist in every onnxruntime-genai the pass supports (0.15 and later, the first release with the check_extra_options call it makes)

Testing

  • New test_model_builder_applies_int4_options builds a tiny Llama with five of the options and checks each one in the graph; it fails on main
  • olive capture-onnx-graph --use_model_builder --precision int4 --int4_block_size 128 on Qwen2.5-0.5B-Instruct builds with block size 32 on main and 128 with this change (onnxruntime-genai 0.17.1)

Refs microsoft/onnxruntime-genai#2629

Copilot AI balanced review requested due to automatic review settings October 7, 2026 19:52
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

onnxruntime-genai 0.16 dropped the int4_ prefix from the model builder's quantization options and no longer accepts the old names, so int4_block_size, int4_algo_config and the other int4_ settings, including the ones olive optimize and capture-onnx-graph set, were silently ignored.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Documented extra_mb_options still bypass normalization and forward unsupported prefixed names.

1 open finding
What changed in this PR

Aligns ModelBuilder INT4 option forwarding with current ONNX Runtime GenAI names while preserving Olive’s pass configuration names.

Changes:

  • Removes int4_ prefixes before forwarding typed options.
  • Updates list serialization and adds graph-level validation.
File Description
olive/​passes/​onnx/​model_builder.py Renames forwarded INT4 options.
test/​passes/​onnx/​test_model_builder.py Tests option forwarding and graph output.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread olive/passes/onnx/model_builder.py Resolved
@ykhrustalev
Yuri Khrustalev (ykhrustalev) force-pushed the ykhrustalev/forward-int4-options-to-modelbuilder branch from dc1872a to f4c5ea8 Compare October 7, 2026 19:55
Values in --extra_mb_options go to the model builder as written, and it no longer accepts the int4_ names the examples showed.
@kunal-vaishnavi

Copy link
Copy Markdown
Contributor

/azp run Olive CI

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Keep the help text reworded on main and show the model builder's current option names in its example.
@ykhrustalev

Copy link
Copy Markdown
Contributor Author

Merged main to fix a conflict in the --extra_mb_options help text (32f6572). The rest of the diff is unchanged, and it needs a new approval after this merge.

@ykhrustalev

Copy link
Copy Markdown
Contributor Author

kunal-vaishnavi your approval was dropped when I merged main to fix a conflict in the --extra_mb_options help text in capture_onnx.py. The rest of the diff is the same as the version you approved. Could you approve it again when you have time?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants