Skip to content

Preserve native GenAI embedding construction - #2795

Open
Baiju Meswani (baijumeswani) wants to merge 1 commit into
microsoft:mainfrom
baijumeswani:baijumeswani/ling-native-components
Open

Baiju Meswani (baijumeswani) wants to merge 1 commit into
microsoft:mainfrom
baijumeswani:baijumeswani/ling-native-components

Conversation

@baijumeswani

Copy link
Copy Markdown

Describe your changes

Preserve native GenAI embedding construction for ordinary source tensors. Olive's ModelBuilder patch now delegates those tensors to the original builder while keeping the legacy Olive-quantized path.

The wrapper is idempotent, so repeated passes do not accumulate patches. This preserves native INT8 embedding selection and component metadata without adding model-specific behavior.

Validation: all 29 Model Builder pass tests and configured lintrunner checks, including Pylint, pass. A real reduced Ling component export also preserves INT8 CPU embedding storage and the native rank-two lookup interface.

Checklist before requesting a review

  • Add unit tests for this change.
  • Make sure all tests can pass.
  • Update documents if necessary. No documentation change is needed.
  • Lint and apply fixes to your code by running lintrunner -a. Configured lint passes without remaining fixes.
  • Is this a user-facing change? Release note: ModelBuilder preserves native GenAI embedding export for ordinary source tensors.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a32500b6-be93-4206-a257-11406b00a3f3
Copilot AI balanced review requested due to automatic review settings November 7, 2026 00:40
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block approval of the embedding-routing change.

No open issues 🎉

What changed in this PR

Preserves native GenAI embedding construction in Olive’s ModelBuilder for ordinary tensors while retaining the existing Olive-quantized embedding path.

Changes:

  • Delegates ordinary tensors to the original GenAI builder and routes tensors with qweight through Olive’s implementation.
  • Makes the embedding wrapper idempotent so repeated patch calls do not stack wrappers.
  • Extends unit coverage for both routing paths and repeated patching.
File Description
test/​passes/​onnx/​test_model_builder.py Verifies native and quantized embedding routing and patch idempotency.
olive/​passes/​onnx/​model_builder.py Adds an idempotent wrapper that preserves native embedding construction.

Review effort: Balanced

@wraps(native_make_embedding)
def make_embedding(model, embedding):
if hasattr(embedding, "qweight"):
return patched_make_embedding(model, embedding)
@ykhrustalev

Copy link
Copy Markdown
Contributor

#2692 makes the same make_embedding change and adds two things. First, a guard that keeps a separate embedding table where onnxruntime-genai 0.17.1 reads a tied LM head back wrong: with the default algorithm, is_symmetric=false and matmul_mixed_precision=last_matmul:int8 builds fail to load, and matmulnbits_weights_prepacked=1 on CUDA generates garbage (Qwen2.5-0.5B-Instruct). Second, an end-to-end test that builds a tied tiny Llama. Once this PR merges, I can cut #2692 down to the guard and the test, or close it if Olive will require a newer onnxruntime-genai. Which do you prefer?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants