Repository navigation
Preserve native GenAI embedding construction - #2795
Baiju Meswani (baijumeswani) wants to merge 1 commit into
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a32500b6-be93-4206-a257-11406b00a3f3
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues block approval of the embedding-routing change.
No open issues 🎉
What changed in this PR
Preserves native GenAI embedding construction in Olive’s ModelBuilder for ordinary tensors while retaining the existing Olive-quantized embedding path.
Changes:
- Delegates ordinary tensors to the original GenAI builder and routes tensors with
qweightthrough Olive’s implementation. - Makes the embedding wrapper idempotent so repeated patch calls do not stack wrappers.
- Extends unit coverage for both routing paths and repeated patching.
| File | Description |
|---|---|
test/passes/onnx/test_model_builder.py |
Verifies native and quantized embedding routing and patch idempotency. |
olive/passes/onnx/model_builder.py |
Adds an idempotent wrapper that preserves native embedding construction. |
Review effort: Balanced
| @wraps(native_make_embedding) | ||
| def make_embedding(model, embedding): | ||
| if hasattr(embedding, "qweight"): | ||
| return patched_make_embedding(model, embedding) |
|
#2692 makes the same |
Describe your changes
Preserve native GenAI embedding construction for ordinary source tensors. Olive's ModelBuilder patch now delegates those tensors to the original builder while keeping the legacy Olive-quantized path.
The wrapper is idempotent, so repeated passes do not accumulate patches. This preserves native INT8 embedding selection and component metadata without adding model-specific behavior.
Validation: all 29 Model Builder pass tests and configured lintrunner checks, including Pylint, pass. A real reduced Ling component export also preserves INT8 CPU embedding storage and the native rank-two lookup interface.
Checklist before requesting a review
lintrunner -a. Configured lint passes without remaining fixes.