Repository navigation
feat(m3): support forced_routed_experts router replay in m3/core/moe.py - #6408
AusarYao28 wants to merge 1 commit into
Conversation
- Thread optional forced_routed_experts (shape [..., num_experts_per_tok], UNSET_ROUTED_EXPERT = -1 for padded/inactive slots) through route_top_k, apply_forced_routed_experts, MoEExecutor, routed_experts, routed_experts_fused, and grouped_unsharded_fused. - Preserve 100% identical HLO when forced_routed_experts is None. - Validate shape, integer dtype, and concrete ID range [-1, num_experts) failing closed on invalid shapes/dtypes/IDs, while clamping -1 slots before take_along_axis and zeroing inactive slot weights and denominators so -1 slots contribute exact 0.0 to outputs, group_sizes, and gradients. - Partition forced_routed_experts along (batch, length) inside shard_map across EP == 1 and EP > 1 (ragged_all_to_all + dropless overflow buffer). - Add 30 4-CPU unit tests in tests/unit/m3/moe_router_replay_test.py covering zero-overhead HLO parity, exact 0.0 fwd+bwd equivalence on natural top_k indices, non-natural forced overrides, -1 partial and all--1 row padding, and fail-closed validation.
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a mechanism for forced expert routing in Mixture-of-Experts (MoE) models. By allowing users to provide explicit expert indices, it facilitates advanced debugging and reinforcement learning workflows while maintaining full compatibility with existing training pipelines. The implementation ensures that forced routing remains deterministic and preserves performance characteristics, including zero-overhead parity when the feature is unused. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
🤖 Hi @AusarYao28, I'm running an automatic MaxText review of this pull request. You can track my progress in the logs. To request another review later, add the |
|
🤖 I'm sorry @AusarYao28, but I was unable to process your request. Please see the logs for more details. |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Description
Depends on #6281 (stacked on
yaoyuchen/m3-moe-dropless-overflow-buffer; review the top commit only).Adds
forced_routed_expertsrouter replay support tosrc/maxtext/m3/core/moe.pyfor deterministic MoE debugging, distillation, and RL/on-policy replay without altering default training HLO:forced_routed_experts(shape[..., num_experts_per_tok],UNSET_ROUTED_EXPERT = -1for padded/inactive slots) throughroute_top_k,apply_forced_routed_experts,MoEExecutor,routed_experts,routed_experts_fused, andgrouped_unsharded_fused.forced_routed_experts is None.[-1, num_experts), while clamping-1slots beforetake_along_axisand zeroing inactive slot weights and denominators so-1slots contribute exact0.0to outputs,group_sizes, and gradients.forced_routed_expertsalong(batch, length)insideshard_mapacross bothEP == 1andEP > 1(ragged_all_to_all+ dropless overflow buffer).Tests
tests/unit/m3/moe_router_replay_test.py+tests/unit/m3/moe_test.py -k "router_replay or dropless": 40 passed (30 router-replay tests + 10 dropless tests; covers zero-overhead HLO parity, exact0.0forward/backward equivalence on natural top-k indices, non-natural forced overrides,-1partial and all--1row padding, and fail-closed validation).pyink,pylint(10.00/10),codespell, and m3 contract audits clean.Checklist
Before submitting this PR, please make sure (put X in square brackets):
gemini-reviewlabel.