Skip to content

refactor(tests): migrate generic test fixtures off sunset models onto retained Tier 1/2 configs (Phase 1b-1) - #6320

Open
entrpn wants to merge 1 commit into
mainfrom
jfacevedo/model-deprecation-pr1b-fixtures
Open

entrpn wants to merge 1 commit into
mainfrom
jfacevedo/model-deprecation-pr1b-fixtures

Conversation

@entrpn

@entrpn entrpn commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Description

This is PR 1b-1 of the phased MaxText Model Deprecation & Lifecycle Policy rollout (split from the original 60-file PR into two focused, independent PRs so Python test fixture changes can be reviewed separately from auto-generated sharding JSON goldens):

  • PR 1a (feat(configs): add models/deprecated/ fallback resolution and Depreca… #6171, merged): Added configs/models/deprecated/<model_name>.yml fallback resolution and DeprecationWarning in pyconfig and pyconfig_deprecated.
  • PR 1b-1 (this PR — 22 Python test files, 0 JSON files): Migrates generic unit, integration, post-training, and MoE test fixtures off sunset models (gpt3-52k, gemma-7b, gemma2-2b, llama2-7b, llama3-8b, mixtral-8x7b/8x22b, deepseek2-16b) onto retained Tier 1/2 configs (qwen3-0.6b, gemma3-4b, qwen3-30b-a3b, qwen3-235b-a22b, deepseek3-test, llama3.1-8b, llama3.3-70b), while keeping llama4, llama2.py (LlamaDecoderLayer), live decoder_block unit checks (decoder_layer_model_mode_test.py), pre-generated GCS checkpoint tests (decode_tests.py, generate_param_only_checkpoint_test.py), and locked GitHub Actions HLO reference baselines (hlo_diff_test.py) untouched.
  • PR 1b-2 (refactor(tests): migrate sharding_dump golden baselines from deepseek2-16b to deepseek3-test (Phase 1b-2) #6367 — independent companion PR): Migrates tests/utils/sharding_dump.py (9 MoE sharding cases) and tests/utils/run_sharding_dump.py (11 modified lines of Python total) plus the 36 auto-generated tests/utils/sharding_info/ JSON golden files from deepseek2-16b to deepseek3-test.
  • PR 1c (next): Soft-sunset git mv of the sunset model configs into src/maxtext/configs/models/deprecated/ + docs/reference/models/tiering.md sync so generic unit/integration fixtures do not emit spurious DeprecationWarnings.
  • PR 1d: Automated Model Lifecycle Audit CLI (src/maxtext/tools/model_lifecycle_audit.py).

Summary of Fixture Migrations (22 Python Files)

  1. Dense / Decoder / Optimizer / FLOPs / Config / Multimodal Unit & Integration Tests:
    • gpt3-52k -> qwen3-0.6b in tests/unit/aqt_serve_roundtrip_nnx_test.py, and model_name="gpt3-52k" -> decoder_block=DecoderBlockType.GPT3 in tests/unit/nnx_decoders_test.py.
    • gemma-7b / gemma2-2b -> gemma3-4b or qwen3-0.6b in tests/unit/pyconfig_test.py, tests/unit/pyconfig_deprecated_test.py, tests/integration/diloco_test.py, tests/inference/benchmark_offline_engine.py, and tests/post_training/integration/grpo_trainer_correctness_test.py.
    • llama2-7b / llama3-8b -> llama3.1-8b / llama3.3-70b in tests/unit/flop_calculation_test.py (retaining llama4-17b-16e), tests/unit/optimizers_test.py, tests/unit/muon_utils_test.py, tests/unit/configs_value_test.py, tests/unit/nnx_decoders_test.py, tests/unit/managed_mldiagnostics_test.py, tests/unit/rl_profiling_config_test.py, tests/unit/multimodal_utils_test.py, tests/post_training/unit/lora_utils_test.py, and tests/post_training/unit/rl_lr_schedule_test.py.
  2. MoE Unit & Equivalence Test Fixtures:
    • mixtral-8x7b / mixtral-8x22b -> qwen3-30b-a3b / qwen3-235b-a22b (preserving num_experts=8, num_experts_per_tok=2, and norm_topk_prob=False where tested) across tests/unit/moe_test.py, tests/unit/router_replay_test.py, tests/unit/pyconfig_test.py, tests/unit/defer_small_all_reduces_test.py, tests/unit/flop_calculation_test.py, and tests/equiv_chunk_test.py.
    • deepseek2-16b -> deepseek3-test across tests/unit/deepseek_decode_consistency_test.py, tests/unit/decoder_layer_model_mode_test.py, tests/unit/flop_calculation_test.py (also retaining explicit q_lora_rank=0 MLA FLOPs coverage), and tests/unit/optimizers_test.py (also retaining explicit q_lora_rank=0 Muon dimension-number coverage).

Tests

  • CPU unit tests across all 18 migrated unit suites (pyconfig_test, pyconfig_deprecated_test, configs_value_test, flop_calculation_test, muon_utils_test, multimodal_utils_test, managed_mldiagnostics_test, rl_profiling_config_test, deepseek_decode_consistency_test, decoder_layer_model_mode_test, nnx_decoders_test, aqt_serve_roundtrip_nnx_test, defer_small_all_reduces_test, router_replay_test, optimizers_test, moe_test, lora_utils_test, and rl_lr_schedule_test), re-verified after rebasing onto main.
  • TPU AOT compilation test (libtpu): tests/integration/diloco_test.py -k two_slices (4/4 passed).
  • Pre-commit linters (pyink --check, pylint, codespell): passed across all 22 modified Python files.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

@github-actions

Copy link
Copy Markdown
Contributor

🤖 Hi @entrpn, I'm running an automatic MaxText review of this pull request. You can track my progress in the logs. To request another review later, add the gemini-review label or comment @gemini-cli /review.

@gemini-code-assist

Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This PR is part of the MaxText Model Deprecation & Lifecycle Policy, focusing on migrating test suites to use currently supported models. This ensures that test coverage remains relevant while phasing out sunset models, without impacting critical reference baselines.

Highlights

  • Test Fixture Migration: Migrated generic unit, integration, and post-training test fixtures from deprecated models to active Tier 1/2 models.
  • MoE & Sharding Updates: Updated MoE and sharding golden fixtures, including regenerating goldens for deepseek3-test.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@github-actions

Copy link
Copy Markdown
Contributor

🤖 I'm sorry @entrpn, but I was unable to process your request. Please see the logs for more details.

@entrpn entrpn removed the skip-ci-tests Add this label to the PR to skip running CI tests label Nov 13, 2026
@codecov

codecov Bot commented Nov 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@entrpn
entrpn force-pushed the jfacevedo/model-deprecation-pr1b-fixtures branch from 85eec3e to 01ec3af Compare November 16, 2026 18:47
@entrpn entrpn changed the title refactor(tests): migrate generic test fixtures off sunset models onto retained Tier 1/2 configs (Phase 1b) refactor(tests): migrate generic test fixtures off sunset models onto retained Tier 1/2 configs (Phase 1b-1) Nov 16, 2026
@entrpn entrpn self-assigned this Nov 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🤖 Hi @entrpn, I've received your request, and I'm working on it now! You can track my progress in the logs for more details.

@entrpn
entrpn force-pushed the jfacevedo/model-deprecation-pr1b-fixtures branch from 01ec3af to 6df5ba4 Compare November 17, 2026 03:34
… retained Tier 1/2 configs (Phase 1b-1)

Migrate generic unit, integration, and post-training test fixtures off
sunset models (gpt3-52k, gemma-7b, gemma2-2b, llama2-7b, llama3-8b,
mixtral-8x7b/8x22b, deepseek2-16b) onto retained Tier 1/2 configs
(qwen3-0.6b, gemma3-4b, qwen3-30b-a3b, qwen3-235b-a22b, deepseek3-test,
llama3.1-8b, llama3.3-70b), while preserving:
- num_experts=8, num_experts_per_tok=2, and norm_topk_prob=False in
  RoutedMoE / fused_moe / router_replay unit tests
- explicit q_lora_rank=0 MLA FLOPs and Muon dimension-number coverage
- live decoder_block unit checks, llama2.LlamaDecoderLayer, llama4,
  pre-generated GCS checkpoint tests, and HLO diff baselines
@entrpn
entrpn force-pushed the jfacevedo/model-deprecation-pr1b-fixtures branch from 6df5ba4 to 0137c99 Compare November 17, 2026 17:10

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants