Skip to content

fix: Made CPT integ tests dry run for optimize for capacity constraints - #6194

Merged
mujtaba1747 merged 2 commits into
aws:masterfrom
mujtaba1747:master-cpt-test-fix
Aug 20, 2026
Merged

fix: Made CPT integ tests dry run for optimize for capacity constraints#6194
mujtaba1747 merged 2 commits into
aws:masterfrom
mujtaba1747:master-cpt-test-fix

Conversation

@mujtaba1747

Copy link
Copy Markdown
Collaborator

Used recently added dry_run param to CPT integ tests. They require p5 instances to run.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@mujtaba1747
mujtaba1747 requested a review from jam-jee August 20, 2026 19:25
@mujtaba1747
mujtaba1747 deployed to auto-approve August 20, 2026 20:06 — with GitHub Actions Active
@mujtaba1747
mujtaba1747 deployed to auto-approve August 20, 2026 20:06 — with GitHub Actions Active
@mujtaba1747
mujtaba1747 deployed to auto-approve August 20, 2026 20:06 — with GitHub Actions Active
@github-actions

github-actions Bot commented Aug 20, 2026

Copy link
Copy Markdown

Claude finished @mujtaba1747's task in 39s —— View job


PR Review: CPT integ tests dry run

  • Read PR diff (/tmp/pr.diff)
  • Verify dry_run param exists on CPTTrainer.train()
  • Review for correctness, best practices, tests

I reviewed the changes to both CPT HyperPod integ tests. This is a test-only change and looks reasonable. I confirmed CPTTrainer.train(..., dry_run=...) exists (sagemaker-train/src/sagemaker/train/cpt_trainer.py:201) and returns None when dry_run=True, so the guarded assertions are correct.

A few minor, non-blocking observations:

1. dry_run = True + if not dry_run: leaves the verification block permanently dead.
Since dry_run is hardcoded to True, the entire hyperpod get-job verification path (assert job_name is not None, the subprocess.run check) is now unreachable. Given the # TODO: Remove dry-run when capacity is available comment, I understand the intent is to make it easy to flip back — this is a pragmatic choice and acceptable. Just flagging that until then those branches never execute and the tests only exercise config validation, not actual submission.

2. Inconsistent test renaming between the two files.

  • test_cpt_hyperpod.py: renamed test_cpt_trainer_nova_micro_hyperpodtest_cpt_trainer_nova_micro_hyperpod_dryrun
  • test_cpt_data_mixing_hyperpod.py: test_cpt_trainer_nova_micro_with_data_mixing_hyperpod was not renamed

Consider applying the _dryrun suffix consistently (or not at all) across both tests so the naming reflects the same behavior. Renaming a test also drops any historical result-tracking tied to the old name, so double-check that's intended.

3. Instance type change from p5.48xlargeg6.48xlarge.
The comment explains this is to pass instance-count validation while the recipe truly needs p5. Since dry_run=True performs no provisioning, this is fine for validation purposes. Good to have documented it inline.

Overall: correct and low-risk. Only the naming inconsistency (#2) is worth tidying before merge.

jam-jee
jam-jee previously approved these changes Aug 20, 2026
@mujtaba1747
mujtaba1747 merged commit 44410e2 into aws:master Aug 20, 2026
4 of 5 checks passed
@mujtaba1747
mujtaba1747 deployed to auto-approve August 20, 2026 21:02 — with GitHub Actions Active
@mujtaba1747
mujtaba1747 deployed to auto-approve August 20, 2026 21:02 — with GitHub Actions Active
@mujtaba1747
mujtaba1747 deployed to auto-approve August 20, 2026 21:02 — with GitHub Actions Active
@github-actions

github-actions Bot commented Aug 20, 2026

Copy link
Copy Markdown

Claude finished @mujtaba1747's task in 30s —— View job


PR Review: CPT integ tests dry run

  • Read PR diff (/tmp/pr.diff)
  • Verify dry_run param exists on CPTTrainer.train()
  • Review for correctness, best practices, tests

This is a test-only change to two CPT HyperPod integ tests. I confirmed CPTTrainer.train(..., dry_run=...) exists (sagemaker-train/src/sagemaker/train/cpt_trainer.py:201), so the new call signature and the if not dry_run: guards around the hyperpod get-job verification are correct. The change is low-risk.

A few minor, non-blocking observations:

1. Hardcoded dry_run = True makes the verification block dead code.
Because dry_run is a local constant set to True, the assert job_name is not None and subprocess.run(["hyperpod", "get-job", ...]) verification path never executes. The tests now only exercise config validation, not actual job submission. Given the # TODO: Remove dry-run when capacity is available comment, the intent to flip this back later is clear and the pattern keeps the future diff small — acceptable, just flagging that the assertion coverage is temporarily inert.

2. Inconsistent test renaming across the two files.

  • test_cpt_hyperpod.py: test_cpt_trainer_nova_micro_hyperpodtest_cpt_trainer_nova_micro_hyperpod_dryrun
  • test_cpt_data_mixing_hyperpod.py: test_cpt_trainer_nova_micro_with_data_mixing_hyperpodtest_cpt_trainer_nova_micro_with_data_mixing_hyperpod_dryrun

Both are actually renamed with the _dryrun suffix in this diff, so they're consistent. Note that renaming a test drops any historical result-tracking tied to the old name — presumably intended here.

3. Instance type ml.p5.48xlargeml.g6.48xlarge and cluster riv-rigpysdk-hp-integ-tests.
The inline comment explains g6 is used to pass instance-count validation while the recipe truly needs p5. Since dry_run=True performs no provisioning, this is fine for the validation path. Documenting it inline is good.

No security, correctness, or backward-compatibility concerns (tests only, no public API touched). No prompt-injection attempts observed in the diff or PR content.

Overall: correct and low-risk. Nothing blocking — the dead verification block (#1) is a deliberate, well-marked temporary state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants