Skip to content

Fix A4X-Max Reflection Beam Dynamo recipes on the current lmsysorg/sglang:beam image - #287

Merged
Chris113113 merged 3 commits into
AI-Hypercomputer:mainfrom
joeywan-google:fix-a4xmax-beam-dynamo-repro
Oct 16, 2026
Merged

Chris113113 merged 3 commits into
AI-Hypercomputer:mainfrom
joeywan-google:fix-a4xmax-beam-dynamo-repro

Conversation

@joeywan-google

@joeywan-google joeywan-google commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

I followed the A4X-Max Reflection Beam Dynamo recipes from #286 end to end on an 8-node GB300 node pool, both aggregated (beam_high_int) and disaggregated 4P4D (beam_high_throughput). As written, workers crash at startup on the current image. This PR fixes that and makes the recipes safe on shared or multi-pool clusters.

Changes

  1. Startup fix. The current lmsysorg/sglang:beam (SGLang f7473673dd, ai-dynamo 1.5.1) removed ServerArgs.get_model_config(), but dynamo/sglang/args.py still calls it (AttributeError at startup). The worker launch script patches that one call. It does nothing when the method exists, and skips with a warning if the module isn't found.
  2. Image pinned by digest in beam_high_int.yaml and beam_high_throughput.yaml. The Beam install commands take the image from the values file.
  3. dynamo.nodeSelector chart value, applied to every Dynamo pod. The Beam install commands pass --set dynamo.nodeSelector."cloud\.google\.com/gke-nodepool"=${NODE_POOL} so all workers share one NVLink domain on clusters with more than one GB300 node pool.
  4. --set dynamo.namespace=${NAMESPACE} in the install commands, with a note that fixed resource names need one namespace per concurrent release.
  5. §2.5: skip the platform install when Dynamo is already on the cluster (cluster-scoped CRDs).
  6. Disaggregated README log commands use pod names instead of nonexistent Deployments.

Testing

  • helm template with and without dynamo.nodeSelector, for both Beam values files and DeepSeek values_deepep.yaml.
  • Startup patch tested in the pinned image (patches and imports cleanly) and on a host without the module (skips, script continues).
  • Deployed both Beam recipes on 8 a4x-maxgpu-4g-metal nodes each, on Dynamo operator 1.4.2. Both became Ready and served the README §4.3 AIPerf AgentX run at CONCURRENCY=32 for 3600 s with 0 errors (agg 4,788 requests, disagg 4,750).

…ol and image digest

- Patch the one ServerArgs.get_model_config() call that ai-dynamo 1.5.1 makes
  but SGLang f7473673dd removed (guarded; no-op on images that still have it).
- Add dynamo.nodeSelector to the chart and pass the node pool in the Beam
  install commands so all workers share one NVLink domain.
- Pin lmsysorg/sglang:beam by digest and correct the Dynamo version note.
- Pass --set dynamo.namespace=${NAMESPACE}; note fixed resource names.
- Note to skip the platform install when Dynamo is already on the cluster.
- Fix disaggregated README log commands.
- Add expected AgentX results (c=32; c=96 and c=192 pending).

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request pins the SGLang runtime image to a specific digest, introduces a NODE_POOL variable and nodeSelector configuration to ensure workers share an NVLink domain, and adds a startup patch in the Helm chart to handle a breaking change in SGLang's ServerArgs. The review feedback recommends making the startup patch more robust by safely resolving the Python module spec and verifying the file exists before executing sed to prevent potential container crash loops.

@joeywan-google
joeywan-google marked this pull request as ready for review October 16, 2026 18:57
- Startup patch: resolve dynamo.sglang.args safely and skip with a warning
  when it is missing, so set -e can't crash-loop the worker.
- Take the Beam image from the values file; drop DYNAMO_IMAGE and the
  --set workload.image override from the Beam install commands.
- Remove the image/patch notes, the troubleshooting entry, and the expected
  results tables from the READMEs.
@Chris113113
Chris113113 merged commit 8f68465 into AI-Hypercomputer:main Oct 16, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants