Skip to content

Add A4X Max SGLang N3U KV/RR benchmarking recipes - #291

Closed
alisachen-google wants to merge 1 commit into
AI-Hypercomputer:mainfrom
alisachen-google:add-a4xmax-dynamo-kv-rr-benchmark
Closed

alisachen-google wants to merge 1 commit into
AI-Hypercomputer:mainfrom
alisachen-google:add-a4xmax-dynamo-kv-rr-benchmark

Conversation

@alisachen-google

Copy link
Copy Markdown
Contributor

Place architecture-specific N3U recipes and deploy/sweep entry points under aggregated-serving/dynamo/sglang/n3u-kv-vs-rr and
disaggregated-serving/dynamo/sglang/n3u-kv-vs-rr. Put the shared comparison, prerequisites, dataset replay, and quickstart guide in the disaggregated Dynamo SGLang README.

Package the measured 24-GPU agg and 64-GPU D88 configurations, performance curves, and same-concurrency and SLO tables. Share the deployment and sweep helpers, image builds, and validated artifacts from the disagg N3U folder. Keep sweeping focused on router settings, concurrency, and AgentX results.

Validation: eight offline tests, both entry points from outside the recipe directory, shell syntax, Python lint, local links, and data provenance hashes. The benchmark logic and measured data are preserved; container builds and GKE execution remain untested.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces benchmarking recipes and deployment scripts for Nemotron-3-Ultra-550B-A55B-NVFP4 on A4X Max GPUs using NVIDIA Dynamo, supporting both aggregated and disaggregated configurations. The changes include configuration files, deployment and sweep scripts, and testing utilities. Review feedback suggests simplifying redundant variable initialization in deploy.py and improving robustness in sweep.py by using safer dictionary access methods to avoid potential KeyErrors.

Place architecture-specific N3U recipes and deploy/sweep entry points under
aggregated-serving/dynamo/sglang/n3u-kv-vs-rr and
disaggregated-serving/dynamo/sglang/n3u-kv-vs-rr. Put the shared comparison,
prerequisites, dataset replay, and quickstart guide in the disaggregated
Dynamo SGLang README.

Package the measured 24-GPU agg and 64-GPU D88 configurations, performance
curves, and same-concurrency and SLO tables. Share the deployment and sweep
helpers, image builds, and validated artifacts from the disagg N3U folder.
Keep sweeping focused on router settings, concurrency, and AgentX results.

Validation: eight offline tests, both entry points from outside the recipe
directory, shell syntax, Python lint, local links, and data provenance hashes.
The benchmark logic and measured data are preserved; container builds and
GKE execution remain untested.

Review fixes folded in: deploy.py builds Kube without a redundant None
assignment, and sweep.py reads the worker GPU limit with .get() so a pod
without one raises the intended "must request four GPUs" error.
@alisachen-google
alisachen-google force-pushed the add-a4xmax-dynamo-kv-rr-benchmark branch from 1baf8c6 to 5ac67e6 Compare October 25, 2026 22:42
@alisachen-google
alisachen-google deleted the add-a4xmax-dynamo-kv-rr-benchmark branch October 25, 2026 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant