Skip to content

Add A4X-MAX Kimi-K3.1-Pro 3P1D llm-d 3P1D recipe - #292

Merged
joeywan-google merged 2 commits into
AI-Hypercomputer:mainfrom
AnishShah:kimi-k3-1-pro-llmd-3p1d
Nov 10, 2026
Merged

joeywan-google merged 2 commits into
AI-Hypercomputer:mainfrom
AnishShah:kimi-k3-1-pro-llmd-3p1d

Conversation

@AnishShah

Copy link
Copy Markdown
Contributor

No description provided.

@AnishShah AnishShah changed the title Add A4X-MAX Kimi-K3.1-Pro 3P1D llm-d Mooncake RDMA vLLM recipe Add A4X-MAX Kimi-K3.1-Pro 3P1D llm-d 3P1D recipe Nov 6, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds a Helm chart and configuration to serve the Kimi-K3.1-Pro model on GKE A4X-MAX using vLLM and llm-d prefill/decode disaggregation. The feedback highlights several critical issues: security risks from running containers with privileged access, potential worker node crashes due to missing tokenizer dependencies, the anti-pattern of installing pip packages at runtime, an excessively long timeout in the client's warmup loop, and a lack of explicit CPU and memory resource definitions for the vLLM containers.

AnishShah added a commit to AnishShah/gpu-recipes that referenced this pull request Nov 6, 2026
@AnishShah
AnishShah force-pushed the kimi-k3-1-pro-llmd-3p1d branch from 3b6989d to 37cd92e Compare November 6, 2026 20:50
… llm-d recipe

- Sanitize internal client image to docker.io/library/python:3.12 and
  install aiperf@1dfaea5f at Job startup when not pre-installed
  (matching sibling A4X-MAX AgentX bench_client.yaml convention), plus
  enable AIPERF_HTTP_X_SESSION_ID_FROM_CORRELATION_ID for session affinity.
- Explicitly pass --block-size=2048 on TP=8 prefill engines and document
  how Kimi-K3.1-Pro's hybrid MLA + KDA cache alignment matches
  blockSizeTokens: 2048 in files/epp-config.yaml.
- Update README troubleshooting entry for tokenizer startup to match the
  current single-process trust_remote_code prewarm in engines.yaml.
@joeywan-google
joeywan-google merged commit a2ca532 into AI-Hypercomputer:main Nov 10, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants