Repository navigation
Add A4X-MAX Kimi-K3.1-Pro 3P1D llm-d 3P1D recipe - #292
Merged
joeywan-google merged 2 commits intoNov 10, 2026
Merged
Conversation
There was a problem hiding this comment.
Code Review
This pull request adds a Helm chart and configuration to serve the Kimi-K3.1-Pro model on GKE A4X-MAX using vLLM and llm-d prefill/decode disaggregation. The feedback highlights several critical issues: security risks from running containers with privileged access, potential worker node crashes due to missing tokenizer dependencies, the anti-pattern of installing pip packages at runtime, an excessively long timeout in the client's warmup loop, and a lack of explicit CPU and memory resource definitions for the vLLM containers.
AnishShah
added a commit
to AnishShah/gpu-recipes
that referenced
this pull request
Nov 6, 2026
AnishShah
force-pushed
the
kimi-k3-1-pro-llmd-3p1d
branch
from
November 6, 2026 20:50
3b6989d to
37cd92e
Compare
… llm-d recipe - Sanitize internal client image to docker.io/library/python:3.12 and install aiperf@1dfaea5f at Job startup when not pre-installed (matching sibling A4X-MAX AgentX bench_client.yaml convention), plus enable AIPERF_HTTP_X_SESSION_ID_FROM_CORRELATION_ID for session affinity. - Explicitly pass --block-size=2048 on TP=8 prefill engines and document how Kimi-K3.1-Pro's hybrid MLA + KDA cache alignment matches blockSizeTokens: 2048 in files/epp-config.yaml. - Update README troubleshooting entry for tokenizer startup to match the current single-process trust_remote_code prewarm in engines.yaml.
joeywan-google
approved these changes
Nov 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.