[Rollout] Add opt-in group-scoped session affinity

[chengcuiping]

Summary

• Add an opt-in group scope for automatically assigned rollout session IDs.
• Preserve the existing per-sample behavior by default.
• Reuse the existing X-SMG-Routing-Key path for consistent-hashing routing.

Motivation

• Slime already preserves session affinity across turns of one sample.
• Grouped sibling rollouts may also need a shared routing identity.
• Previously this required callers to pre-populate the same session_id manually.

Behavior

• sample: unchanged default, one generated ID per sample.
• group: one generated or inherited ID per rollout group.
• conflicting explicit IDs fail closed.
• group scope requires consistent_hashing.
• non-consistent-hashing routing remains unchanged.

Testing

CPU-only qualification was run against the latest upstream main after cherry-picking this PR.

Additional production-path coverage was added for:

• generate_and_rm_group -> generate -> HTTP post
• verifying that 3 sibling requests from the same group carry the same X-SMG-Routing-Key
• preserving the default per-sample routing behavior for non-grouped generation

Commands:

CUDA_VISIBLE_DEVICES='' python -m compileall -q slime/rollout/sglang_rollout.py slime/utils/arguments.py tests/test_group_session_affinity.py tests/test_megatron_argument_validation.py
CUDA_VISIBLE_DEVICES='' python -m pytest tests/test_group_session_affinity.py tests/test_sample.py tests/test_megatron_argument_validation.py -m unit
CUDA_VISIBLE_DEVICES='' pre-commit run --files slime/utils/arguments.py slime/rollout/sglang_rollout.py tests/test_megatron_argument_validation.py tests/test_group_session_affinity.py docs/en/advanced/sglang-config.md docs/zh/advanced/sglang-config.md

Qualification result: 31 tests passed. git diff --check passed. Testing was CPU-only; no GPU was used.

Limitations

• This controls routing identity only.
• It does not implement KV pinning, eviction, transfer, TTL, or cache management.
• It does not guarantee a cache hit or claim a throughput or other performance improvement.
• Group affinity may reduce cross-worker load-balancing flexibility.

Related work

Related to #1469.
Builds on the consistent-hashing routing path introduced by #1588.

[chengcuiping]

Hi @yitianlian — I ran a preliminary 2×A100 A/B benchmark and a CPU-only paired follow-up analysis to understand the cache-locality versus load-balancing trade-off of this PR.

The follow-up reconstructed 20 trace-matched A/B pairs across four cells:

• All four cell-level samples/s point estimates favored group scope. The stratified aggregate geometric-mean ratio was 1.191 with a 95% paired-bootstrap CI of 1.116–1.269.
• Group scope reduced actual prefill tokens by 30.00%–47.61%, and all four p95 group-completion point estimates improved.
• However, both 8192-token cell-specific confidence intervals still crossed 1, and the benchmark runtime was not a clean pinned dependency closure.

The variance appears closely related to having only four group routing keys. The observed 2:2 / 3:1 / 4:0 placements were 8 / 10 / 2, nearly matching the theoretical 37.5% / 50% / 12.5% distribution. The paired throughput ratios were:

• balanced 2:2: 1.357, with group scope winning 8/8 pairs;
• 3:1: 1.149, winning 10/10 pairs;
• 4:0: 0.848, with both pairs regressing.

The association between group-placement imbalance and paired throughput effect was negative (Spearman ρ = -0.691; 200,000-permutation p = 0.00101). This is exploratory association rather than causal proof, especially because there were only two 4:0 observations.

Before spending more GPU time, could you advise whether an opt-in framework-level group session scope fits Slime’s intended direction?

If the scope is acceptable, I can run a confirmation benchmark on a clean pinned SM80 runtime with 16/32 concurrent groups, paired traces, and preregistered paired confidence intervals. If maintainers prefer callers to assign a shared Sample.session_id explicitly, I am also happy to simplify or close the PR.