All pull requests

Pull requests

• Preserve completed CUDA prefill across cancellation and retry

  #1228·Sapphire-Rapids opened Oct 24, 2026·7 of 9 tasks·

  ·
  1

• Docs: GLM-5.3-Flash on CUDA, a native page for its ranks, linked from the pages that said it had no CUDA backend

  #1224·Bizuayeu opened Oct 24, 2026Contributor·9 tasks done·

  ·

• donation: lend and borrow inference through a card, and reach a lent machine through a relay

  #1223·zweck opened Oct 23, 2026Contributor·

  ·

• qwen CUDA: attention partials with rows fastest in the grid, so a window's rows share each chunk's K/V in L2

  #1219·jschmied opened Oct 23, 2026Contributor·

  ·

• qwen27: TF_Q27_PROFILE logs the round's class GPU times, and replies carry gpu_seconds

  #1215·grantoverton opened Oct 23, 2026Contributor·

  ·

• glm/cuda: the engine's own sequence on KV pages too, one window of them planned first (stacked on #1163)

  #1211·Bizuayeu opened Oct 23, 2026Contributor·9 tasks done·

  ·
  1

• deepseek_v41: four-rank tensor parallel (whole-block shares, padded prefill heads)

  #1206·ashhart opened Oct 22, 2026Owner·

  ·
  1

  3

• Flash Next CUDA: the PLE gather keeps a parked worker pool instead of spawning threads per call

  #1195·jschmied opened Oct 22, 2026Contributor·

  ·

• lanes: a queued decision takes a free slot beside the live streams, as a new stream's prompt pass would

  #1192·grantoverton opened Oct 22, 2026Contributor·

  ·

• qwen27: a JointHead side-head scores options on post-norm hidden rows at their final-token positions

  #1191·grantoverton opened Oct 22, 2026Contributor·

  ·

• qwen27: score and decide passes queued onto the lane host's idle engine thread

  #1189·grantoverton opened Oct 22, 2026Contributor·

  ·

• lanes: prompt steps for the 27B and behind --drafter, and a share of wall time for prompt work

  #1188·jschmied opened Oct 22, 2026Contributor·

  ·
  1

• cuda gemma4: images on the 31B, the image tower (#1048 step 12)

  #1186·jschmied opened Oct 22, 2026Contributor·

  ·

• cuda gemma4: the assistant's hold on the device, as CUDA graphs (+1.7 % on code, the rest within noise)

  #1185·jschmied opened Oct 22, 2026Contributor·

  ·

• cuda gemma4: z-lab's DFlash drafter on the 31B (#1048 step 11)

  #1184·jschmied opened Oct 22, 2026Contributor·

  ·

• cuda gemma4: images on the 12B, and LayerNorm rows to 8192 wide (#1048 step 12)

  #1183·jschmied opened Oct 22, 2026Contributor·

  ·

• cuda gemma4: every MLX affine width the family admits, and the 31B on CUDA (#1048 step 9)

  #1182·jschmied opened Oct 22, 2026Contributor·

  ·

• cuda gemma4: the 12B's MTP assistant as the drafter (#1048 step 10)

  #1181·jschmied opened Oct 22, 2026Contributor·

  ·

• glm cuda: rank 0's NCCL id goes over the ranks' lines, so a /32 ring's followers bootstrap on NCCL_SOCKET_IFNAME

  #1178·Bizuayeu opened Oct 22, 2026Contributor·9 tasks done·

  ·
  1

• flint: serve Recursive Flint (40B-A2.5B MoE) natively on Metal

  #1177·BobClawblaw opened Oct 21, 2026Contributor·

  ·
  1

• qwen CUDA: --prompt-precision int8, prompt matmuls on int8 activations (cold prefill 20-27% shorter)

  #1176·BobClawblaw opened Oct 21, 2026Contributor·

  ·
  1

• cuda gemma4: the dense family on the shared operators (text, MLX affine 4-bit), decode under CUDA graphs

  #1174·jschmied opened Oct 21, 2026Contributor·9 tasks done·

  ·

• server: --idle-unload closes the engine after idle seconds and the next request reloads it

  #1172·heitke opened Oct 21, 2026Contributor·8 tasks done·

  ·
  2

• cuda qmmf: half tiles for decode calls whose 64-column grid leaves SMs idle

  #1171·jschmied opened Oct 21, 2026Contributor·8 tasks done·

  ·
  1

• Flash Next CUDA: the streams beside the first take KV pages from the core pool

  #1170·jschmied opened Oct 21, 2026Contributor·8 tasks done·

  ·
  2