[void-mckenzie]

Serves the GLM-5.3-Flash vision tower from a llama.cpp glm5v mmproj on the #803 shared base: gguf_mm_config and iter_mmproj_weights in glm5_next/gguf.py, and the GGUF registry row gains encoders / mm_processor and switches to the ConditionalGeneration class. Closes the glm5v item of #794 (sub-issue of #671).

The tower config comes from the mmproj metadata: geometry from the clip.vision.* keys, swiglu_limit from clip.vision.swiglu_clamp, image_token_id looked up in the text GGUF's tokenizer (<|image|>), and the processor's token budgets from llama.cpp's single-frame pixel budgets divided by (patch * merge)^2, refusing a non-whole result. Glm5NextImageProcessor builds from the derived preprocessor config, so the multimodal path above is the HF checkpoint's own. The only tensor transform is stacking the Conv3d patch embedding's two temporal halves back; everything else is HF layout (the #794 survey measured the set bit-exact vs HF; here the reader matches the constructed tower 347/347 in names and shapes). A gelu tower (clip.use_silu not true) and a mmproj missing its second patch half refuse.

gguf_check sweep

fetch_headers --search GLM-5.3-Flash (2,145 shards, 107 repos), plus a header stub of every *mmproj*.gguf in those repos (72/73; one 403-gated), kernels.json probed on the GPU. 270/792 files of served architectures pass.

arch	pass
glm5-next	150/208
glm5next	120/479

Every failure is a #721 ignore class or a clean refusal; nothing new: per-layer streaming exports, ik_llama / custom ggml types, mixed gate/up expert banks, IQ1_M kernel gaps (18, first sweep here with a kernel table), missing head_count_kv / swiglu_clamp_exp refusals, MTP drafts, broken uploads.

Vision: 25 shard<->mmproj pairs exercised (11 auto-discovered in-folder, 14 forced with FREETOKEN_MMPROJ for the subfolder layouts), all glm5v, BF16 / F16 / Q8_0 alike - tower config, weights, image processor and chat-template placeholder, 0 problems.

Not covered

item	why
glm5next projector spelling (18 files, 17 repos)	llama.cpp aborts on it too (PROJECTOR_TYPE_UNKNOWN, checked on dd002b91f: the names table has glm5v only); refused under --mmproj, skipped with a warning by discovery
arch glm5-next-vision (2 files)	standalone tower GGUFs, not clip mmprojs
orcarouter mmproj	403-gated repo, not fetchable
subfolder-layout repos	discovery reads the model file's own folder only (#803 behavior); --mmproj picks the file

Real-file serving

unsloth GLM-5.3-Flash-UD-IQ4_XS (5 shards) + the sibling mmproj-F16.gguf, auto-discovered: projector=glm5v type=F16 -> bf16, 348 tensors, 1.05 GiB.
RTX 5090 32 GB (driver 580.95.05), Ryzen 9 7950X3D, 187 GB RAM, python 3.10.19, torch 2.14.1+cu130, triton 3.8.0, transformers 5.18.0.

Test	Result
Image description (shapes, colors, printed text)	correct; the caption text quoted verbatim
Image prompt	1071 tokens (1035 soft); a repeat request hits 1024 cached prefix tokens (radix cache through the image span)
Decode, bs=1 steady	14.4-16.7 tok/s, expert-fetch bound (same as the text-only #721 run; the tower costs nothing at decode)
Prefill, 25k-token text prompt	1.6k-2.6k tok/s per 8192 chunk as slots warm (matches #721)

ft serve --gpu 0 --model-path <gguf> --max-seq-len-override 40000 --moe-cache-mem 6GiB --memory-ratio 0.82

Tests

No per-family GGUF test file, per the #721 review direction; tests/models/test_models_registry.py covers the row, and the glm5_next family tests pass (18).

[jason-fxz]

LGTM. Checked on real files:

• unsloth mmproj-BF16.gguf is bit-exact against the HF safetensors, 347/347 tower tensors after the patch-embedding stack. F16 differs only in f16 subnormals, max rel 5e-8.
• The derived vision_config, image_token_id and processor budgets (16 / 8000 tokens) match the HF config. The image processor built from the derived config gives pixel values and grids identical to HF AutoImageProcessor on 11 images, from 20x20 to 8000x8000.
• Served unsloth UD-IQ4_XS next to RedHatAI GLM-5.3-Flash-NVFP4 (HF checkpoint) on 10 probes: text, photos, OCR, shapes, two images, 48x48, 2400x240, a 5184x3456 photo at the 8000-token cap, and /v1/messages. Prompt token counts are identical on every request, and the answers match. Same with ggml-org's Q8_0 mmproj via --mmproj. Auto-discovery picks BF16 over F16 and logs the skipped file. --mm-disable vision serves text and rejects images.