[jason-fxz]

Part of #671 (the MiniMax-M3 item in #794). Serves the MiniMax-M3 vision tower from a llama.cpp minimax_m3 mmproj on the #803 base. MiniMax-M3.1-Flash GGUFs share the arch and projector and load through the same code.

What it does

• gguf_mm_config derives the tower config from the mmproj metadata and tensor shapes; image_token_id is ]<]image[>[ in the text GGUF's tokenizer. Rope theta 10000 (llama.cpp hardcodes it too), min / max pixels 3136 / 451584 (what MiniMaxM3VLImageProcessor uses in place of the stale [672, 672] in the HF preprocessor_config.json), gelu and temporal patch 2 are family constants equal to the HF release.
• iter_mmproj_weights stacks the Conv3d patch embedding's two temporal halves back and undoes the converter's per-head q / k row order ([t t | h h | w w | pad] back to HF's [t h w | t h w | pad], head_dim 80); the rest is HF layout.
• The GGUF registry row gains encoders / mm_processor and switches to the ConditionalGeneration class.
• IMAGE_START_ID / IMAGE_END_ID move from the processor to minimax_m3/config.py, which both the processor and gguf.py import (the DeepSeek-V4 processor reads its image id the same way).
• Refusals (an error under --mmproj, a logged skip under auto-discovery): another projector, an output width other than the text model's, no ]<]image[>[ token, ]<]start of image[>[ / ]<]end of image[>[ not at the ids the processor uses (200029 / 200030), and a tensor set other than the one llama.cpp loads (Serpen's mm.merge.* files, which llama.cpp cannot load either).

Tower parity

mmproj	HF checkpoint	result
bartowski BF16, AesSedai F32	MiniMaxAI/MiniMax-M3	395/395 tensors bit-exact, tower output bit-exact on 8 images
unsloth M3.1-Flash BF16	MiniMaxAI/MiniMax-M3.1-Flash	395/395 bit-exact
bartowski F16	MiniMaxAI/MiniMax-M3	max 2.4e-4 (the fp32 patch embedding rounded to f16)
AesSedai Q8_0	MiniMaxAI/MiniMax-M3	max 2.3e-3

Image processor vs AutoImageProcessor: pixels and grids identical on 20 images, also under --image-min-tokens / --image-max-tokens. Image token ids and the chat template's image rendering equal HF. tok_compare vs the HF tokenizer: 0/198,941 vocab, 0/20,000 combos, 1/13 edge cases (a combining accent: HF has an NFC normalizer; same on main).

Check sweep

scripts/gguf_check on header stubs of every MiniMax-M3 / M3.1-Flash GGUF repo on HF (230 files, 34 repos) plus their 26 mmproj files, kernels.json probed on the GPU. 83/204 files of the served arch pass, the same files as on main 0bf7dbb (0 of 230 differ on the text side). With each repo's mmproj set by FREETOKEN_MMPROJ, the tower checks on 87 files with 0 vision-side problems; 23/26 mmproj files load (the other 3 below). Text-side failures are the same as main: unknown ggml types, pre-indexer conversions, quantized indexer projections, per-tensor .scale, IQ1_M.

Serving

RTX PRO 6000 Blackwell Server Edition (96 GB), driver 595.91.07, Xeon Platinum 8559C, 249 GB RAM, torch 2.14.1+cu130, transformers 5.18.0.

ft serve --model-path .../MiniMax-M3-IQ2_XXS-00001-of-00003.gguf --max-seq-len-override 32768 --moe-cache-mem 40GiB

bartowski MiniMax-M3-IQ2_XXS (116.6 GB), 15 probes (OCR, shapes, two images, wide / tiny / tall images, a repeat). No HF checkpoint fits this box, so the reference is the prompt length from the HF tokenizer, chat template and MiniMaxM3VLProcessor.

mmproj	prompt tokens = HF	answers
BF16, auto-discovered	15/15	headline, synthetic text, cat count, two-image order correct
AesSedai Q8_0, --mmproj	15/15	11/15 same as BF16, the rest reworded
bartowski F16, --mmproj	15/15	12/15 same as BF16, the rest reworded

The 2400x240 and 300x1500 squashes are misread at IQ2_XXS. --mm-disable vision and a folder without an mmproj serve text and reject images with a 400.

Not covered

item	why
jkyamog's mmproj (minimax_m3_vl), Serpen's (mm.merge.*)	llama.cpp refuses them too; refused under --mmproj, skipped by discovery
M3 tower with an M3.1-Flash text file (or the reverse)	same arch, projector and width, so it loads from --mmproj or the model folder, as in llama.cpp; M3.1-Flash ships its own tower, so pair each release with its own mmproj
subfolder-layout repos (bartowski, AesSedai, ...)	discovery reads the model file's own folder (#803); --mmproj picks the file

No per-family GGUF test file, per the #721 review direction. pytest tests/models tests/mm tests/tokenizer tests/server: 1153 passed, 7 failed, the same 7 as on main 0bf7dbb (test_quant_config.py cases that read the local HF cache).