Skip to content

Sub-channel (1 × 128 and 1 × 512) quantized matrix multiplication applies dequantization scales per K-block inside the contracting reduction loop, which in plain JAX requires 3D batched dots or unrolled HBM passes. Even for full-row (1 × K) scaling, fusing scale + cast into the VMEM matmul tile loop avoids an extra HBM round-trip without relying on compiler downcast-convert fusion. - #166

Closed
copybara-service[bot] wants to merge 1 commit into
mainfrom
test_1023312592

Commits

  1. Commits on Nov 17, 2026