Repository navigation
Sub-channel (1 × 128 and 1 × 512) quantized matrix multiplication applies dequantization scales per K-block inside the contracting reduction loop, which in plain JAX requires 3D batched dots or unrolled HBM passes. Even for full-row (1 × K) scaling, fusing scale + cast into the VMEM matmul tile loop avoids an extra HBM round-trip without relying on compiler downcast-convert fusion. - #166
Closed
copybara-service[bot] wants to merge 1 commit into
Closed
copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into