Skip to content

Add the experimental gemm_quantized microbenchmark (QuantizedGemmBenchmark and QuantizedGemmParams) in benchmarks/matmul.py to measure end-to-end matrix multiplication with on-the-fly dynamic per-channel quantization of both A[M, K] and B[K, N] operands. - #159

Merged
copybara-service[bot] merged 1 commit into
mainfrom
test_1013640925
Nov 16, 2026

Conversation

@copybara-service

@copybara-service copybara-service Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Add the experimental gemm_quantized microbenchmark (QuantizedGemmBenchmark and QuantizedGemmParams) in benchmarks/matmul.py to measure end-to-end matrix multiplication with on-the-fly dynamic per-channel quantization of both A[M, K] and B[K, N] operands.

  • Subclasses BaseGemmBenchmark in benchmarks/matmul.py (sharing _replicated_normal, compute roofline efficiency, XProf fallback matching, and 2 * M * N * K FLOP throughput calculation) and extracts MatmulShapeParams for shared (M, K, N) shape, dtype, and seed fields.
  • Quantizes bfloat16/float32 operands dynamically per row/column (inv_scale = Q_MAX / amax, q = clip(x * inv_scale, -Q_MAX, Q_MAX)) to quant_dtype (float8_e4m3fn, float8_e5m2, int8) inside the timed step before jax.lax.dot_general.
  • Counts 1-pass algorithmic HBM bytes (reading A and B once plus writing the output) and uses quant_dtype for the hardware MXU peak TFLOPS ceiling.

@google-cla

google-cla Bot commented Nov 10, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@copybara-service
copybara-service Bot force-pushed the test_1013640925 branch 3 times, most recently from d975aa8 to 5604b3d Compare November 16, 2026 06:12
@copybara-service copybara-service Bot changed the title Add the experimental gemm_quantized microbenchmark (QuantizedGemmBenchmark and QuantizedGemmParams) in benchmarks/matmul.py to measure end-to-end matrix multiplication with on-the-fly dynamic row-wise quantization of both A[M, K] and B[K, N] operands. Add the experimental gemm_quantized microbenchmark (QuantizedGemmBenchmark and QuantizedGemmParams) in benchmarks/matmul.py to measure end-to-end matrix multiplication with on-the-fly dynamic per-channel quantization of both A[M, K] and B[K, N] operands. Nov 16, 2026
…enchmark` and `QuantizedGemmParams`) in `benchmarks/matmul.py` to measure end-to-end matrix multiplication with on-the-fly dynamic per-channel quantization of both `A[M, K]` and `B[K, N]` operands.

*   Subclasses `BaseGemmBenchmark` in `benchmarks/matmul.py` (sharing `_replicated_normal`, compute roofline efficiency, XProf fallback matching, and `2 * M * N * K` FLOP throughput calculation) and extracts `MatmulShapeParams` for shared `(M, K, N)` shape, dtype, and seed fields.
*   Quantizes `bfloat16`/`float32` operands dynamically per row/column (`inv_scale = Q_MAX / amax`, `q = clip(x * inv_scale, -Q_MAX, Q_MAX)`) to `quant_dtype` (`float8_e4m3fn`, `float8_e5m2`, `int8`) inside the timed step before `jax.lax.dot_general`.
*   Counts 1-pass algorithmic HBM bytes (reading `A` and `B` once plus writing the output) and uses `quant_dtype` for the hardware MXU peak TFLOPS ceiling.

PiperOrigin-RevId: 1022517434
@copybara-service
copybara-service Bot merged commit 79da05c into main Nov 16, 2026
1 of 2 checks passed
@copybara-service
copybara-service Bot deleted the test_1013640925 branch November 16, 2026 06:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants