Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,20 @@ The table below summarizes which `<metric_type>` and `<hardware_scope>` each ben
| `use_scaling_factors` | `bool` | `False` | Allocate and apply row-wise (`M × 1`) and column-wise (`1 × N`) `float32` scaling factor vectors for quantized/FP8 rescaling. |
| `seed` | `int` | `0` | PRNG seed for initializing input tensors in HBM. |

> [!NOTE]
> **`float32` shares the `bfloat16` peak.** MXU multiplies take `bfloat16`
> inputs and accumulate in `float32` (see
> [TPU architecture](https://docs.cloud.google.com/tpu/docs/system-architecture-tpu-vm)),
> and the GEMM benchmarks pass no `precision` to the matmul, so JAX's default
> precision applies and a `float32` matmul runs as one `bfloat16` pass. JAX's
> [precision levels](https://docs.jax.dev/en/latest/201/precision.html#precision-the-classic-three-levels)
> affect only `float32` computations. On TPU:
>
> - `Precision.DEFAULT` (what TPUMS runs): fastest, least accurate; computes in
> `bfloat16`.
> - `Precision.HIGH`: slower, more accurate; three `bfloat16` passes.
> - `Precision.HIGHEST`: slowest, most accurate; six `bfloat16` passes.

#### Reported Metrics

- **Compute Throughput & Roofline**:
Expand Down
6 changes: 5 additions & 1 deletion src/accelerator_microbenchmarks/specs/tpu7x.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,11 @@
tflops=schema.TflopsSpec(
peak_tflops_per_device={
"bfloat16": 1153.5,
"float32": 576.75, # 2-pass BF16 emulation on MXU
# Same peak as bfloat16: MXU multiplies take bfloat16 inputs and
# accumulate in FP32, so a float32 matmul at JAX's default
# precision is one bfloat16 pass (see docs/BENCHMARKS.md).
# https://docs.cloud.google.com/tpu/docs/system-architecture-tpu-vm
"float32": 1153.5,
"float8_e5m2": 2307.0,
"float8_e4m3fn": 2307.0,
"int8": 1153.5, # Emulated on BF16 datapath
Expand Down
6 changes: 5 additions & 1 deletion src/accelerator_microbenchmarks/specs/v6e.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,11 @@
tflops=schema.TflopsSpec(
peak_tflops_per_device={
"bfloat16": 918.0,
"float32": 459.0, # 2-pass BF16 emulation on MXU
# Same peak as bfloat16: MXU multiplies take bfloat16 inputs and
# accumulate in FP32, so a float32 matmul at JAX's default
# precision is one bfloat16 pass (see docs/BENCHMARKS.md).
# https://docs.cloud.google.com/tpu/docs/system-architecture-tpu-vm
"float32": 918.0,
"float8_e5m2": 918.0,
"float8_e4m3fn": 918.0,
"int8": 1836.0,
Expand Down
4 changes: 2 additions & 2 deletions tests/test_system.py
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ def test_hardware_spec_tpu7x_presets(self):
self.assertEqual(
hw_spec.tflops.peak_tflops_per_device["bfloat16"], 1153.5
)
self.assertEqual(hw_spec.tflops.peak_tflops_per_device["float32"], 576.75)
self.assertEqual(hw_spec.tflops.peak_tflops_per_device["float32"], 1153.5)
self.assertEqual(
hw_spec.tflops.peak_tflops_per_device["float8_e5m2"], 2307.0
)
Expand Down Expand Up @@ -215,7 +215,7 @@ def test_hardware_spec_v6e_presets(self):
self.assertEqual(
hw_spec.tflops.peak_tflops_per_device["bfloat16"], 918.0
)
self.assertEqual(hw_spec.tflops.peak_tflops_per_device["float32"], 459.0)
self.assertEqual(hw_spec.tflops.peak_tflops_per_device["float32"], 918.0)
self.assertEqual(
hw_spec.tflops.peak_tflops_per_device["float8_e5m2"], 918.0
)
Expand Down
Loading