diff --git a/README.md b/README.md
index f6d0a4ba..fc1927ba 100644
--- a/README.md
+++ b/README.md
@@ -67,7 +67,7 @@
| **Compute** | **Matrix Multiplication (GEMM)** | Dense matrix multiplication ($$C = \alpha (A \times B) + \beta C$$) across configurable data types | • Compute Throughput (**TFLOPS**)
• Compute Roofline Efficiency (**%**)
• Compute Latency (**ms**) |
| **Memory (HBM)** | **HBM Memory Bandwidth** | High Bandwidth Memory STREAM operations (Copy, Scale, Add, Triad, Read-Only, Write-Only) | • Memory Bandwidth (**GB/s**)
• Memory Roofline Efficiency (**%**)
• Memory Access Latency (**ms**) |
| **Host I/O (PCIe)** | **Host-to-Device (H2D)**
**Device-to-Host (D2H)** | Host CPU memory to/from accelerator HBM data transfers | • PCIe Transfer Bandwidth (**GB/s**)
• Transfer Latency (**ms**) |
-| **Interconnect (ICI)** | **Device-to-Device (D2D)** | Point-to-point inter-chip data transfers across ICI links | • ICI Link Bandwidth (**GB/s**)
• Pairwise N × N Device Bandwidth Matrix
• Transfer Latency (**ms**) |
+| **Interconnect (ICI)** | **Device-to-Device (D2D)** | Data transfers between each device pair, over ICI (different chips) or die-to-die (same chip) | • D2D Link Bandwidth (**GB/s**)
• Pairwise N × N Device Bandwidth Matrix
• Transfer Latency (**ms**) |
| **Collectives** | **All-Gather**
**All-Reduce**
**All-to-All** | Distributed collective communication across multi-chip topologies | • Collective Bus Bandwidth (**GB/s**)
• Collective Step Latency (**ms**) |
---
@@ -210,7 +210,7 @@ tpums benchmark run host_to_device --xprof_timing --data_size_mib 256 --dtype bf
# 4. Device-to-Host (PCIe) Transfer Latency & Bandwidth on all local devices
tpums benchmark run device_to_host --xprof_timing --data_size_mib 256 --dtype bfloat16 --transfer_backend tpu_sync --scope host
-# 5. Device-to-Device (ICI) point-to-point transfer across all pairs
+# 5. Device-to-device (D2D) transfer across all pairs
tpums benchmark run device_to_device --xprof_timing --data_size_mib 1024 --direction uni --dtype bfloat16
# 6. Multi-Device All-Reduce Collective
@@ -487,7 +487,7 @@ TPUMS records primary performance metrics across both timing domains:
| **Latency (`ms`)** | `wall_clock_p50_ms`
*(+ `p90`, `avg`, `std`)* | `xprof_p50_ms`
*(+ `p90`, `avg`, `std`)* | Execution duration across timed iterations (`p50` median, `p90`, mean, and standard deviation). |
| **Compute Throughput (`TFLOPS`)** | `wall_clock_tflops_per_chip`
`wall_clock_tflops_per_device` | `xprof_tflops_per_chip`
`xprof_tflops_per_device` | Achieved compute throughput per physical TPU chip or per logical device ($$10^{12}$$ FLOPS). |
| **Bandwidth (`GB/s`)** | `wall_clock_bandwidth_per_chip_gb_s`
`wall_clock_bandwidth_per_device_gb_s`
`wall_clock_bandwidth_per_host_gb_s` | `xprof_bandwidth_per_chip_gb_s`
`xprof_bandwidth_per_device_gb_s`
`xprof_bandwidth_per_host_gb_s` | Achieved HBM, PCIe, ICI link, or collective bus bandwidth per chip, per device, or per host ($$10^9$$ bytes/sec). |
-| **Roofline Efficiency (`%`)** | `wall_clock_compute_roofline_efficiency_pct`
`wall_clock_memory_roofline_efficiency_pct`
`wall_clock_ici_roofline_efficiency_pct` | `xprof_compute_roofline_efficiency_pct`
`xprof_memory_roofline_efficiency_pct`
`xprof_ici_roofline_efficiency_pct` | Achieved compute, memory, or ICI (`all_to_all` only) performance as a percentage of the hardware's theoretical peak (`0–100%`). |
+| **Roofline Efficiency (`%`)** | `wall_clock_compute_roofline_efficiency_pct`
`wall_clock_memory_roofline_efficiency_pct`
`wall_clock_ici_roofline_efficiency_pct`
`wall_clock_roofline_efficiency_pct` | `xprof_compute_roofline_efficiency_pct`
`xprof_memory_roofline_efficiency_pct`
`xprof_ici_roofline_efficiency_pct`
`xprof_roofline_efficiency_pct` | Achieved compute, memory, ICI (`all_to_all` only), or device-to-device link (`device_to_device` only) performance as a percentage of the hardware's theoretical peak. |
See **[Throughput & Bandwidth Metric Conventions in `docs/BENCHMARKS.md`](docs/BENCHMARKS.md#per-device-vs-per-chip-scaling)** for how `_per_chip` vs. `_per_device` scales across dual-device (`tpu7x`) and single-device (`v6e`) TPU chips and which metrics each benchmark reports.
diff --git a/automation/report_generator.py b/automation/report_generator.py
index 2e875425..9eb44284 100644
--- a/automation/report_generator.py
+++ b/automation/report_generator.py
@@ -64,7 +64,7 @@
),
"device_to_device": (
"🔗 Device-to-Device (D2D)",
- "Peer-to-peer inter-chip transfer bandwidth and latency",
+ "Device-to-device transfer bandwidth and latency",
),
}
diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md
index fbcd1cd3..ed4f9284 100644
--- a/docs/BENCHMARKS.md
+++ b/docs/BENCHMARKS.md
@@ -20,7 +20,7 @@ This guide provides the reference for **TPUMS (TPU Microbenchmark Suite)** param
- [2.1.3 GEMM Pulse (`gemm_pulse`) [experimental]](#213-gemm-pulse-gemm_pulse-experimental)
- [2.2 HBM Memory Bandwidth (`hbm`)](#22-hbm-memory-bandwidth-hbm)
- [2.3 Host I/O Bandwidth (`host_to_device`, `device_to_host`)](#23-host-io-bandwidth-host_to_device-device_to_host)
- - [2.4 Inter-Chip Interconnect (`device_to_device`)](#24-inter-chip-interconnect-device_to_device)
+ - [2.4 Device-to-Device Transfers (`device_to_device`)](#24-inter-chip-interconnect-device_to_device)
- [2.5 Collective Communications (`all_reduce`, `all_gather`, `all_to_all`)](#25-collective-communications-all_reduce-all_gather-all_to_all)
- [Collective Concepts: `mesh_shape`, `sharding_strategy` & Parallel vs. Non-Parallel Replica Groups](#collective-concepts)
@@ -101,7 +101,7 @@ The table below summarizes which `` and `` each ben
| **`gemm`** | **Compute Throughput (`TFLOPS`)** & Roofline (`%`) | **Both Per-Chip & Per-Device**
(`*_tflops_per_chip`, `*_tflops_per_device`) | **Measures** per-device and aggregate per-chip matrix compute throughput (`TFLOPS`), and **compares** against peak chip compute rooflines (`%`). |
| **`hbm`** | **Memory Bandwidth (`GB/s`)** & Roofline (`%`) | **All Three Scopes (`Per-Device`, `Per-Chip`, `Per-Host`; active measured, smaller derived, larger `None`)**
(`*_bandwidth_per_{device,chip,host}_gb_s`) | **Measures** HBM memory bandwidth (`GB/s`) across the selected hardware `scope` (`device` default, `chip`, or `host`), **derives** smaller scopes as the average share of the concurrent aggregate measurement, leaves larger scopes as `None` (no upward extrapolation), and **compares** against peak per-device HBM bandwidth rooflines (`%`). |
| **`host_to_device` / `device_to_host`** | **Host I/O Bandwidth (`GB/s`)** | **All Three Scopes (`Per-Device`, `Per-Chip`, `Per-Host`; active measured, smaller derived, larger `None`)**
(`*_bandwidth_per_{device,chip,host}_gb_s`) | **Measures** PCIe transfer bandwidth (`GB/s`) between the host CPU and the selected hardware `scope`: all local devices concurrently (`scope: "host"`, default), one physical chip (`scope: "chip"`), or one local device (`scope: "device"`), **deriving** smaller scopes as the average share while leaving larger scopes as `None` (since shared PCIe links do not scale linearly across cores). |
-| **`device_to_device`** | **ICI Link Bandwidth (`GB/s`)** | **Per-Device**
(`*_bandwidth_per_device_gb_s`) | **Measures** point-to-point ICI link bandwidth (`GB/s`) between each logical device pair `(src_device_index, dst_device_index)`. |
+| **`device_to_device`** | **Device-to-Device Link Bandwidth (`GB/s`)** & Roofline (`%`) | **Per-Device**
(`*_bandwidth_per_device_gb_s`) | **Measures** transfer bandwidth (`GB/s`) between each device pair and **compares** it against the peak of the link used, ICI or die-to-die (`%`). |
| **`all_reduce` / `all_gather` / `all_to_all`** | **Collective Bus Bandwidth (`GB/s`)** | **Per-Chip**
(`*_bandwidth_per_chip_gb_s`) | **Measures** aggregate external ICI collective bus bandwidth (`GB/s`) per physical TPU chip across the active device mesh. |
---
@@ -408,9 +408,9 @@ metrics when `enable_checksum=True`.
---
-### 2.4 Inter-Chip Interconnect (`device_to_device`)
+### 2.4 Device-to-Device Transfers (`device_to_device`)
-- **Operation**: Measures point-to-point Inter-Chip Interconnect (ICI) link bandwidth (**GB/s**) and latency (**ms**) across all **`N × (N - 1)` directional device pairs** `(src_device_index, dst_device_index)` in the active topology (`src != dst`), supporting both single-host and multi-host topologies.
+- **Operation**: Measures device-to-device transfer bandwidth (**GB/s**) and latency (**ms**) across all **`N × (N - 1)` directional device pairs** `(src_device_index, dst_device_index)` in the active topology (`src != dst`), supporting both single-host and multi-host topologies. Pairs on different chips transfer over the Inter-Chip Interconnect (ICI); on chips with two devices (such as **`tpu7x`**), the two devices of one chip transfer over the on-package die-to-die link.
#### Parameters
@@ -423,8 +423,12 @@ metrics when `enable_checksum=True`.
#### Reported Metrics
-- **ICI Link Bandwidth**:
- - `{wall_clock|xprof}_bandwidth_per_device_gb_s` — Point-to-point link bandwidth between `(src_device_index, dst_device_index)` (`GB/s`), also formatted in `stdout` as an **`N × N` pairwise device bandwidth matrix** (`src` rows × `dst` columns).
+- **Link Bandwidth**:
+ - `{wall_clock|xprof}_bandwidth_per_device_gb_s` — Device-to-device transfer bandwidth between `(src_device_index, dst_device_index)` (`GB/s`), also formatted in `stdout` as an **`N × N` pairwise device bandwidth matrix** (`src` rows × `dst` columns). For `"bi"`, the bytes of both directions are counted.
+- **Link Roofline**:
+ - `roofline_bound_by` — Link that carries the transfer: `ici` (devices on different chips) or `die_to_die` (the two devices of one chip, e.g. `tpu7x`).
+ - `peak_link_bandwidth_per_device_gb_s` — Theoretical peak bandwidth of that link (`GB/s`), doubled for `"bi"`.
+ - `{wall_clock|xprof}_roofline_efficiency_pct` — Measured bandwidth as a percentage of the peak (`%`).
- **Latency (`ms`)**:
- `{wall_clock|xprof}_{p50|p90|avg|std}_ms` — Iteration execution latency across median (`p50`), 90th percentile (`p90`), mean (`avg`), and standard deviation (`std`) in milliseconds (`ms`).
- **Benchmark Attributes**:
@@ -432,6 +436,9 @@ metrics when `enable_checksum=True`.
- `direction` — Active transfer direction (`"uni"` or `"bi"`).
- `data_size_mib` — Payload size per directional stream (`MiB`).
+> [!NOTE]
+> Only one device pair transfers at a time, so each transfer gets the full link. ICI links are shared by the devices of a chip, so don't sum ICI peaks per chip.
+
---
diff --git a/src/accelerator_microbenchmarks/benchmarks/device_to_device.py b/src/accelerator_microbenchmarks/benchmarks/device_to_device.py
index bc1d8e97..7dbd3366 100644
--- a/src/accelerator_microbenchmarks/benchmarks/device_to_device.py
+++ b/src/accelerator_microbenchmarks/benchmarks/device_to_device.py
@@ -1,11 +1,13 @@
"""Device-to-Device (D2D) transfer performance microbenchmark."""
import dataclasses
+import logging
from typing import Any, Callable, Optional, Sequence
from accelerator_microbenchmarks.core import base
from accelerator_microbenchmarks.core import constants
from accelerator_microbenchmarks.core import dtypes
+from accelerator_microbenchmarks.core import platform
from accelerator_microbenchmarks.core import registry
from accelerator_microbenchmarks.core import report
from accelerator_microbenchmarks.core import system
@@ -22,6 +24,85 @@ class TransferDirection(constants.ParamEnum):
BI = "bi"
+def classify_transfer_link(
+ src: jax.Device, dst: jax.Device
+) -> constants.TransferLink:
+ """Returns the physical link that carries a `src -> dst` transfer.
+
+ Devices on the same chip share physical chip coordinates and talk over the
+ on-package die-to-die link; devices on different chips of the same slice
+ talk over ICI.
+
+ Args:
+ src: Source device.
+ dst: Destination device.
+
+ Returns:
+ `TransferLink.DIE_TO_DIE` if both devices are on the same chip, otherwise
+ `TransferLink.ICI`.
+
+ Raises:
+ ValueError: If the devices have no physical chip coordinates (non-TPU) or
+ are on different slices (the transfer then crosses DCN).
+ """
+ src_coords = getattr(src, "coords", None)
+ dst_coords = getattr(dst, "coords", None)
+ if src_coords is None or dst_coords is None:
+ raise ValueError("devices have no physical chip coordinates (non-TPU)")
+ if platform.get_device_slice_index(src) != platform.get_device_slice_index(
+ dst
+ ):
+ raise ValueError("src and dst are on different slices (DCN)")
+ if tuple(src_coords) == tuple(dst_coords):
+ return constants.TransferLink.DIE_TO_DIE
+ return constants.TransferLink.ICI
+
+
+def device_to_device_peak_bandwidth_gb_s(
+ link: constants.TransferLink,
+ direction: TransferDirection,
+ hardware_spec: system.HardwareSpec,
+) -> float:
+ """Returns the theoretical peak bandwidth of one D2D transfer (GB/s).
+
+ A device-to-device transfer is bounded by a single physical link per
+ direction. For ICI this holds independent of the hop count: a single
+ transfer follows one route, so measured cross-chip transfers saturate one
+ link whether the chips are neighbors or several hops apart. Links are
+ full-duplex and `DeviceToDeviceBenchmark.get_total_bytes()` counts the
+ bytes of both directions for `BI`, so the bidirectional peak is twice the
+ per-direction peak.
+
+ Scope: the result is the ceiling for the single measured `src -> dst`
+ transfer, which runs alone (one device pair at a time) and so has the whole
+ link to itself. It is not a per-device share of a chip resource: ICI links
+ belong to the chip and are shared by all of its devices, so ICI peaks must
+ not be summed across the devices of a chip.
+
+ Args:
+ link: The link carrying the transfer.
+ direction: Unidirectional or bidirectional transfer.
+ hardware_spec: Hardware spec holding the link bandwidths.
+
+ Returns:
+ The peak bandwidth in decimal GB/s, in the same units as
+ `*_bandwidth_per_device_gb_s`.
+
+ Raises:
+ ValueError: If `hardware_spec` has no bandwidth for `link`.
+ """
+ if link == constants.TransferLink.DIE_TO_DIE:
+ if hardware_spec.die_to_die is None:
+ raise ValueError(f"no die-to-die spec for {hardware_spec.name}")
+ per_direction_gb_s = hardware_spec.die_to_die.bandwidth_per_direction_gb_s
+ else:
+ if hardware_spec.ici is None:
+ raise ValueError(f"no ICI spec for {hardware_spec.name}")
+ per_direction_gb_s = hardware_spec.ici.link_bw_per_direction_gb_s
+ num_directions = 2.0 if direction == TransferDirection.BI else 1.0
+ return per_direction_gb_s * num_directions
+
+
@dataclasses.dataclass
class DeviceToDeviceParams(base.SingleDtypeBenchmarkParams):
"""YAML / CLI configuration specification (NO src/dst fields)."""
@@ -93,6 +174,7 @@ class DeviceToDeviceBenchmark(base.BaseBenchmark[DeviceToDeviceTestCaseParams]):
Config = DeviceToDeviceParams
derive_chip_bandwidth: bool = False
+ roofline_mode: constants.RooflineMode = constants.RooflineMode.NETWORK_LINK
REPORT_SCHEMA: Sequence[tuple[str, Callable[[Any], str]]] = (
("dtype", report.format_str),
("direction", report.format_str),
@@ -234,3 +316,31 @@ def calculate_throughput_metrics(
return {
f"{prefix}_bandwidth_per_device_gb_s": bandwidth_gb_s,
}
+
+ def get_peak_link_bandwidth(
+ self,
+ ) -> tuple[float, constants.TransferLink] | None:
+ """Returns the D2D peak and its bounding link, or None (with a warning).
+
+ The peak is not modeled (and the roofline columns are omitted) for:
+ - Non-TPU devices without chip coordinates.
+ - Pairs on different slices (DCN).
+ - Hardware specs without a bandwidth for the bounding link.
+ """
+ if self.mesh is None:
+ raise ValueError("Mesh not initialized.")
+ # Indices refer to mesh positions, which need not follow chip order, so
+ # classify the actual devices rather than the indices.
+ src = self.mesh.devices.flat[self.config.src_device_index]
+ dst = self.mesh.devices.flat[self.config.dst_device_index]
+ try:
+ link = classify_transfer_link(src, dst)
+ peak_gb_s = device_to_device_peak_bandwidth_gb_s(
+ link, self.config.direction, self.hardware_spec
+ )
+ except ValueError as e:
+ logging.warning(
+ "Skipping device_to_device *_roofline_efficiency_pct: %s.", e
+ )
+ return None
+ return peak_gb_s, link
diff --git a/src/accelerator_microbenchmarks/core/base.py b/src/accelerator_microbenchmarks/core/base.py
index 21b5e929..fcec2904 100644
--- a/src/accelerator_microbenchmarks/core/base.py
+++ b/src/accelerator_microbenchmarks/core/base.py
@@ -557,6 +557,20 @@ def get_peak_ici_bandwidth_per_chip(self) -> float | None:
" RooflineMode.NETWORK_ICI."
)
+ def get_peak_link_bandwidth(
+ self,
+ ) -> tuple[float, constants.TransferLink] | None:
+ """Return the peak link bandwidth per device (GB/s) and the bounding link.
+
+ Required when roofline_mode == RooflineMode.NETWORK_LINK. Returns None
+ when the peak cannot be modeled for the current configuration.
+ """
+ raise NotImplementedError(
+ f"{self.__class__.__name__} must implement"
+ " get_peak_link_bandwidth() when roofline_mode =="
+ " RooflineMode.NETWORK_LINK."
+ )
+
@abc.abstractmethod
def get_total_bytes(self) -> float:
"""Calculate total bytes moved to/from HBM."""
diff --git a/src/accelerator_microbenchmarks/core/constants.py b/src/accelerator_microbenchmarks/core/constants.py
index 262f7872..807502e1 100644
--- a/src/accelerator_microbenchmarks/core/constants.py
+++ b/src/accelerator_microbenchmarks/core/constants.py
@@ -42,12 +42,29 @@ class RooflineMode(str, enum.Enum):
supplied by the benchmark (currently only `all_to_all`, whose peak is
bounded by bisection bandwidth). Emits `peak_ici_bandwidth_per_chip_gb_s`
and `_ici_roofline_efficiency_pct`.
+ NETWORK_LINK: Device-to-device transfer roofline: the measured per-device
+ bandwidth is compared against the peak of the single physical link that
+ carries the transfer (ICI between chips, or the die-to-die link between
+ the dies of one chip), as supplied by the benchmark. Emits
+ `peak_link_bandwidth_per_device_gb_s`, `roofline_bound_by` (a
+ `TransferLink` value), and `_roofline_efficiency_pct`.
"""
NONE = "none"
COMPUTE = "compute"
MEMORY_HBM = "memory_hbm"
NETWORK_ICI = "network_ici"
+ NETWORK_LINK = "network_link"
+
+
+class TransferLink(enum.StrEnum):
+ """Physical link bounding a device-to-device transfer.
+
+ The values are the strings emitted in the `roofline_bound_by` metric.
+ """
+
+ ICI = "ici"
+ DIE_TO_DIE = "die_to_die"
class TimingDomain(enum.StrEnum):
diff --git a/src/accelerator_microbenchmarks/core/roofline.py b/src/accelerator_microbenchmarks/core/roofline.py
index f69bf554..80b619df 100644
--- a/src/accelerator_microbenchmarks/core/roofline.py
+++ b/src/accelerator_microbenchmarks/core/roofline.py
@@ -26,6 +26,8 @@ def apply_roofline_analysis(
`_memory_roofline_efficiency_pct` apply invariantly to both scopes.
`_ici_roofline_efficiency_pct` is defined per chip only, since ICI
collectives report exclusively `*_bandwidth_per_chip_gb_s`.
+ `_roofline_efficiency_pct` (NETWORK_LINK) is defined per device,
+ since device-to-device transfers report `*_bandwidth_per_device_gb_s`.
Args:
benchmark_instance: The benchmark instance being analyzed.
@@ -119,4 +121,19 @@ def apply_roofline_analysis(
) * 100.0
return metrics
+ if mode == constants.RooflineMode.NETWORK_LINK:
+ peak = benchmark_instance.get_peak_link_bandwidth()
+ if peak is None:
+ return metrics
+ peak_bw, link = peak
+ metrics["peak_link_bandwidth_per_device_gb_s"] = peak_bw
+ metrics["roofline_bound_by"] = str(link)
+ for prefix in constants.TimingDomain:
+ actual_bw = metrics.get(f"{prefix}_bandwidth_per_device_gb_s")
+ if actual_bw is not None and actual_bw >= 0 and peak_bw > 0:
+ metrics[f"{prefix}_roofline_efficiency_pct"] = (
+ actual_bw / peak_bw
+ ) * 100.0
+ return metrics
+
return metrics
diff --git a/src/accelerator_microbenchmarks/core/system.py b/src/accelerator_microbenchmarks/core/system.py
index 55cfb3a3..95abfb23 100644
--- a/src/accelerator_microbenchmarks/core/system.py
+++ b/src/accelerator_microbenchmarks/core/system.py
@@ -16,6 +16,7 @@
HardwareSpec = schema.HardwareSpec
TflopsSpec = schema.TflopsSpec
IciSpec = schema.IciSpec
+DieToDieSpec = schema.DieToDieSpec
WraparoundRule = schema.WraparoundRule
HbmSpec = schema.HbmSpec
diff --git a/src/accelerator_microbenchmarks/specs/schema.py b/src/accelerator_microbenchmarks/specs/schema.py
index b681f829..acd1e8df 100644
--- a/src/accelerator_microbenchmarks/specs/schema.py
+++ b/src/accelerator_microbenchmarks/specs/schema.py
@@ -106,6 +106,19 @@ class IciSpec:
wraparound_rule: WraparoundRule = WraparoundRule.NONE
+@dataclasses.dataclass(frozen=True)
+class DieToDieSpec:
+ """On-package link between the compute dies of one chip (theoretical peak).
+
+ Attributes:
+ bandwidth_per_direction_gb_s: Theoretical peak bandwidth between the two
+ dies of a chip in one direction, in decimal GB/s. The link is
+ full-duplex, so each direction carries this bandwidth independently.
+ """
+
+ bandwidth_per_direction_gb_s: float
+
+
@dataclasses.dataclass(frozen=True)
class HbmSpec:
"""High Bandwidth Memory specifications (per-chip).
@@ -128,6 +141,9 @@ class HardwareSpec:
devices_per_chip: int = 1
tflops: TflopsSpec | None = None
ici: IciSpec | None = None
+ # `None` when the chip has no die-to-die link (one device per chip) or its
+ # bandwidth has not been verified yet.
+ die_to_die: DieToDieSpec | None = None
hbm: HbmSpec | None = None
@property
diff --git a/src/accelerator_microbenchmarks/specs/tpu7x.py b/src/accelerator_microbenchmarks/specs/tpu7x.py
index 476e4773..4634fbb2 100644
--- a/src/accelerator_microbenchmarks/specs/tpu7x.py
+++ b/src/accelerator_microbenchmarks/specs/tpu7x.py
@@ -21,5 +21,9 @@
link_bw_per_direction_gb_s=100.0,
wraparound_rule=schema.WraparoundRule.ALL_DIMS_GE_4,
),
+ # Each chip has two chiplets connected by a die-to-die (D2D) interface
+ # (https://docs.cloud.google.com/tpu/docs/tpu7x#dual-chiplet_architecture).
+ # The D2D link is full duplex.
+ die_to_die=schema.DieToDieSpec(bandwidth_per_direction_gb_s=800.0),
hbm=schema.HbmSpec(peak_bw_gb_s=7380.0),
)
diff --git a/tests/test_device_to_device.py b/tests/test_device_to_device.py
index 2ca6d449..03d1192c 100644
--- a/tests/test_device_to_device.py
+++ b/tests/test_device_to_device.py
@@ -1,8 +1,10 @@
"""Unit tests for device_to_device.py."""
import argparse
+import dataclasses
import io
import os
+import types
from unittest import mock
from absl.testing import absltest
@@ -10,6 +12,7 @@
from accelerator_microbenchmarks import cli
from accelerator_microbenchmarks.benchmarks import device_to_device
from accelerator_microbenchmarks.core import base
+from accelerator_microbenchmarks.core import constants
from accelerator_microbenchmarks.core import registry
from accelerator_microbenchmarks.core import report
from accelerator_microbenchmarks.core import system
@@ -254,6 +257,10 @@ def test_run_e2e(self):
self.assertNotIn("compute_roofline_efficiency_pct", result.metrics)
self.assertNotIn("peak_hbm_bw_gb_s", result.metrics)
self.assertNotIn("memory_roofline_efficiency_pct", result.metrics)
+ # CPU devices have no chip coordinates, so the link roofline is skipped.
+ self.assertNotIn("peak_link_bandwidth_per_device_gb_s", result.metrics)
+ self.assertNotIn("roofline_bound_by", result.metrics)
+ self.assertNotIn("wall_clock_roofline_efficiency_pct", result.metrics)
def test_get_arithmetic_intensity(self):
"""Verify DeviceToDeviceBenchmark does not implement get_arithmetic_intensity."""
@@ -497,5 +504,133 @@ def test_generate_benchmark_report_dual_representation(self):
self.assertIn("X", rep)
+def _fake_device(coords=None, slice_index=0):
+ """Returns a stand-in for a jax.Device with the attributes D2D inspects."""
+ device = types.SimpleNamespace(slice_index=slice_index)
+ if coords is not None:
+ device.coords = coords
+ return device
+
+
+class ClassifyTransferLinkTest(absltest.TestCase):
+ """Unit tests for classify_transfer_link."""
+
+ def test_same_chip_is_die_to_die(self):
+ self.assertEqual(
+ device_to_device.classify_transfer_link(
+ _fake_device((0, 0, 0)), _fake_device([0, 0, 0])
+ ),
+ constants.TransferLink.DIE_TO_DIE,
+ )
+
+ def test_different_chips_is_ici(self):
+ self.assertEqual(
+ device_to_device.classify_transfer_link(
+ _fake_device((0, 0, 0)), _fake_device((1, 1, 0))
+ ),
+ constants.TransferLink.ICI,
+ )
+
+ def test_different_slices_raises(self):
+ with self.assertRaisesRegex(ValueError, "different slices"):
+ device_to_device.classify_transfer_link(
+ _fake_device((0, 0, 0), slice_index=0),
+ _fake_device((0, 0, 0), slice_index=1),
+ )
+
+ def test_missing_coords_raises(self):
+ with self.assertRaisesRegex(ValueError, "no physical chip coordinates"):
+ device_to_device.classify_transfer_link(
+ _fake_device(), _fake_device((0, 0, 0))
+ )
+
+
+class DeviceToDevicePeakBandwidthTest(parameterized.TestCase):
+ """Unit tests for device_to_device_peak_bandwidth_gb_s."""
+
+ @parameterized.parameters(
+ (constants.TransferLink.ICI, "uni", 100.0),
+ (constants.TransferLink.ICI, "bi", 200.0),
+ (constants.TransferLink.DIE_TO_DIE, "uni", 800.0),
+ (constants.TransferLink.DIE_TO_DIE, "bi", 1600.0),
+ )
+ def test_tpu7x_peak(self, link, direction, expected_gb_s):
+ self.assertEqual(
+ device_to_device.device_to_device_peak_bandwidth_gb_s(
+ link,
+ device_to_device.TransferDirection(direction),
+ system.TPU7X_HARDWARE_SPEC,
+ ),
+ expected_gb_s,
+ )
+
+ def test_missing_die_to_die_spec_raises(self):
+ with self.assertRaisesRegex(ValueError, "no die-to-die spec for v6e"):
+ device_to_device.device_to_device_peak_bandwidth_gb_s(
+ constants.TransferLink.DIE_TO_DIE,
+ device_to_device.TransferDirection.UNI,
+ system.V6E_HARDWARE_SPEC,
+ )
+
+ def test_missing_ici_spec_raises(self):
+ spec = dataclasses.replace(system.V6E_HARDWARE_SPEC, ici=None)
+ with self.assertRaisesRegex(ValueError, "no ICI spec for v6e"):
+ device_to_device.device_to_device_peak_bandwidth_gb_s(
+ constants.TransferLink.ICI,
+ device_to_device.TransferDirection.UNI,
+ spec,
+ )
+
+
+class GetPeakLinkBandwidthTest(parameterized.TestCase):
+ """Unit tests for DeviceToDeviceBenchmark.get_peak_link_bandwidth."""
+
+ def _make_benchmark(self, devices, src, dst, direction="uni", spec=None):
+ bm = device_to_device.DeviceToDeviceBenchmark(
+ device_to_device.DeviceToDeviceTestCaseParams(
+ direction=direction, src_device_index=src, dst_device_index=dst
+ ),
+ hardware_spec=spec or system.TPU7X_HARDWARE_SPEC,
+ )
+ bm.mesh = types.SimpleNamespace(devices=types.SimpleNamespace(flat=devices))
+ return bm
+
+ def test_classifies_mesh_devices_not_indices(self):
+ # Mesh positions 0 and 1 hold devices on different chips, and positions
+ # 0 and 2 hold the two dies of one chip.
+ devices = [
+ _fake_device((0, 0, 0)),
+ _fake_device((1, 0, 0)),
+ _fake_device((0, 0, 0)),
+ ]
+ self.assertEqual(
+ self._make_benchmark(devices, 0, 1).get_peak_link_bandwidth(),
+ (100.0, constants.TransferLink.ICI),
+ )
+ self.assertEqual(
+ self._make_benchmark(devices, 0, 2, "bi").get_peak_link_bandwidth(),
+ (1600.0, constants.TransferLink.DIE_TO_DIE),
+ )
+
+ def test_unmodeled_pair_returns_none(self):
+ devices = [_fake_device((0, 0, 0)), _fake_device((0, 0, 0))]
+ bm = self._make_benchmark(devices, 0, 1, spec=system.V6E_HARDWARE_SPEC)
+ with self.assertLogs(level="WARNING") as logs:
+ self.assertIsNone(bm.get_peak_link_bandwidth())
+ self.assertIn("no die-to-die spec", logs.output[0])
+
+ def test_roofline_metrics(self):
+ devices = [_fake_device((0, 0, 0)), _fake_device((0, 0, 0))]
+ bm = self._make_benchmark(devices, 0, 1)
+ metrics = bm.apply_roofline_analysis({
+ "wall_clock_bandwidth_per_device_gb_s": 400.0,
+ "xprof_bandwidth_per_device_gb_s": 680.0,
+ })
+ self.assertEqual(metrics["peak_link_bandwidth_per_device_gb_s"], 800.0)
+ self.assertEqual(metrics["roofline_bound_by"], "die_to_die")
+ self.assertAlmostEqual(metrics["wall_clock_roofline_efficiency_pct"], 50.0)
+ self.assertAlmostEqual(metrics["xprof_roofline_efficiency_pct"], 85.0)
+
+
if __name__ == "__main__":
absltest.main()
diff --git a/tests/test_roofline.py b/tests/test_roofline.py
index 4c5513f3..5548ddd5 100644
--- a/tests/test_roofline.py
+++ b/tests/test_roofline.py
@@ -458,6 +458,45 @@ def test_apply_roofline_analysis_network_ici_no_peak(self):
res = roofline.apply_roofline_analysis(self.mock_benchmark, metrics)
self.assertEqual(res, {"wall_clock_bandwidth_per_chip_gb_s": 100.0})
+ def test_apply_roofline_analysis_network_link_mode(self):
+ """Verify NETWORK_LINK emits peak, bounding link and per-domain efficiency."""
+ self.mock_benchmark.roofline_mode = constants.RooflineMode.NETWORK_LINK
+ self.mock_benchmark.get_peak_link_bandwidth.return_value = (
+ 100.0,
+ constants.TransferLink.ICI,
+ )
+ metrics = {
+ "wall_clock_bandwidth_per_device_gb_s": 80.0,
+ "xprof_bandwidth_per_device_gb_s": 93.0,
+ }
+ res = roofline.apply_roofline_analysis(self.mock_benchmark, metrics)
+ self.assertEqual(res["peak_link_bandwidth_per_device_gb_s"], 100.0)
+ self.assertEqual(res["roofline_bound_by"], "ici")
+ self.assertAlmostEqual(res["wall_clock_roofline_efficiency_pct"], 80.0)
+ self.assertAlmostEqual(res["xprof_roofline_efficiency_pct"], 93.0)
+ self.assertNotIn("peak_ici_bandwidth_per_chip_gb_s", res)
+ self.assertNotIn("wall_clock_ici_roofline_efficiency_pct", res)
+
+ def test_apply_roofline_analysis_network_link_not_clamped(self):
+ """Verify efficiency above 100% is reported as-is."""
+ self.mock_benchmark.roofline_mode = constants.RooflineMode.NETWORK_LINK
+ self.mock_benchmark.get_peak_link_bandwidth.return_value = (
+ 100.0,
+ constants.TransferLink.ICI,
+ )
+ metrics = {"wall_clock_bandwidth_per_device_gb_s": 120.0}
+ res = roofline.apply_roofline_analysis(self.mock_benchmark, metrics)
+ self.assertAlmostEqual(res["wall_clock_roofline_efficiency_pct"], 120.0)
+ self.assertNotIn("xprof_roofline_efficiency_pct", res)
+
+ def test_apply_roofline_analysis_network_link_no_peak(self):
+ """Verify no link roofline keys are added without a modeled peak."""
+ self.mock_benchmark.roofline_mode = constants.RooflineMode.NETWORK_LINK
+ self.mock_benchmark.get_peak_link_bandwidth.return_value = None
+ metrics = {"wall_clock_bandwidth_per_device_gb_s": 80.0}
+ res = roofline.apply_roofline_analysis(self.mock_benchmark, metrics)
+ self.assertEqual(res, {"wall_clock_bandwidth_per_device_gb_s": 80.0})
+
if __name__ == "__main__":
absltest.main()
diff --git a/tests/test_system.py b/tests/test_system.py
index 1eb80b22..b8f28e62 100644
--- a/tests/test_system.py
+++ b/tests/test_system.py
@@ -194,6 +194,9 @@ def test_hardware_spec_tpu7x_presets(self):
hw_spec.ici.wraparound_rule, system.WraparoundRule.ALL_DIMS_GE_4
)
+ # Test die-to-die stats
+ self.assertEqual(hw_spec.die_to_die.bandwidth_per_direction_gb_s, 800.0)
+
# Test HBM stats
self.assertEqual(hw_spec.hbm.peak_bw_gb_s, 7380.0)
self.assertEqual(hw_spec.peak_hbm_bandwidth_per_device, 3690.0)
@@ -225,6 +228,9 @@ def test_hardware_spec_v6e_presets(self):
hw_spec.ici.wraparound_rule, system.WraparoundRule.PER_DIM_GE_16
)
+ # One device per chip: no die-to-die link.
+ self.assertIsNone(hw_spec.die_to_die)
+
# Test HBM stats
self.assertEqual(hw_spec.hbm.peak_bw_gb_s, 1638.4)
self.assertEqual(hw_spec.peak_hbm_bandwidth_per_device, 1638.4)