Skip to content

basic: size every operand in the block model by its own element size (#426) - #431

Open
shinaoka wants to merge 1 commit into
mainfrom
fix/426-per-operand-elem-size
Open

shinaoka wants to merge 1 commit into
mainfrom
fix/426-per-operand-elem-size

Conversation

@shinaoka

@shinaoka shinaoka commented Nov 9, 2026

Copy link
Copy Markdown
Member

Closes #426.

block::compute_block_sizes took one element size for every operand, and the callers passed the largest one (size_of::<D>().max(size_of::<A>())). A mixed-size operation therefore overstated its smaller operand's footprint, so the BLOCK_MEMORY_SIZE comparison and the cache-line rounding were computed at the wrong size. Strided.jl forms byte strides per array: map((s, stride) -> s .* stride, sizeof.(eltype.(arrays)), strides) (src/mapreduce.jl:201, 2.6.6).

What changed

  • Model (strided-basic/src/block.rs): compute_block_sizes, reduced_block_sizes, memory_region, region_in and Rows take elem_sizes: &[usize], one per entry of strides_list, and form each operand's byte strides, region term and cache-line rounding from its own size. Stride orders and costs stay on the element strides, as the reference does: a positive size is an exact positive multiple, so the order is unchanged, and a zero size makes every byte stride 0, hence order 1 on every axis.
  • The increasing-strides exit now needs every size positive. A zero-size operand's constant order 1 is the global minimum at every recursion level, so another operand's larger order stops matching it and the keep-first recursion continues; the exit would be unsound for mixed zero/non-zero sizes. All-zero sizes still keep the full dimensions (all orders 1 everywhere). This was caught in the pre-implementation review and is pinned below.
  • Public API unchanged: execution::build_plan_fused(..., elem_size: usize) keeps its signature and fills a uniform InlineVec (inline, no heap allocation), so strided-fused is untouched and no public item is added.
  • Callers: map_view.rs (5 sites), ops_view.rs (5), reduce_view.rs (2), update_view.rs (run_update takes elem_sizes), raw_ops.rs (tile_blocks, exits_before_metadata) pass size_of per operand in strides_list order, through one kernel::pointwise_plan that owns the small/large branch instead of 13 copies. raw_ops's tiling decision now agrees with the no-reuse decline it sits beside (the issue's point). The reduce entry in ops_view.rs no longer includes the accumulator type R, which has no strides.

Zero-element-size contract (now explicit)

A zero-size operand contributes one cache line to the region and stride order 1 on every axis. It never shrinks a block by itself, but it is not neutral in a mixed operation: it is the global minimum on every axis, so the other operands' increasing orders no longer satisfy the keep-first condition. All-zero-size operand sets keep the full dimensions. Three tests pin this (a_zero_sized_source_stops_the_increasing_strides_exit, a_zero_sized_destination_leaves_the_reduction_to_the_source, a_zero_element_size_keeps_every_axis_for_any_operand_count).

The all-1-orders convention for a zero-size operand is a deliberate Rust decision, not Julia parity: Strided.jl ranks element strides (map(indexorder, strides)), so it would rank a zero-size operand's element strides instead. The pre-existing a_zero_element_size_keeps_every_axis_for_any_operand_count test already pinned that convention for the all-zero case; this change makes it per operand and states it.

Cross-check against Strided.jl 2.6.6 itself

Loading the installed package (~/.julia/packages/Strided/gJBvl) and calling its own _computeblocks with per-array sizeof gives exactly the block vectors of this implementation:

2x4096x8  f64<-u8  sizes=(8,1)  region 589824  blocks (2, 1816, 1)
2x4096x8  f64<-f64 sizes=(8,8)  region 1048576 blocks (2, 1024, 1)
128x128   f64<-u8  sizes=(8,1)  region 147456  blocks (57, 56)
128x128   f64<-f64 sizes=(8,8)  region 262144  blocks (64, 32)
128x128   c64<-f64 sizes=(16,8) region 393216  blocks (40, 32)

The pre-#426 wiring gave (2, 1024, 1), (64, 32) and (32, 32) for the three mixed rows.

Verification

$ cargo test -p strided-basic --lib
test result: ok. 254 passed; 0 failed; 0 ignored

$ cargo test --workspace
78 test binaries: 1147 passed, 0 failed; plus 128 doctests, 0 failed (rc 0)

$ cargo fmt --all -- --check            # clean
$ cargo clippy --workspace --all-targets  # clean

$ cargo test -p strided-basic
35 test binaries: 649 passed, 0 failed (lib 254, integration plus doctests)

$ RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps     # clean
# ~/.hermes/profiles/tensor4all/scripts/check_body.sh does not exist on this host;
# its equivalent is the org script the PR workflow itself runs:
$ python3 /tmp/check_pr_body.py --body-file <this body>
::notice::Evidence found in a code block (11 line(s)).   (rc 0)

Red-first, both pasted from real runs on this branch:

# 1. model reverted to one shared size (max) inside reduced_block_sizes
test block::tests::a_mixed_size_pair_blocks_at_each_operands_own_footprint ... FAILED
  assertion `left == right` failed
    left: [64, 32]
   right: [57, 56]
test block::tests::compute_block_sizes_equals_the_exact_model_per_operand_on_the_sweep ... FAILED
  assertion `left == right` failed: [1000, 3] [[-1, -64], [9223372036854775807, 1]] e=[0, 2]
    left: [467, 3]
   right: [511, 3]
test block::tests::a_zero_sized_source_stops_the_increasing_strides_exit ... FAILED
  left: [2, 1024]  right: [2, 2044]
test block::tests::a_zero_sized_destination_leaves_the_reduction_to_the_source ... FAILED
  left: [2, 8192]  right: [2, 511]
test result: FAILED. 31 passed; 5 failed

# 2. only the wiring reverted to the max (kernel::pointwise_plan)
---- block::tests::the_pointwise_plan_uses_each_operands_element_size stdout ----
  assertion `left == right` failed: [8, 1]
    left: [64, 32]
   right: [57, 56]
test result: FAILED. 253 passed; 1 failed

The second red is the one that matters for "the model is not the only defect class": with the size seam reverted, the end-to-end mixed map_into test still passes (values are correct under either tiling), and only the plan-level test fails. A caller-side &[max; N] literal is not observable through the public entries at all, which is why the pin sits on the shared seam every typed entry feeds verbatim.

Measurement

Shared host primerose (64 cores, other users' jobs running; broccoli was handed over to Terasaki-san on 2026-11-03 and is not dispatched). One L3 domain: the harness is pinned with taskset -c 24, and the agent process was pinned to 0-15 with taskset -apc so it never shared the measured core. Exec::serial() (1 thread, verified: the harness builds only Exec::serial()), release build, identical harness source for both revisions (only the strided-basic path dependency differs), 3 cycles of ABBA (6 samples per arm), medians, boost=1, governor schedutil, load average 3.7-8.9 during the runs.

row before (edcf71f) after (ef0264a) delta
map_into f64<-u8 mixed, 128x128 (block 64x32 -> 57x56) 19.80 µs 10.23 µs -48.4%
map_into f64<-u8 mixed, 2x4096x8 (block 2x1024x1 -> 2x1816x1) 243.7 µs 239.8 µs -1.6%
replay_map_raw f64<-u8 mixed, 2x4096x8 123.4 µs 127.7 µs +3.5%
map_into f64<-f64 same, 128x128 23.32 µs 24.23 µs +3.9%
map_into f64<-f64 same, 2x4096x8 234.3 µs 237.0 µs +1.2%
replay_map_raw f64<-f64 same, 2x4096x8 108.1 µs 113.5 µs +5.0%

The three same-size rows run provably identical code (the sweep test proves the block vectors are unchanged), so their +1.2% to +5.0% spread is the harness's drift/alignment floor between two independently compiled binaries. Only the mixed 128x128 map is outside it; the two mixed 2x4096x8 rows are inside it. copy_into is single-dtype by contract (dest: &mut StridedViewMut<T>, src: &StridedView<T, Op>), so its mixed-size form is the generic map_into row above; replay_copy_raw is single-dtype too.

Review

review: gpt-6.1-sol pre-review (plan) — 1 blocker, 2 important, 1 minor: the mixed-ZST exit rule was wrong (fixed: the increasing-strides exit now requires every size positive), copy_into is single-dtype so the mixed row is stated as map_into, crossing the budget alone does not change blocks so the fixtures use non-fusing transposed strides, and the five-helper family became one non-generic helper. Nothing left unfixed.
review: gpt-6.1-sol post-review (diff) — 1 important, 3 minor: the important one (public rustdoc linking to a private item, -D warnings docs gate) was already fixed before the report was read; the three minors are applied in this commit (two test names that claimed more than they assert, and stale "larger element size" wording in raw_ops comments). Nothing left unfixed.

…426)

`compute_block_sizes` took one element size for all operands and the callers
passed the largest one, so a mixed-size operation under-used the budget for
its smaller operand: the footprint was overstated and the cache-line rounding
was computed at the wrong size. Strided.jl forms byte strides per array
(`map((s, stride) -> s .* stride, sizeof.(eltype.(arrays)), strides)`).

- `compute_block_sizes`, `reduced_block_sizes`, `memory_region`, `region_in`
  and `Rows` take `elem_sizes: &[usize]`, one per entry of `strides_list`, and
  form each operand's byte strides, region and cache-line rounding from its
  own size. Stride orders and costs stay on the element strides, as the
  reference does (orders are unchanged by a positive multiplier; a zero size
  gives order 1 on every axis).
- The exit that keeps every axis now needs every element size positive: a
  zero-size operand's constant order 1 is the global minimum at every
  recursion level, so another operand's larger order no longer matches it. All
  zero sizes still keep the full dimensions.
- `execution::build_plan_fused(..., elem_size: usize)` stays as the
  compatibility wrapper and fills a uniform slice, so `strided-fused` is
  unchanged and nothing public is added.
- Typed callers pass `size_of` of each operand in `strides_list` order through
  one `kernel::pointwise_plan`; `raw_ops` passes both sizes to `tile_blocks`
  and `exits_before_metadata`, which now agree with the no-reuse check they
  sit beside.

Tests: `model_blocks_sized` is the per-operand oracle; the sweep runs again
with per-operand sizes, the same-size sweep is unchanged, and three ZST cases
pin the contract. Checked with Strided.jl 2.6.6 itself (per-array
`sizeof`): 2x4096x8 f64<-u8 gives (2, 1816, 1), 128x128 gives (57, 56),
128x128 c64<-f64 gives (40, 32), matching the implementation.
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

Repository rules review

Repository rules review (edcf71fafb91f949553fb3f7d6df94af6a0d6e04...2b0ef866361f2d5e83487c8725d67449a325cc56)
Verdict: pass
Findings:
- [warn] llm-skipped (External LLM Review) <unknown>: External LLM review was skipped
  External LLM review is permanently disabled; deterministic checks only.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The shared block model sizes every operand with one element size; Strided.jl uses each array's own

1 participant