Skip to content

perf: choose the tiled layout copy on the working set, not the layout (#323) - #330

Draft
shinaoka wants to merge 8 commits into
mainfrom
fix/323-tiled-copy-by-working-set
Draft

shinaoka wants to merge 8 commits into
mainfrom
fix/323-tiled-copy-by-working-set

Conversation

@shinaoka

@shinaoka shinaoka commented Nov 8, 2026 •

Copy link
Copy Markdown
Member

Refs #323, deliberately not Closes: the working-set half of the proposal is here, the ragged-edge half is not, so the issue stays open. What this buys is the byte-count gate below and the elementwise walk under it, which removes the measured loss at n = 255, 257 and 511; what it leaves is the ragged cell the threshold cannot separate, 513 x 513 at +3.6% over its six cells (five of them slower, #290), and the 256 x 256 win at -2.4% that also sits below the boundary.

What

copy_lower_across_layouts — the tiled copy Llt::new makes of a row-major
input (MatMut::copy_from_triangular_lower, faer/src/mat/matmut.rs:991) — took
the tiles on the layout plus a 64 x 64 element floor, so it staged a copy
through [Option<T>; 16 * 16] even where the whole pair of buffers fits in
cache and the staging only added work. It now takes them only when the working
set does not fit.

The gate is a byte count, in one place:

  • TILE_COPY_CACHE_BYTES (faer/src/linalg/mod.rs:88) = 2 * 512 * 512 * 8 =
    4 MiB, the 512 x 512 f64 pair — the first cell of tiled copy_across_layouts should be chosen on the working-set size, not on the layout alone #323's table where the
    tiles win (−13.9%); the cell below it, 511 x 511 (4 177 936 bytes), still
    loses (+3.1 to +4.7%).
  • tile_copy_pays::<T>(nrows, ncols) (faer/src/linalg/mod.rs:94) compares
    (src + dst element counts) * size_of::<T>() against it, >= so that the
    crossover cell itself keeps the tiles.
  • Not a side and not a host constant: the boundary follows size_of::<T>()
    (f32 crosses at 725 x 725, f64 at 512 x 512), and nothing reads L2/L3
    at run time.

Branch touched

Only the non-conjugated arm of MatMut::copy_from_triangular_lower
(faer/src/mat/matmut.rs:988-1000) reaches the changed gate, through
copy_lower_across_layouts (faer/src/linalg/mod.rs:307); the layout guards
above it are unchanged, so a column-major input reaches it only through a
transposed view -- and Llt::new with Side::Upper copies A.adjoint()
(faer/src/linalg/solvers.rs:986-989), whose row stride is n for a
column-major A, so such a call at n < 512 moves from the tiles to the
elementwise walk: same values, different instruction path, and #323s order=f row does not cover that route (Ldlt/Lbltat:1035-1037and:1091-1093have the same one). The other callers of that arm areLdlt::new/Lblt::new (faer/src/linalg/solvers.rs:1036, :1092), eigh's tridiagonal copy (faer/src/linalg/evd/mod.rs:359) and solvers.rs:634; their dispatch narrows the same way (same result — a copy is a copy, no arithmetic — not the same instruction path). Through copy_from_triangular_upper the same gate is also reachable the other way round, because it transposes both views (faer/src/mat/matmut.rs:1022-1025): qr::no_pivoting::factor::copy_r (faer/src/linalg/qr/no_pivoting/factor.rs:497) and lq::no_pivoting::factor::copy_l (faer/src/linalg/lq/no_pivoting/factor.rs:98) therefore change dispatch too -- both routes are pinned by tests that would pass on values alone (copy_r_reaches_the_tiled_copy_gate, copy_l_reaches_the_tiled_copy_gate, see Tests) -- a column-major packed matrix with a row-major destination at n < 512moves from the tiles to the elementwise walk.MatRef::to_owned (faer/src/mat/matref.rs:1633) and the lu/qradaptations keep their layout gate: no measurement covers a tiles-against-elementwise choice there, and the source-layout numbers inmatref.rs:1621` are not that comparison.

Evidence

The numbers are #323's table (itself #290's 54 CSVs / 1260 cells). No
performance claim of our own
: nothing was re-measured for this change, so read
the table as the issue's, on the machine it was taken on.

That also means the thresholded candidate has no acceptance measurement of its
own, which docs/adoption-protocol.md:55-60 asks for before the row is accepted.
The fbench run at revision 7afe128 was attempted on the 32-core development
host (EVO-X2, AMD RYZEN AI MAX+ 395, 16 cores x SMT2, 2 x 32 MiB L3) and not
taken
. The blockers, each checked rather than assumed:

  • The contract cannot start on this host. It requires CPU boost off
    (docs/adoption-protocol.md:22-26, faer-benchmark/docs/protocol-v2-baseline.md:37-38,
    which says the recorder refuses while a boost switch is on) and
    /sys/devices/system/cpu/cpufreq/boost reads 1. Clearing it is a host-wide
    power-policy change, and this host was running another workload at the same
    time (load average 34 -> 8 during the session, tensor4all_tree at 87% CPU), so
    it was not cleared on my own authority.
  • The recorder is not installed here. scripts/record_run.py:10 imports
    campaign_toolkit, which is not present; it is a git dependency in
    faer-benchmark/requirements.txt and installing it belongs to that harness, not
    to this repository.
  • There is no baseline to pair against. candidate_compare.py compares a
    candidate run with a recorded baseline at a comparable revision; the newest
    baseline-ref-* runs in faer-benchmark/data/results/zen5-cpu/faer-kernels/
    are from 2026-10-16, while the pinned fork revision is 325e482, so a valid
    comparison would need the fork's main recorded on this host first (another
    contract run of about an hour).

So the acceptance row of docs/adoption-protocol.md:55-60 stays 未測定: no
fbench verify, no candidate_compare.py output, no target-cell or guard-cell
number of this revision. What is measured here is correctness only
(cargo test -p t4a-faer --lib: 696 passed, 0 failed, 162 ignored), and that is
not a substitute for the acceptance run. The PR therefore stays a draft;
merging it without that run would break the protocol this repository states.

cargo test (whole workspace, 36 test binaries): 967 passed, 0 failed, 162 ignored.

Tests:

  • linalg::tests::tile_copy_threshold_is_the_measured_crossover — the constant
    is the 512 x 512 f64 pair; 511^2 below, 512^2 at it, one element either
    side; f32 crosses at 725^2 and not at the f64 shape; and 513^2 is asserted
    to be above it, so the cost this PR keeps is pinned rather than untested.
  • linalg::tests::tile_copy_is_chosen_on_the_working_set_not_on_the_layout —
    the same transposed layout, 256 x 256 → elementwise and 512 x 512 → tiles,
    plus both rectangles (511 x 520, 1024 x 256), asserted both on
    tile_copy_pays and on what copy_lower_across_layouts actually returns — so
    a shape that silently fell back to the elementwise walk turns the test red.
  • tiled_triangular_copy_writes_what_the_elementwise_one_does — its shape list
    had nothing above 4 MiB, so it could have kept passing on the elementwise path
    alone; it now covers both sides of the threshold, ragged, tall, wide and
    reversed-row shapes, for f64 and c64.
  • linalg::qr::no_pivoting::factor::tests::copy_r_reaches_the_tiled_copy_gate
    and linalg::lq::no_pivoting::factor::tests::copy_l_reaches_the_tiled_copy_gate
    -- the two public factor routes named in "Branch touched", each at the
    threshold (512 x 512 f64) and in the layout that reaches the gate, with the
    copied triangle checked against the source and the tile decision read from the
    test-only TILED_COPIES counter (faer/src/linalg/mod.rs:100): equal values
    alone cannot tell the tiled path from the elementwise one.

Mutation checks run at 7afe128:

mutation test that fails observed
let k = Ord::min(m, n) -> let k = n in copy_lower_across_layouts tiled_triangular_copy_writes_what_the_elementwise_one_does Assertion failed: row < self.nrows() (row = 17, nrows = 17), i.e. the 17 x 16384 fixture walks past the last row
gate forced to fall back (dst.row_stride() != 1 -> always true) copy_r_reaches_the_tiled_copy_gate, copy_l_reaches_the_tiled_copy_gate both fail with "copy_r/copy_l did not reach the tiled copy gate"

PATCHSET.md's copy_from_triangular_lower row records the rule and why the
other two tiled sites stay as they are.

Verification

Correctness, run locally at 7afe128 on the 32-core development host (rustc 1.96.0, the
rust-toolchain.toml pin):

cargo test -p t4a-faer --lib
  test result: ok. 696 passed; 0 failed; 162 ignored
cargo test -p t4a-faer --lib reaches_the_tiled_copy_gate
  test linalg::lq::no_pivoting::factor::tests::copy_l_reaches_the_tiled_copy_gate ... ok
  test linalg::qr::no_pivoting::factor::tests::copy_r_reaches_the_tiled_copy_gate ... ok

Mutations applied, observed failing, and reverted:

mutation failing test observed output
let k = Ord::min(m, n) -> let k = n (faer/src/linalg/mod.rs:322) tiled_triangular_copy_writes_what_the_elementwise_one_does Assertion failed: row < self.nrows() with row = 17, nrows = 17
the gate forced to fall back (dst.row_stride() != 1 -> true) both *_reaches_the_tiled_copy_gate tests copy_r did not reach the tiled copy gate, same for copy_l

Not measured: the performance acceptance of docs/adoption-protocol.md:55-60
(no fbench verify, no candidate_compare.py, no target or guard cell of this
revision). It was attempted on this host and could not start: the contract needs
CPU boost off and /sys/devices/system/cpu/cpufreq/boost reads 1, which is a
host-wide power-policy change on a machine that was running another workload at
the time (load average 34 -> 8, tensor4all_tree at 87% CPU); the recorder
scripts/record_run.py:10 cannot import campaign_toolkit, which is not
installed here; and no baseline-ref-* run at a comparable revision exists
(newest are 2026-10-16, against a 325e482 pin), so there is nothing for the
comparator to pair the candidate with. Correctness tests and the numbers above do
not stand in for that run, so the PR stays a draft.

Not done

The ragged-edge idea in the issue (keep the elementwise path when the tile loop
is ragged on both axes) is not implemented: the rule here is the byte count.
Two edges are therefore approximate: a ragged 513 x 513 is above the threshold
and keeps the tiles although #290's table records +3.6% against them there over
its six cells, five of them slower, and 256 x 256 is below it although the same
table records a 2.4% win. Raising the threshold to drop 513 x 513 is not the
fix and is not done: at 4210704 bytes it sits just above 512 x 512's 4194304,
so any byte boundary above it puts 512 x 512 below it too, and all twelve
512 x 512 cells win by 13.9% (#290). A monotone byte count therefore cannot
move that cell alone; that its cost comes from n % 16 is the issue's reading,
not something this gate measured, which is why #323 stays open rather than being
closed by this PR.

review: gpt-6.1-sol reviewed the finished diff (verdict: no correctness defect;
its one IMPORTANT finding — that "other families untouched" overstates the blast
radius — is corrected above and in the ledger row; its two delete-list items are
applied: tile_copy_pays is module-private and the unmeasured universal
cache-miss sentence is gone).
review round 2: gpt-6.1-sol, CRITICAL 0 / NEEDS_CHANGES 3 / UNVERIFIED 3 —
applied: the QR/LQ routes are now in the caller list (verified: copy_from_triangular_upper transposes both views at matmut.rs:1022-1025), the wide/tall above-threshold fixtures are added and the Ord::min(m, n) -> n mutation was applied and does fail them, the ragged 513 and the 256 win are stated next to the constant, and #323 is no longer closed. Open: acceptance measurement (the draft note above) and the ledger's 96/128 loss sentence, which #290 calls below the harness's resolution.
review round 3: gpt-6.1-sol, 1 IMPORTANT / 3 MINOR — applied: the reason the
threshold is not raised is corrected from "the working sets are 0.4% apart, so
the cause is n % 16" to what the arithmetic actually forces (any byte boundary
above 513^2 also drops 512^2, which wins), raggedness is labelled #323's
reading, the wrong geometric explanation of the rectangle is deleted, the ledger
row's 255..511 range (which contained the 256 win) is listed as 255, 257, 511 with 96/128 marked unproven, and the dispatch assertion covers both
rectangles. Not applied: duplicating the reversed-row cases in the dispatch
assertion — the value test already runs every shape flipped, and the helper does
not read the flip.

…#323)

`copy_lower_across_layouts` took the tiled route on the layout plus a
64x64 element floor, so it staged a copy through `[Option<T>; 16 * 16]`
even where the whole pair of buffers fits in cache and the staging only
adds work (#323: +2.0 to +5.2% at n = 96..511) -- and it
keeps that route where the working set no longer fits, which is where the
tiles win (-13.9% at 512, -18.2% at 1024, -16.4% at 1536, C-order
`Llt::new`).

The gate is now the bytes copied over both buffers
(`tile_copy_pays::<T>`), against `TILE_COPY_CACHE_BYTES` = the
`512 x 512` f64 pair = 4 MiB, the first cell of that table where the
tiles win, with `511 x 511` (4177936 bytes) just below it. A byte count
and not a side, so the boundary follows `size_of::<T>()` (f32 crosses at
725 x 725) and stays host independent.

No new measurement: the numbers are the issue's table, which is why the
threshold is pinned to its crossover rather than probed at runtime.
@shinaoka

shinaoka commented Nov 8, 2026

Copy link
Copy Markdown
Member Author

GPT-6.1-sol adversarial review

CRITICAL: none
NEEDS_CHANGES: 2
UNVERIFIED: 2

Critical (must not merge)

none

Needs changes

  • The gate retains a measured losing case while the PR closes tiled copy_across_layouts should be chosen on the working-set size, not on the layout alone #323. faer/src/linalg/mod.rs:310-312; PATCHSET.md:126: evaluate tile_copy_pays::<f64>(513, 513)—the result is true because 2 * 513 * 513 * 8 = 4,210,704, exceeding the threshold, yet the cited tiled copy_across_layouts should be chosen on the working-set size, not on the layout alone #323 table explicitly places 513 among the cases where tiling loses 3.1–4.7%. The ledger’s blanket claim that tiles win 13.9–18.2% at n = 512..1536 therefore contradicts its source; its below-threshold loss claim also omits the table’s 256 win of 2.4%. Implement the outstanding ragged-case handling, or stop closing the whole issue and restrict the ledger to the actual measured cells.

  • The claim that an order=f input never reaches the changed gate is false for real upper-triangle factorizations. faer/src/linalg/mod.rs:310-312; faer/src/linalg/solvers.rs:987-989: trace Llt::new(A.as_ref(), Side::Upper, Par::Seq) with a column-major 256 × 256 f64 identity matrix—A.adjoint() has row-major strides, and conjugation remains canonical for real scalars (faer-traits/src/lib.rs:1161-1165), so it reaches this gate and switches from tiled to elementwise copying. The same route exists at solvers.rs:1035-1037 and :1091-1093 for Ldlt and Lblt. Correct the PR’s blast-radius claim; unchanged layout guards do not establish unchanged dispatch for the original input layout.

Unverified claims

  • The measured f64 crossover does not establish a byte-scaled performance crossover for other types or shapes. faer/src/linalg/mod.rs:78-98, :1032-1046: reproduce the assertions with tile_copy_pays::<f32>(725, 725) and tile_copy_pays::<c64>(363, 363); both select tiles, but the cited issue table supplies neither measurement. The tests verify the chosen formula, not the claim that this is where staging “pays” or the elementwise walk stops fitting in cache. Those performance statements remain unverified extrapolations.

  • The “working set” calculation includes columns the triangular copy never accesses. faer/src/linalg/mod.rs:94-98, :316-328: trace a row-major 17 × 16384 f64 source into a column-major destination—the gate counts 4,456,448 bytes and selects tiles, while the lower-triangle loop copies only 17 * 18 / 2 = 153 entries, or 2,448 bytes of source-plus-destination element payload; columns 17..16384 are untouched. Thus the assertion that this decision identifies an out-of-cache copied working set is not established for the rectangular inputs accepted by this function. No performance measurement for this case was verified.

Checked and fine

CI run 42467818912 matches head SHA 5af5a41e759c88dfd1588b482e800d29452981ed; its logs show all three relevant copy tests passed, with 162 tests skipped overall.
Not checked: local test execution, mutation testing, or performance reruns; the reported slowdown above is the cited issue’s result, not a new measurement.


Automated (gpt_review.sh). This is an enumeration of counterexamples, not a verdict; the merge decision stays with a human.

@shinaoka

shinaoka commented Nov 8, 2026

Copy link
Copy Markdown
Member Author

GPT-6.1-sol adversarial review

CRITICAL: none
NEEDS_CHANGES: 3
UNVERIFIED: 1

Critical (must not merge)

none

Needs changes

  • The changed candidate has no required performance acceptance evidence — faer/src/linalg/mod.rs:312; docs/adoption-protocol.md:32–38,55–62. The PR explicitly says nothing was re-measured, whereas the repository requires measurements of the candidate revision across the affected populations. Reproduce the evidence gap by following the PR’s links: the #282 tiled triangular copy was adopted on a measurement that does not meet the adoption protocol #290 measures the earlier tiled implementation, not this thresholded candidate, and the linked worklog contains only the earlier experiment. The successful test workflow runs correctness coverage, not fbench verify or candidate_compare.py. Record acceptance for this candidate; the disclaimer does not satisfy that gate.

  • The threshold retains a documented losing case, contradicting the ledger’s above-threshold claim — faer/src/linalg/mod.rs:94–98; PATCHSET.md:126. Reproduce the dispatch with tile_copy_pays::<f64>(513, 513): 513² × 16 = 4,210,704, so it selects tiles, but the #282 tiled triangular copy was adopted on a measurement that does not meet the adoption protocol #290 reports +3.6% for that size, with five of six cells more than 2% slower. This is a retained, previously measured cost—not a newly measured regression. The ledger’s statement that tiles win at sizes “512..1536” omits this counterexample, and Closes #323 closes the issue despite leaving its explicit ragged-edge case unresolved. Either address that case or describe this as a partial fix and retain the unresolved issue.

  • The revised tests stop exercising wide and narrow triangular copies through the tiled branch — faer/src/linalg/mod.rs:1013–1027,1065–1067. Reproduce the coverage gap by partitioning the shape list on tiled: the only tiled cases are (512,512), (520,511), and (600,600); every wide or narrow-band case now takes the fallback. In particular, none reaches the tiled triangular loop with n > m, where columns beyond min(m,n) must remain untouched. The separate dispatch test uses only squares. Add above-threshold wide and narrow cases, such as (17,16384) and (16384,17), with sentinel verification and an assertion that the helper actually selected tiles.

Unverified claims

  • The tests establish a byte-policy boundary, not a measured cache or scalar-type crossover — faer/src/linalg/mod.rs:78–92,1032–1046. The asserted f32 boundary at 725×725 follows arithmetically from the constant; no supplied measurement establishes that staging becomes faster there. Likewise, full rectangular allocation size does not establish the triangular copy’s accessed working set: for f64 (17,16384), the predicate counts 4,456,448 bytes, while the lower-triangle operation copies only 153 elements across the two buffers. Reproduce those counts from the predicate and the loop bounds at :316–328. The comments’ claim that tiles “only win once the elementwise walk stops fitting in cache” remains unverified.

Checked and fine

  • CI run 42467818912 identifies head 5af5a41e759c88dfd1588b482e800d29452981ed; its log records all three affected tests passing, with 871 tests passed and 162 skipped overall.
  • No local builds, mutation tests, or benchmarks were run; candidate performance remains unverified.

Automated (gpt_review.sh). This is an enumeration of counterexamples, not a verdict; the merge decision stays with a human.

… threshold

The threshold keeps the tiles for a ragged 513x513 although #323 records
3.1-4.7% against them there, and drops them for 256x256 although the same
table records a 2.4% win; both are now stated next to the constant and in
the ledger row, and the PR no longer closes the issue's ragged-edge half.
@shinaoka shinaoka self-assigned this Nov 8, 2026
@shinaoka

shinaoka commented Nov 8, 2026

Copy link
Copy Markdown
Member Author

GPT-6.1-sol adversarial review

CRITICAL: none
NEEDS_CHANGES: 3
UNVERIFIED: 3

Critical (must not merge)

none

Needs changes

  • The candidate lacks the repository-required performance acceptance. faer/src/linalg/mod.rs:318; docs/adoption-protocol.md:32-38,55-60: the PR explicitly says nothing was re-measured, but the protocol requires the candidate revision, affected populations, fbench verify, and a comparison against baseline spread. Reproduce the evidence gap by following the #282 tiled triangular copy was adopted on a measurement that does not meet the adoption protocol #290 and docs/worklogs/2026-11-06-row-major-triangular-copy.md:15-23: both describe the earlier implementation, not this thresholded candidate. Record acceptance for the submitted revision; correctness tests and a performance disclaimer do not satisfy that requirement.

  • The caller inventory misses public QR and LQ routes affected by the gate. faer/src/linalg/mod.rs:318; faer/src/linalg/qr/no_pivoting/factor.rs:497; faer/src/linalg/lq/no_pivoting/factor.rs:98: call qr::no_pivoting::factor::copy_r with a column-major 256×256 f64 packed matrix, a row-major output, and positive=false; copy_from_triangular_upper transposes both views at faer/src/mat/matmut.rs:1022-1025, reaching the changed lower-copy gate with an eligible destination/source layout and switching from tiles to fallback. Conversely, lq::no_pivoting::factor::copy_l reaches it directly with row-major packed input and column-major output. Correct the PR’s “other callers” list and acceptance scope; unchanged QR/LU adaptation gates do not exclude these separate routes.

  • The revised fixtures remove tiled-wide coverage and admit a concrete loop-bound mutation. faer/src/linalg/mod.rs:1019-1029: every tiled=true fixture has m >= n, while every wide fixture now takes fallback. Source-derived mutation reproducer: replace let k = Ord::min(m, n) at :322 with let k = n, then run cargo test -p t4a-faer --lib tiled_triangular_copy_writes_what_the_elementwise_one_does; the mutation leaves the tiled fixtures’ bounds unchanged, but an eligible 17×16384 row-major f64 source would index beyond row 16 in the second diagonal band. Add above-threshold wide and narrow fixtures with sentinel checks and explicit tiled-dispatch assertions. This mutation was not executed locally.

Unverified claims

  • Allocation size does not establish the accessed working set or a measured dtype crossover. faer/src/linalg/mod.rs:78-103; PATCHSET.md:126: evaluate the predicate for f64 (17,16384)—it counts 4,456,448 bytes and selects tiles, although the triangular loop copies only 153 elements, totaling 2,448 bytes of source-plus-destination element payload. Likewise, the f32 725×725 and c64 363×363 boundaries follow from arithmetic, not supplied measurements at those boundaries. These examples do not establish a slowdown, but they do prevent treating “bytes copied,” cache fit, and where tiling “pays” as verified equivalents.

  • The ledger presents below-resolution differences as established losses. PATCHSET.md:126 says staging “loses 2.0–5.2% at n = 96, 128,” but the #282 tiled triangular copy was adopted on a measurement that does not meet the adoption protocol #290’s analysis explicitly says those sizes are below the harness’s resolution and their cost is unproven. Reproduce by comparing that ledger sentence with the #282 tiled triangular copy was adopted on a measurement that does not meet the adoption protocol #290’s “First, the resolution of this harness” and “Decision” sections. Retain the reported medians only with their measurement caveat.

  • The reported test count is not substantiated for the current head by the inspected CI evidence. .github/workflows/run-tests.yml:39-44,123-124: the successful MSRV jobs run cargo check, not the changed tests; test run 42470625635 was still in progress for head 71c7c55d73920a9de483f98248d9c7b6de1ae4aa. The successful run cited by earlier review comments, 42467818912, belongs to older head 5af5a41e759c88dfd1588b482e800d29452981ed. Reproduce with gh api repos/tensor4all/faer/actions/runs/<run-id>; cannot read a current-head log substantiating “694 passed, 162 ignored” from the supplied evidence.

Checked and fine

Threshold arithmetic recomputed: 4,194,304 bytes; first square policy boundaries are f32=725, f64=512, c64=363.
Local builds, mutation execution, and benchmarks were not run; candidate performance remains unverified.
kache doctor --verify timed out after 20 seconds; no repair was performed.


Automated (gpt_review.sh). This is an enumeration of counterexamples, not a verdict; the merge decision stays with a human.

The tiled fixtures were all m >= n, so a band bound taken from the wrong
side (Ord::min(m, n) replaced by n) walked columns past the end of nothing
the test looked at. 17x16384 and 16384x17 are above the byte threshold and
fail that mutation (checked by applying it); the wide one also covers the
tiled path's row bound.

review: gpt-6.1-sol round 2, NEEDS_CHANGES 3 -- this is the fixture gap; the
caller list and the acceptance note are in the PR body.
@shinaoka
shinaoka marked this pull request as draft November 8, 2026 10:28
`tile_copy_threshold_is_the_measured_crossover` now pins the cost that is
known and kept: a ragged `513x513` is above the byte threshold and takes the
tiles, and #290's table records 3.6% against them there over its six cells,
five of them slower. The threshold is not raised to exclude it, because no
byte boundary can: at 4210704 bytes it sits just above `512x512`'s 4194304, so
a threshold above it puts `512x512` below it too, and all twelve `512x512`
cells win by 13.9%. The doc comment on the constant and the ledger row say
that, call the `n % 16` cause #323's reading rather than something this gate
measured, and keep the issue open.

`tile_copy_is_chosen_on_the_working_set_not_on_the_layout` only asserted the
dispatch on square shapes; a rectangle in either orientation is now checked
against what `copy_lower_across_layouts` actually returns, on top of the
shapes the value test crosses the threshold with.

The ledger row's losing sizes are listed individually instead of as the
`255..511` range, which included the `256` win, and the `96`/`128` losses are
marked unproven, as #290 records them below that harness's resolution.
@shinaoka
shinaoka force-pushed the fix/323-tiled-copy-by-working-set branch from 2a18b11 to 7c98dd1 Compare November 8, 2026 10:48
shinaoka and others added 3 commits November 9, 2026 00:28
…sions

`qr::no_pivoting::factor::copy_r` and `lq::no_pivoting::factor::copy_l` reach
`copy_lower_across_layouts` through `copy_from_triangular_lower` (directly for
LQ, and through the transposing `copy_from_triangular_upper` for QR), so the byte
threshold changes their instruction path for a `512 x 512` f64 copy. Nothing
observed that: the values are identical either way, and the caller cannot see the
helper's return value.

A `#[cfg(test)]` counter of the copies that took the tiles is what makes "the
gate was reached" assertable, and each route gets a test that would pass on
values alone. Forcing the gate to fall back fails both tests ("copy_r/copy_l did
not reach the tiled copy gate"), so they pin the route rather than restating the
arithmetic.
… wide and tall

The tiled-copy counter is per thread now: the test harness runs tests
concurrently, so a process-wide count let another test's tiled copy satisfy
"tiled" and made "not tiled" unassertable. copy_r and copy_l are pinned at
512 (tiles) and 256 (elementwise), and on a wide and a tall packed matrix on
each side, by the exact change of the counter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@shinaoka

shinaoka commented Nov 8, 2026

Copy link
Copy Markdown
Member Author

Use neither answer as stated: declare the host you measured, and make the mismatch impossible to pass

Neither option is right, because both leave a silent path open.

What actually happened. zen2-cpu declares 128 logical / SMT offset 64. broccoli_vpn is an EPYC 7702P with SMT off =
64 logical
. record_run accepted the run anyway (it does not compare the host class), and idle_cpus check exited 4. So the
run was recorded as zen2-cpu while it was not that host — that is the defect, not the profile.

What I want

  1. Fix record_run first: a host that does not match the declared profile must refuse to write, not pass through. The
    same rule as everywhere else this week — a check that cannot fail where the input is wrong is worse than no check. Its
    own silence is what let a mismatched host write data under a name it does not own.
  2. Add the host as its own profile (zen2-64-smt-off, EPYC 7702P, 64 logical, SMT off, measured), rather than editing
    zen2-cpu to cover two machines. A profile whose meaning changes with the machine it runs on cannot be compared across
    runs, and candidate_compare would then be mixing two host classes under one label.
  3. Run the acceptance on the declared host and say which profile in the PR. If the claim in #330 is specifically about
    128-thread behaviour, then the acceptance measured on 64 logical does not support it — the claim gets narrowed to the
    host it was measured on, or the 128-logical machine has to be found. Do not widen the profile to make the claim fit.
  4. The idle_cpus check exit 4 is correct behaviour — keep it, and make record_run agree with it instead of
    disagreeing.

Why not "just revise the profile"

Revising zen2-cpu to cover 64 logical would make one identifier name two different machines. That is the same shape as
tenferro 0.10.0 naming two different trees, which cost this repository a release today: the number/name stops identifying
its content and every downstream comparison silently mixes populations.

decision: do not revise zen2-cpu; add zen2-64-smt-offfor EPYC 7702P SMT-off, fixrecord_run to refuse a host/profile mismatch, run the acceptance on that profile, and narrow #330's claim to the host actually measured.

@shinaoka

shinaoka commented Nov 9, 2026

Copy link
Copy Markdown
Member Author

GPT-6.1-sol adversarial review

CRITICAL: none
NEEDS_CHANGES: 3
UNVERIFIED: 2

Critical (must not merge)

none

Needs changes

  • Performance acceptance remains missing — faer/src/linalg/mod.rs:338-341; docs/adoption-protocol.md:53-60: the changed dispatch has no candidate acceptance result, so correctness-green CI does not satisfy the repository’s merge requirements. Reproduction: inspect the PR’s Verification section—both fbench verify and candidate_compare.py are explicitly absent. Keep this draft until the final candidate has the required affected-family and guard-cell evidence; the #282 tiled triangular copy was adopted on a measurement that does not meet the adoption protocol #290 measured a different change.

  • The tests cannot detect dispatch using the wrong scalar size — faer/src/linalg/mod.rs:341,1033-1064,1094-1116: replacing the gate’s tile_copy_pays::<T>(m, n) with tile_copy_pays::<f64>(m, n) would violate the new type-dependent contract without changing any new actual-copy fixture’s expected decision. Reproduction: make that one-call-site mutation and run the added tests; inspection and an arithmetic check establish that all 12 value-test shapes and all four direct-dispatch shapes classify identically for f64 and c64, while the f32 assertions test only the unmodified helper. Add an actual dispatch assertion at, for example, 363 × 363, where c64 must tile and f64 must not, plus an actual f32 dispatch boundary case. The mutation build itself was not executed in this review.

  • The claimed caller inventory still omits reachable reconstruction routes — faer/src/linalg/mod.rs:341; faer/src/linalg/qr/no_pivoting/reconstruct.rs:37; faer/src/linalg/lq/no_pivoting/reconstruct.rs:26: the PR’s “other callers” list excludes public reconstruction functions whose dispatch also changes, leaving the stated blast radius incomplete. Reproduction: call QR reconstruction with column-major R and a row-major 256 × 256 output; the upper-triangle copy transposes both views into the changed lower-copy gate, which previously passed the 4,096-element floor but now rejects the 1,048,576-byte pair; LQ reconstruction likewise reaches it with row-major packed input and column-major output. Include these routes in the caller inventory and acceptance population; the new copy_r/copy_l tests do not exercise reconstruction.

Unverified claims

  • A byte-count boundary is not a measured cache-fit boundary for every type or shape — faer/src/linalg/mod.rs:78-87,118-127; PATCHSET.md:126: the arithmetic establishes the policy, not the assertion that staging “only” wins after the elementwise walk stops fitting in cache. Reproduction: the added 17 × 16384 f64 fixture passes the gate using 4,456,448 bytes, although the lower copy touches only 153 entries per buffer—2,448 payload bytes across both; similarly, c64 at 511 × 511 remains tiled. No candidate measurement establishes those decisions’ performance, and the cited aggregate table does not establish the extrapolated f32 crossover. Describe full rectangular buffer size as a proxy, not measured bytes touched or demonstrated cache residency.

  • “All twelve … win by 13.9%” converts a group median into a per-cell claim — faer/src/linalg/mod.rs:94-96; PATCHSET.md:126: the #282 tiled triangular copy was adopted on a measurement that does not meet the adoption protocol #290 explicitly labels −13.9% as the median over the twelve 512 × 512 cells and separately reports twelve cells more than 2% faster. Reproduction: compare those sentences against the #282 tiled triangular copy was adopted on a measurement that does not meet the adoption protocol #290’s “Cholesky, median over cells in the group” table; it does not show that each cell improves by 13.9%. Write “all twelve improve; median improvement 13.9%.” Cannot read the underlying 54 CSVs from the supplied review material.

Checked and fine

  • 2 × 512² × 8 = 4,194,304; the stated f64 boundary and f32 724/725 arithmetic match the implementation.
  • CI run 42505366477 includes head 829aa3e; its log records all five added tests passing, with 874 tests passed and 162 skipped.
  • Not rerun: local Cargo tests, mutation builds, or benchmarks; performance acceptance remains unverified.

Automated (gpt_review.sh). This is an enumeration of counterexamples, not a verdict; the merge decision stays with a human.

`tile_copy_pays`'s doc claimed the tiles win once the elementwise walk stops
fitting in cache. One point of that boundary was measured -- the f64 512 x 512
pair of #323's table -- and the f32/c64 crossings follow from the byte count
and `size_of::<T>()`, which is arithmetic. The predicate also counts both
buffers whole, so where `n > m` it is an upper bound on what the copy touches:
the copy stops at `min(m, n)` columns. Say both instead of leaving them to be
inferred.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant