Repository navigation
Conversation
…#323) `copy_lower_across_layouts` took the tiled route on the layout plus a 64x64 element floor, so it staged a copy through `[Option<T>; 16 * 16]` even where the whole pair of buffers fits in cache and the staging only adds work (#323: +2.0 to +5.2% at n = 96..511) -- and it keeps that route where the working set no longer fits, which is where the tiles win (-13.9% at 512, -18.2% at 1024, -16.4% at 1536, C-order `Llt::new`). The gate is now the bytes copied over both buffers (`tile_copy_pays::<T>`), against `TILE_COPY_CACHE_BYTES` = the `512 x 512` f64 pair = 4 MiB, the first cell of that table where the tiles win, with `511 x 511` (4177936 bytes) just below it. A byte count and not a side, so the boundary follows `size_of::<T>()` (f32 crosses at 725 x 725) and stays host independent. No new measurement: the numbers are the issue's table, which is why the threshold is pinned to its crossover rather than probed at runtime.
GPT-6.1-sol adversarial reviewCRITICAL: none Critical (must not merge)none Needs changes
Unverified claims
Checked and fineCI run Automated ( |
GPT-6.1-sol adversarial reviewCRITICAL: none Critical (must not merge)none Needs changes
Unverified claims
Checked and fine
Automated ( |
… threshold The threshold keeps the tiles for a ragged 513x513 although #323 records 3.1-4.7% against them there, and drops them for 256x256 although the same table records a 2.4% win; both are now stated next to the constant and in the ledger row, and the PR no longer closes the issue's ragged-edge half.
GPT-6.1-sol adversarial reviewCRITICAL: none Critical (must not merge)none Needs changes
Unverified claims
Checked and fineThreshold arithmetic recomputed: 4,194,304 bytes; first square policy boundaries are f32=725, f64=512, c64=363. Automated ( |
The tiled fixtures were all m >= n, so a band bound taken from the wrong side (Ord::min(m, n) replaced by n) walked columns past the end of nothing the test looked at. 17x16384 and 16384x17 are above the byte threshold and fail that mutation (checked by applying it); the wide one also covers the tiled path's row bound. review: gpt-6.1-sol round 2, NEEDS_CHANGES 3 -- this is the fixture gap; the caller list and the acceptance note are in the PR body.
`tile_copy_threshold_is_the_measured_crossover` now pins the cost that is known and kept: a ragged `513x513` is above the byte threshold and takes the tiles, and #290's table records 3.6% against them there over its six cells, five of them slower. The threshold is not raised to exclude it, because no byte boundary can: at 4210704 bytes it sits just above `512x512`'s 4194304, so a threshold above it puts `512x512` below it too, and all twelve `512x512` cells win by 13.9%. The doc comment on the constant and the ledger row say that, call the `n % 16` cause #323's reading rather than something this gate measured, and keep the issue open. `tile_copy_is_chosen_on_the_working_set_not_on_the_layout` only asserted the dispatch on square shapes; a rectangle in either orientation is now checked against what `copy_lower_across_layouts` actually returns, on top of the shapes the value test crosses the threshold with. The ledger row's losing sizes are listed individually instead of as the `255..511` range, which included the `256` win, and the `96`/`128` losses are marked unproven, as #290 records them below that harness's resolution.
2a18b11 to
7c98dd1
Compare
…sions
`qr::no_pivoting::factor::copy_r` and `lq::no_pivoting::factor::copy_l` reach
`copy_lower_across_layouts` through `copy_from_triangular_lower` (directly for
LQ, and through the transposing `copy_from_triangular_upper` for QR), so the byte
threshold changes their instruction path for a `512 x 512` f64 copy. Nothing
observed that: the values are identical either way, and the caller cannot see the
helper's return value.
A `#[cfg(test)]` counter of the copies that took the tiles is what makes "the
gate was reached" assertable, and each route gets a test that would pass on
values alone. Forcing the gate to fall back fails both tests ("copy_r/copy_l did
not reach the tiled copy gate"), so they pin the route rather than restating the
arithmetic.
… wide and tall The tiled-copy counter is per thread now: the test harness runs tests concurrently, so a process-wide count let another test's tiled copy satisfy "tiled" and made "not tiled" unassertable. copy_r and copy_l are pinned at 512 (tiles) and 256 (elementwise), and on a wide and a tall packed matrix on each side, by the exact change of the counter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Use neither answer as stated: declare the host you measured, and make the mismatch impossible to passNeither option is right, because both leave a silent path open. What actually happened. What I want
Why not "just revise the profile"Revising
|
GPT-6.1-sol adversarial reviewCRITICAL: none Critical (must not merge)none Needs changes
Unverified claims
Checked and fine
Automated ( |
`tile_copy_pays`'s doc claimed the tiles win once the elementwise walk stops fitting in cache. One point of that boundary was measured -- the f64 512 x 512 pair of #323's table -- and the f32/c64 crossings follow from the byte count and `size_of::<T>()`, which is arithmetic. The predicate also counts both buffers whole, so where `n > m` it is an upper bound on what the copy touches: the copy stops at `min(m, n)` columns. Say both instead of leaving them to be inferred.
Refs #323, deliberately not
Closes: the working-set half of the proposal is here, the ragged-edge half is not, so the issue stays open. What this buys is the byte-count gate below and the elementwise walk under it, which removes the measured loss at n = 255, 257 and 511; what it leaves is the ragged cell the threshold cannot separate,513 x 513at +3.6% over its six cells (five of them slower, #290), and the256 x 256win at -2.4% that also sits below the boundary.What
copy_lower_across_layouts— the tiled copyLlt::newmakes of a row-majorinput (
MatMut::copy_from_triangular_lower,faer/src/mat/matmut.rs:991) — tookthe tiles on the layout plus a
64 x 64element floor, so it staged a copythrough
[Option<T>; 16 * 16]even where the whole pair of buffers fits incache and the staging only added work. It now takes them only when the working
set does not fit.
The gate is a byte count, in one place:
TILE_COPY_CACHE_BYTES(faer/src/linalg/mod.rs:88) =2 * 512 * 512 * 8=4 MiB, the
512 x 512f64 pair — the first cell of tiled copy_across_layouts should be chosen on the working-set size, not on the layout alone #323's table where thetiles win (−13.9%); the cell below it,
511 x 511(4 177 936 bytes), stillloses (+3.1 to +4.7%).
tile_copy_pays::<T>(nrows, ncols)(faer/src/linalg/mod.rs:94) compares(src + dst element counts) * size_of::<T>()against it,>=so that thecrossover cell itself keeps the tiles.
size_of::<T>()(f32 crosses at
725 x 725, f64 at512 x 512), and nothing readsL2/L3at run time.
Branch touched
Only the non-conjugated arm of
MatMut::copy_from_triangular_lower(
faer/src/mat/matmut.rs:988-1000) reaches the changed gate, throughcopy_lower_across_layouts(faer/src/linalg/mod.rs:307); the layout guardsabove it are unchanged, so a column-major input reaches it only through a
transposed view -- and
Llt::newwithSide::UppercopiesA.adjoint()(
faer/src/linalg/solvers.rs:986-989), whose row stride isnfor acolumn-major
A, so such a call atn < 512moves from the tiles to theelementwise walk: same values, different instruction path, and #323
sorder=frow does not cover that route (Ldlt/Lbltat:1035-1037and:1091-1093have the same one). The other callers of that arm areLdlt::new/Lblt::new(faer/src/linalg/solvers.rs:1036,:1092),eigh's tridiagonal copy (faer/src/linalg/evd/mod.rs:359) andsolvers.rs:634; their dispatch narrows the same way (same result — a copy is a copy, no arithmetic — not the same instruction path). Throughcopy_from_triangular_upperthe same gate is also reachable the other way round, because it transposes both views (faer/src/mat/matmut.rs:1022-1025):qr::no_pivoting::factor::copy_r(faer/src/linalg/qr/no_pivoting/factor.rs:497) andlq::no_pivoting::factor::copy_l(faer/src/linalg/lq/no_pivoting/factor.rs:98) therefore change dispatch too -- both routes are pinned by tests that would pass on values alone (copy_r_reaches_the_tiled_copy_gate,copy_l_reaches_the_tiled_copy_gate, see Tests) -- a column-major packed matrix with a row-major destination atn < 512moves from the tiles to the elementwise walk.MatRef::to_owned(faer/src/mat/matref.rs:1633) and thelu/qradaptations keep their layout gate: no measurement covers a tiles-against-elementwise choice there, and the source-layout numbers inmatref.rs:1621` are not that comparison.Evidence
The numbers are #323's table (itself #290's 54 CSVs / 1260 cells). No
performance claim of our own: nothing was re-measured for this change, so read
the table as the issue's, on the machine it was taken on.
That also means the thresholded candidate has no acceptance measurement of its
own, which
docs/adoption-protocol.md:55-60asks for before the row is accepted.The
fbenchrun at revision7afe128was attempted on the 32-core developmenthost (
EVO-X2, AMD RYZEN AI MAX+ 395, 16 cores x SMT2, 2 x 32 MiB L3) and nottaken. The blockers, each checked rather than assumed:
(
docs/adoption-protocol.md:22-26,faer-benchmark/docs/protocol-v2-baseline.md:37-38,which says the recorder refuses while a boost switch is on) and
/sys/devices/system/cpu/cpufreq/boostreads1. Clearing it is a host-widepower-policy change, and this host was running another workload at the same
time (load average 34 -> 8 during the session,
tensor4all_treeat 87% CPU), soit was not cleared on my own authority.
scripts/record_run.py:10importscampaign_toolkit, which is not present; it is a git dependency infaer-benchmark/requirements.txtand installing it belongs to that harness, notto this repository.
candidate_compare.pycompares acandidate run with a recorded baseline at a comparable revision; the newest
baseline-ref-*runs infaer-benchmark/data/results/zen5-cpu/faer-kernels/are from 2026-10-16, while the pinned fork revision is
325e482, so a validcomparison would need the fork's
mainrecorded on this host first (anothercontract run of about an hour).
So the acceptance row of
docs/adoption-protocol.md:55-60stays 未測定: nofbench verify, nocandidate_compare.pyoutput, no target-cell or guard-cellnumber of this revision. What is measured here is correctness only
(
cargo test -p t4a-faer --lib: 696 passed, 0 failed, 162 ignored), and that isnot a substitute for the acceptance run. The PR therefore stays a draft;
merging it without that run would break the protocol this repository states.
cargo test(whole workspace, 36 test binaries): 967 passed, 0 failed, 162 ignored.Tests:
linalg::tests::tile_copy_threshold_is_the_measured_crossover— the constantis the
512 x 512f64 pair;511^2below,512^2at it, one element eitherside; f32 crosses at
725^2and not at the f64 shape; and513^2is assertedto be above it, so the cost this PR keeps is pinned rather than untested.
linalg::tests::tile_copy_is_chosen_on_the_working_set_not_on_the_layout—the same transposed layout,
256 x 256→ elementwise and512 x 512→ tiles,plus both rectangles (
511 x 520,1024 x 256), asserted both ontile_copy_paysand on whatcopy_lower_across_layoutsactually returns — soa shape that silently fell back to the elementwise walk turns the test red.
tiled_triangular_copy_writes_what_the_elementwise_one_does— its shape listhad nothing above 4 MiB, so it could have kept passing on the elementwise path
alone; it now covers both sides of the threshold, ragged, tall, wide and
reversed-row shapes, for f64 and c64.
linalg::qr::no_pivoting::factor::tests::copy_r_reaches_the_tiled_copy_gateand
linalg::lq::no_pivoting::factor::tests::copy_l_reaches_the_tiled_copy_gate-- the two public factor routes named in "Branch touched", each at the
threshold (
512 x 512f64) and in the layout that reaches the gate, with thecopied triangle checked against the source and the tile decision read from the
test-only
TILED_COPIEScounter (faer/src/linalg/mod.rs:100): equal valuesalone cannot tell the tiled path from the elementwise one.
Mutation checks run at
7afe128:let k = Ord::min(m, n)->let k = nincopy_lower_across_layoutstiled_triangular_copy_writes_what_the_elementwise_one_doesAssertion failed: row < self.nrows()(row = 17,nrows = 17), i.e. the17 x 16384fixture walks past the last rowdst.row_stride() != 1-> always true)copy_r_reaches_the_tiled_copy_gate,copy_l_reaches_the_tiled_copy_gatePATCHSET.md'scopy_from_triangular_lowerrow records the rule and why theother two tiled sites stay as they are.
Verification
Correctness, run locally at
7afe128on the 32-core development host (rustc 1.96.0, therust-toolchain.tomlpin):Mutations applied, observed failing, and reverted:
let k = Ord::min(m, n)->let k = n(faer/src/linalg/mod.rs:322)tiled_triangular_copy_writes_what_the_elementwise_one_doesAssertion failed: row < self.nrows()withrow = 17,nrows = 17dst.row_stride() != 1->true)*_reaches_the_tiled_copy_gatetestscopy_r did not reach the tiled copy gate, same forcopy_lNot measured: the performance acceptance of
docs/adoption-protocol.md:55-60(no
fbench verify, nocandidate_compare.py, no target or guard cell of thisrevision). It was attempted on this host and could not start: the contract needs
CPU boost off and
/sys/devices/system/cpu/cpufreq/boostreads1, which is ahost-wide power-policy change on a machine that was running another workload at
the time (load average 34 -> 8,
tensor4all_treeat 87% CPU); the recorderscripts/record_run.py:10cannot importcampaign_toolkit, which is notinstalled here; and no
baseline-ref-*run at a comparable revision exists(newest are 2026-10-16, against a
325e482pin), so there is nothing for thecomparator to pair the candidate with. Correctness tests and the numbers above do
not stand in for that run, so the PR stays a draft.
Not done
The ragged-edge idea in the issue (keep the elementwise path when the tile loop
is ragged on both axes) is not implemented: the rule here is the byte count.
Two edges are therefore approximate: a ragged
513 x 513is above the thresholdand keeps the tiles although #290's table records +3.6% against them there over
its six cells, five of them slower, and
256 x 256is below it although the sametable records a 2.4% win. Raising the threshold to drop
513 x 513is not thefix and is not done: at 4210704 bytes it sits just above
512 x 512's 4194304,so any byte boundary above it puts
512 x 512below it too, and all twelve512 x 512cells win by 13.9% (#290). A monotone byte count therefore cannotmove that cell alone; that its cost comes from
n % 16is the issue's reading,not something this gate measured, which is why #323 stays open rather than being
closed by this PR.
review: gpt-6.1-sol reviewed the finished diff (verdict: no correctness defect;
its one IMPORTANT finding — that "other families untouched" overstates the blast
radius — is corrected above and in the ledger row; its two delete-list items are
applied:
tile_copy_paysis module-private and the unmeasured universalcache-miss sentence is gone).
review round 2: gpt-6.1-sol, CRITICAL 0 / NEEDS_CHANGES 3 / UNVERIFIED 3 —
applied: the QR/LQ routes are now in the caller list (verified:
copy_from_triangular_uppertransposes both views atmatmut.rs:1022-1025), the wide/tall above-threshold fixtures are added and theOrd::min(m, n)->nmutation was applied and does fail them, the ragged513and the256win are stated next to the constant, and #323 is no longer closed. Open: acceptance measurement (the draft note above) and the ledger's 96/128 loss sentence, which #290 calls below the harness's resolution.review round 3: gpt-6.1-sol, 1 IMPORTANT / 3 MINOR — applied: the reason the
threshold is not raised is corrected from "the working sets are 0.4% apart, so
the cause is
n % 16" to what the arithmetic actually forces (any byte boundaryabove
513^2also drops512^2, which wins), raggedness is labelled #323'sreading, the wrong geometric explanation of the rectangle is deleted, the ledger
row's
255..511range (which contained the256win) is listed as255, 257, 511with96/128marked unproven, and the dispatch assertion covers bothrectangles. Not applied: duplicating the reversed-row cases in the dispatch
assertion — the value test already runs every shape flipped, and the helper does
not read the flip.