Skip to content

test(blas): port the OpenBLAS BLAS-level checks, first batch - #349

Open
shinaoka wants to merge 2 commits into
mainfrom
test/166-blas-inventory
Open

shinaoka wants to merge 2 commits into
mainfrom
test/166-blas-inventory

Conversation

@shinaoka

@shinaoka shinaoka commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Summary

Part of #166 (the follow-up to #31): the LAPACK validation covered the
decompositions, not the BLAS-level kernels. This PR adds the inventory of the
OpenBLAS test/ and ctest/ checks at the pinned revision, and a first batch of
bounded unit tests next to the owning modules.

Inventory: docs/blas-test-inventory.md. It maps
every check family in test/{s,d,c,z}blat{1,2,3}.f, ctest/c_* (CBLAS,
row-major, argument errors) and the OpenBLAS-only files to its faer counterpart,
marks each row ported / covered / no counterpart, and gives the reason. Test-only
change: no production path is touched, only #[cfg(test)] module wiring in
matmul/mod.rs and triangular_solve.rs.

Ported in this batch

Source check faer call under test Test
test/blat3.f DCHK3, DTRMM (and level-2 TRMV as the rectangular destination) matmul::triangular::matmul_with_conj matmul::blas::*::trmm_matches_an_independent_product, ...::trmm_reads_and_writes_nothing_outside_its_operands
test/blat3.f DCHK3, DTRSM; test/blat2.f DCHK3, DTRSV triangular_solve::solve_{,unit_}{lower,upper}_triangular_in_place_with_conj triangular_solve::blas::*::trsm_recovers_the_exact_solution_with_an_implicit_diagonal, ...::trsm_does_not_touch_memory_outside_its_operand
test/blat2.f DCHK4 DGER matmul::matmul with K = 1 (the private rank_update module), column-major and row-major destination matmul::blas::*::rank_one_update_matches_the_source_contract
test/blat2.f DCHK5 DSYR; test/blat3.f DCHK4 DSYRK matmul_with_conj with a triangular destination matmul::blas::*::symmetric_rank_k_update_writes_only_the_stored_triangle

The checked properties are the source's: an independently accumulated reference
product, the operand left unchanged, only the referenced triangle of the
destination written, and the parts a routine must not read (the unreferenced
triangle, an implicit unit diagonal, the leading-dimension padding) holding a
rogue value. The source's ROGUE = -1.0D10 becomes NaN (a masked read still
poisons the result), its random DBEG/DMAKE fixture and dimension sweep
become bounded exact-integer fixtures, and DMMCH's eps-ratio comparison
becomes exact comparison. DTRSM's alpha has no faer parameter, so the
fixture scales the right-hand side by alpha before the solve - the same
computation.

Inventory summary (the judgement, not every row)

  • Ported: TRMM, TRSM/TRSV, GER, SYR/SYRK (4 checks, 6 tests).
  • Already covered, no port: NRM2 (the norm_l2 tests already read a stride-2
    row view and the exponent range is norm_l2: strided fallback overflows or underflows for finite norms #36/faer-008) and ASUM (norm_l1).
    DOT has a counterpart (matmul::dot::inner_prod) but no unit test of its
    own, so it is a next-batch candidate, not covered.
  • No counterpart: banded and packed storage (GBMV/SBMV/SPMV/TBMV/TPMV/
    TBSV/TPSV/SPR/SPR2), symmetric and Hermitian structured multiply
    (SYMM/SYMV), SYR2K, the 3m variants, bfloat16/half comparisons, all of
    level-1 (AXPY/COPY/SCAL/ROT/SWAP/IAMAX), and XERBLA-style argument
    validation (faer panics on a bad shape instead of returning an info code).
  • Not ported in this batch: GEMM itself - matmul::tests::test_matmul covers
    the product but is #[ignore]d as too slow, so a bounded non-ignored GEMM case
    is the next candidate.

Provenance and licence

test/LICENSE and ctest/LICENSE in OpenBLAS are the netlib reference BLAS
test-program notice (credit requested). No Fortran is translated, so
faer/COPYING.BLAS.NETLIB carries that notice and the credit, NOTICE records
it and states why the crate's license expression is unchanged. Each adapted
file's header names the source file, the pinned revision, the routine, the URL
and the material changes; the oracle helpers are reimplemented in
faer/src/linalg/tests/blas_support.rs.

Verification

cargo test -p t4a-faer --lib blas - the package is t4a-faer (the lib is named
faer), so -p faer does not resolve. Full output, no filter:

running 24 tests
test linalg::matmul::blas::c32::rank_one_update_matches_the_source_contract ... ok
test linalg::matmul::blas::c32::trmm_reads_and_writes_nothing_outside_its_operands ... ok
test linalg::matmul::blas::c64::rank_one_update_matches_the_source_contract ... ok
test linalg::matmul::blas::c32::trmm_matches_an_independent_product ... ok
test linalg::matmul::blas::c64::trmm_matches_an_independent_product ... ok
test linalg::matmul::blas::c64::trmm_reads_and_writes_nothing_outside_its_operands ... ok
test linalg::matmul::blas::f32::rank_one_update_matches_the_source_contract ... ok
test linalg::matmul::blas::c64::symmetric_rank_k_update_writes_only_the_stored_triangle ... ok
test linalg::matmul::blas::c32::symmetric_rank_k_update_writes_only_the_stored_triangle ... ok
test linalg::matmul::blas::f32::symmetric_rank_k_update_writes_only_the_stored_triangle ... ok
test linalg::matmul::blas::f32::trmm_reads_and_writes_nothing_outside_its_operands ... ok
test linalg::matmul::blas::f64::rank_one_update_matches_the_source_contract ... ok
test linalg::matmul::blas::f32::trmm_matches_an_independent_product ... ok
test linalg::matmul::blas::f64::trmm_reads_and_writes_nothing_outside_its_operands ... ok
test linalg::triangular_solve::blas::c32::trsm_does_not_touch_memory_outside_its_operand ... ok
test linalg::matmul::blas::f64::trmm_matches_an_independent_product ... ok
test linalg::triangular_solve::blas::c64::trsm_does_not_touch_memory_outside_its_operand ... ok
test linalg::triangular_solve::blas::c32::trsm_recovers_the_exact_solution_with_an_implicit_diagonal ... ok
test linalg::triangular_solve::blas::f32::trsm_does_not_touch_memory_outside_its_operand ... ok
test linalg::triangular_solve::blas::c64::trsm_recovers_the_exact_solution_with_an_implicit_diagonal ... ok
test linalg::triangular_solve::blas::f64::trsm_does_not_touch_memory_outside_its_operand ... ok
test linalg::triangular_solve::blas::f32::trsm_recovers_the_exact_solution_with_an_implicit_diagonal ... ok
test linalg::triangular_solve::blas::f64::trsm_recovers_the_exact_solution_with_an_implicit_diagonal ... ok
test linalg::matmul::blas::f64::symmetric_rank_k_update_writes_only_the_stored_triangle ... ok

test result: ok. 24 passed; 0 failed; 0 ignored; 0 measured; 861 filtered out; finished in 0.58s

Regression on the two owning modules, nothing else changed:

$ cargo test -p t4a-faer --lib linalg::matmul
test result: ok. 34 passed; 0 failed; 1 ignored; 0 measured; 850 filtered out; finished in 4.09s

$ cargo test -p t4a-faer --lib linalg::triangular_solve
test result: ok. 34 passed; 0 failed; 8 ignored; 0 measured; 843 filtered out; finished in 1.13s

Red before green: one deliberate mutation per ported property

Each mutation was applied to the kernel, the matching test run, then the mutation
reverted and the test re-run. Only the f64 arm is shown; the other three types
fail and pass identically.

  1. TRMM: the kernel reads the stored diagonal of a unit-triangular operand
    (diag_kind(): Unit -> Generic). The unit diagonal holds NaN.
$ cargo test -p t4a-faer --lib linalg::matmul::blas::f64::trmm
test linalg::matmul::blas::f64::trmm_reads_and_writes_nothing_outside_its_operands ... ok
test linalg::matmul::blas::f64::trmm_matches_an_independent_product ... FAILED
trmm structure=UnitTriangularLower trans=0 m=1 n=1 left=true alpha=0.0
got=[
[NaN],
want=[
[0.0],
test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 883 filtered out
  1. SYRK: the destination structure is widened to Rectangular, so the
    referenced-half-only contract is broken and the other half is overwritten.
$ cargo test -p t4a-faer --lib linalg::matmul::blas::f64::symmetric_rank_k
rank-k structure=TriangularLower trans=0 n=2 k=3 beta=Replace alpha=0.0
got=[
[0.0, -0.0],
[-0.0, 0.0],
want=[
[0.0, NaN],
[-0.0, 0.0],
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 884 filtered out
  1. GER: the rank-1 update ignores beta (let beta = Accum::Replace; in
    rank_update_imp).
$ cargo test -p t4a-faer --lib linalg::matmul::blas::f64::rank_one
test linalg::matmul::blas::f64::rank_one_update_matches_the_source_contract ... FAILED
ger beta=Add alpha=0.0
got=[
[-0.0, 0.0, 0.0],
[-0.0, 0.0, 0.0],
want=[
[-1.0, 2.0, 0.0],
[1.0, -1.0, 2.0],
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 884 filtered out
  1. TRSM: the unit lower solve reads the stored diagonal (the n = 2 base case
    multiplies by tril[(i1, i1)] instead of 1). The unit diagonal holds NaN.
$ cargo test -p t4a-faer --lib linalg::triangular_solve::blas::f64::trsm
test linalg::triangular_solve::blas::f64::trsm_does_not_touch_memory_outside_its_operand ... ok
test linalg::triangular_solve::blas::f64::trsm_recovers_the_exact_solution_with_an_implicit_diagonal ... FAILED
trsm structure=UnitTriangularLower trans=0 left=true rows=2 cols=1
got=[
[-1.0],
[NaN],
want=[
[-1.0],
[1.0],
test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 883 filtered out

After reverting each mutation:

$ cargo test -p t4a-faer --lib linalg::matmul::blas::f64::trmm
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 883 filtered out

$ cargo test -p t4a-faer --lib linalg::matmul::blas::f64::symmetric_rank_k
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 884 filtered out

$ cargo test -p t4a-faer --lib linalg::matmul::blas::f64::rank_one
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 884 filtered out

$ cargo test -p t4a-faer --lib linalg::triangular_solve::blas::f64::trsm
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 883 filtered out

The two padding tests (trmm_reads_and_writes_nothing_outside_its_operands,
trsm_does_not_touch_memory_outside_its_operand) are the source's
leading-dimension check: they fail only when a kernel writes outside its output
region, which none of the four mutations does, and injecting that would need an
out-of-bounds write in the kernel. Their red state is therefore not demonstrated
here. git status --short after the last mutation run shows only the intended
files.

Formatting

cargo fmt --all -- --check is red on origin/main already with the pinned
1.96.0 toolchain (unrelated files: faer/examples/*, faer-ffi, operator/*,
...), because the tree's layout comes from a nightly formatter; the same is true
for cargo +nightly fmt --check. Neither reports a diff in this PR's files:

$ cargo fmt --all -- --check 2>&1 | grep -c 'blas_\|matmul/tests'
0
$ cargo +nightly fmt --check 2>&1 | grep -c 'blas_\|matmul/tests'
0
$ git diff --check
(no output, exit 0)

The new files were formatted with the repository's rustfmt.toml under
nightly 1.11.0 (imports granularity, comment width, style edition 2024).

Continuous integration

All checks pass. Two jobs needed one rerun each, both for reasons outside this
diff:

State

Pushed 3f45961f on test/166-blas-inventory (PR #349): the batch itself is
ee0ac939, plus one docs correction that names the source's CHECK
subroutines per row and stops claiming a unit test for matmul::dot::inner_prod
(it has none). Based on origin/main 4ccd69b3. origin/main has since moved to
828fd9e4 (#344, #342), which this branch does not conflict with.

Not verified

  • No GPU, wasm or non-x86 host: the new tests are host-independent sequential
    Par::Seq calls, but only x86_64 has been run here.
  • No hosted campaign or timing claim; this is a test-only change.
  • Banded/packed storage, SYMM/SYMV, SYR2K, the 3m variants and the
    #[ignore]d full GEMM sweep remain unported (see the inventory).

Inventory every check in OpenBLAS test/ and ctest/ at 791b02a against its
faer counterpart (docs/blas-test-inventory.md), and port the bounded ones:
TRMM through matmul::triangular, TRSM/TRSV through triangular_solve, the
rank-1 (GER) and symmetric rank-k (SYR/SYRK) updates through matmul with
K = 1 and a triangular destination.

Each test asserts an independently accumulated reference product and the
source's rogue-value properties: the unreferenced triangle, an implicit
unit diagonal and the leading-dimension padding must not be read, the
triangular operand must not change, and only the referenced triangle of the
destination is written. NaN replaces the source's ROGUE = -1.0D10 and exact
integers replace its random fixture, so the comparison is exact.

Test-only, four scalar types, 24 tests in the default run:
  cargo test -p t4a-faer --lib blas

Refs #166
`matmul::dot::inner_prod` has no unit test of its own; it is exercised only
through the decompositions that call it, so the reference BLAS dot check is a
candidate for the next batch rather than covered. Also name the source's
CHECK subroutines per row: DNRM2/DASUM/IDAMAX are CHECK1 (`incx` 1/2), while
DDOT and the vector updates are CHECK2/CHECK3 with `incx`/`incy` in
{1, 2, -2, -1}.

Refs #166
@shinaoka

shinaoka commented Nov 9, 2026

Copy link
Copy Markdown
Member Author

Note for review: after the first green run I pushed 3f45961f (docs only, docs/blas-test-inventory.md) to correct a wrong claim in the inventory — matmul::dot::inner_prod has no unit test of its own, so the reference BLAS DDOT check is a next-batch candidate rather than covered — and to name the source's CHECK0-CHECK3 subroutines per row (level-1 vector ops sweep incx/incy over {1, 2, -2, -1}; DNRM2/DASUM/IDAMAX only use 1/2). No code changed; cargo test -p t4a-faer --lib blas is 24 passed again.

The two CI failures seen earlier were unrelated flakes: faer-ffi/tests/fork.rs (from #344, same message as #348) passed on rerun, and the coverage job hit four QZ/LU SIGTERM timeouts under runner contention and then reported 918/918 passed.

@shinaoka

shinaoka commented Nov 9, 2026

Copy link
Copy Markdown
Member Author

GPT-6.1-sol adversarial review

CRITICAL: none
NEEDS_CHANGES: 2
UNVERIFIED: 2

Critical (must not merge)

none

Needs changes

  • The TRMM padding test never checks the computed destination — faer/src/linalg/matmul/tests/blas_trmm.rs:194–205: the only assertion compares padding around the read-only triangular input, while dst is discarded without checking its values. Reproduce by inserting dst.fill(nan()) immediately before the assertion: the test still passes for every scalar type despite an entirely invalid result. Consequently, a padding read that corrupts the result is invisible, and output-padding writes are not tested because the destination is not a guarded submatrix. Give dst guarded parent storage and assert both its numerical result and its surrounding padding.

  • The TRSM padding test guards the input rather than the in-place output — faer/src/linalg/tests/blas_trsm_cases.rs:168–202: parent contains the read-only system matrix, whereas the actual writable operand, b, has no guarded parent and its result is never inspected. Reproduce by inserting b.fill(nan()) before untouched_outside: the test still passes. This does not establish the leading-dimension output-write or padding-read properties claimed in docs/blas-test-inventory.md. Place b inside guarded storage, construct a known system, and check the solution and output guards.

Unverified claims

  • Inventory completeness and existing coverage — docs/blas-test-inventory.md:43–96: “every check” and the covered classifications require comparison with the pinned OpenBLAS directories and existing faer tests; the supplied diff does not contain those sources, so I cannot read them from this review material. Reproduce the audit by enumerating the pinned source routines and comparing each check, including complex-specific families, against inventory rows and cited existing tests; completeness is unverified.

  • Current-head CI and mutation results — PATCHSET.md:199: the recorded test counts and mutation outcomes, together with the PR body's “All checks pass,” are not independently established by the diff; I cannot read workflow runs or execution artifacts from the supplied material. Reproduce verification by matching successful check runs to head 3f45961fb8b323094cceededbcd8f208e68d5b80, inspecting enabled features and skipped tests, and rerunning the documented mutations; quoted logs alone do not establish current-head coverage.

Checked and fine

none

Not checked: executable tests, repository code outside the supplied diff, pinned upstream sources, or live CI; the padding-test failures above are established by assertion flow, not executed mutations.

Reviewed head: 3f45961fb8b323094cceededbcd8f208e68d5b80 at 2026-11-09T15:28:10Z. This review is valid only for that commit: if the head has moved, it is a review of a tree that is not being merged.
Automated (gpt_review.sh). This is an enumeration of counterexamples, not a verdict; the merge decision stays with a human.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant