[lmoresi]

The rest of the fault-split work, stacked on #889 (split-level fault multigrid); retarget to development when that merges. Together with #882 (the rebuild leak) this is everything UWTN 2026-018 and 2026-022 describe.

What it adds

• The add-on partitioned once (utilities/addon_partition.py, docs/developer/design/parallel-addon-distribution-2026-10.md): the finest mesh built on a static base by adapt, a cut, a placement or a split is partitioned for load once, with its add-on levels, and every transfer to it is partition-agnostic (recorded provenance, not location). A second partition keeps a split fault whole; faults are glued by name. Reviewed 2026-10-17 (record in docs/reviews/2026-10/).
• Custom multigrid on ranks with no coarse cells (bfae3e6 and its parent): the custom-P hierarchy and the cut survive a rank that holds none of a coarse level (the kaiju np 32/52 death).
• The rotated solve's Newton switch measured against the same reference as its convergence test (a warm start no longer stays in Picard to the end), and its record: the residual history and the line-search steps, read by the moving-fault drivers.
• A thin sheet's prescribed normal velocity (ThinSheet.normal_velocity): the breathing of a tidally flexed shell.
• The plex topology tables in Cython (_plex_topology.pyx).

Measured

UWTN 2026-022's fixture on this head: split 5 / welded 13 velocity iterations at k = 3, peak slip 0.1589860839 / 0.1589859512 (the note's figures); the note's "Using it" recipe runs as written. On the head before today's merges, the note's README numbers reproduced at np 1 (all of them) and np 4 (same hierarchies; iterations within one; slip within 2.4e-6).

Review

The parallel-addon lineage was reviewed on 2026-10-17. Today a pass on the whole of this PR's delta (the merged rotated solve, today's merges, the add-on and empty-rank code, the tests) found two must-fix items and several should-fix ones; the first comment lists them and what was done.

Suites (laptop, macOS, this head): tier A level 1 serial 1498 passed, 44 skipped, 0 failed; tests/parallel at np 2 180 passed, 15 skipped, 1 failed; at np 4 191 passed, 4 skipped, 1 failed; no hangs. The one failure is test_1063_constrained_freeslip_parallel[ti] (#826), as on development. Until #882 merges this PR's diff includes it; it drops out when #882 lands on development and #889 takes development in.

Issues

Left open by this PR, found in its review: #883 (the cold-start datum lift's convergence reference), #884 (the rotation-gauge projection measured from the second assembly), #885 (custom_mg.last_builder_stats is module-global; curved boundaries trigger the cross-partition rebuild), #886 (a rank-local exception decides build_transfers' retry), #887 (docs and messages behind the code), #888 (test reach: the cross-partition builder, an np 2 empty-rank case, test_1094's clamp).

Underworld development team with AI support from Claude Code

[lmoresi]

Adversarial review

The parallel-addon lineage had its own review on 2026-10-17 (docs/reviews/2026-10/). This pass (2026-10-29, read-only) covered the whole of this PR's delta: the merged rotated_bc.py (two lineages edited the same hunk of solve_rotated_freeslip), today's merges of #889 and #882 into this branch, the add-on and empty-rank code where the earlier review did not reach, and the tests. Every item below is fixed in this branch unless it names an issue.

Fixed

• guides/faults-in-convection.md (a capability guide, read by sessions as a skill) said solve(picard=N) raises on the rotated path and that the Newton switch reads the warm start's own residual, with newton_switch_rtol = 1.0 as the workaround. On this branch picard=N stages Picard then Newton (test_1018::test_rotated_freeslip_picard_under_newton_stages_picard_then_newton) and the switch reads the convergence reference (371d3bd). Corrected.
• Campaign drivers and probes (scripts/sessions/parallel_addon, tidal_shell, box_fault_cyclic, a fault_mg figure script) left the branch (Louis, 2026-10-15 and 2026-10-28); the design note says where the probes live.
• The rotation-gauge projection leaked one Vec per mode per solve, and a forcing with a net torque on free-slip walls (no solution) reported converged in silence; the modes are destroyed, and a dominant projected-out component warns, as the pressure gauge does.
• test_1093 duplicated development's test_1053 (same fixture and checks); removed.
• New tests started at tier A (test_1098, test_1100's table test, two new tests in test_1018 inheriting its module tier); tier B, and test_1018's tiers go on each test.
• FAULT_MULTIGRID_HIERARCHIES.md listed co-location as to come; it was done and retired.

Merge resolutions (c842c11)

• constitutive_models.py, ddt.py: development's _etd_maxwell_time / _update_exp_values (ETD Maxwell integrator: a viscosity that varies in space keeps its elastic response #851) over this lineage's 9e97670, which made the same repair (a viscosity varying in space keeps its elastic response); no caller of the old shapes remains.
• solvers.py: development's Ask an un-set-up field split for its blocks quietly (#827) #850 guard in _check_velocity_preconditioner, with this lineage's fault-contact marker kept.
• thin_sheet.py: the lineage's normal_velocity, then Split-level fault multigrid, the thin-sheet solver, and sheet extrusion #889's add_basal_drag(conds, ...).
• fault-split-multigrid.md: the lineage's text (co-location is gone from the code; nothing references colocate_split_level).

Left open (filed)

• Rotated solve: the cold-start datum lift sets the convergence reference to the lifted iterate's residual #883: in the hand-resolved hunk, the widened cold-start condition (a datum-only sheet) sets the convergence reference to the lifted iterate's residual. Measured on test_1092's breathing sphere: rnorm_history [0, 1.87e-8, 8.7e-15], reference 1.87e-8, two increments where one suffices; the answer is right. Under continuation the switch is evaluated at iteration 0 with reference 0 (not measured).
• Rotated solve: the rotation-gauge projection is measured from the second assembly on an uncached solve #884: on an uncached solve the rotation-gauge modes are collected after the first assembly, so the first residual includes what later ones exclude.
• custom_mg: last_builder_stats is module-global, and outside_hull counts nodes beyond a curved boundary's chords #885: custom_mg.last_builder_stats is module-global (an RBF build reads the last barycentric build's count), and outside_hull counts nodes beyond a curved boundary's chords, sending every annulus pair cross-partition in parallel.
• custom_mg.build_transfers: a rank-local exception decides a collective retry #886: build_transfers catches Exception per rank and retries or degrades on one rank alone (the mechanism of the kaiju np 32/52 death; bfae3e6 removed the trigger).
• Rotated solve and thin sheet: docs and messages behind the code #887: rotated-freeslip.md and thin-sheet.md do not describe the quotient-space residuals, the datum lift or normal_velocity; the GAMG fallback warning misleads on a manifold; solve_rotated_freeslip's Returns section is incomplete.
• Fault-split integration tests: cross-partition transfer and empty-rank cases not exercised in CI #888: test_1095 no longer reaches the cross-partition rebuild it was written for; test_1103 needs four ranks and CI runs two; test_1094's closed form is offset by its clamp; test_1092's bounds are wide; the fault-contact marker has no regression test.

Attacks that failed

• Co-location is gone: colocate_split_level, fault_star_owner and _fault_star_cells have no reference in src or tests.
• normal_velocity touches neither Split-level fault multigrid, the thin-sheet solver, and sheet extrusion #889's constitutive_model setter nor the pressure refresh; a datum forfeits the cross-solve cache, so a changed datum cannot ride a stale Q. On fault pairs the datum row matches the stored factor.
• Parallel uniformity of the rotated loop: every new branch reads collective norms or dots; _fitted_normal_traction reaches _collective_raise once on every rank, early return included.
• The info keys the note's recipe and the tests read (vel_its_last, pres_its_last, converged, velocity_pc) are present; rnorm_history and line_search_alpha have consistent lengths on all three increment branches.
• _plex_topology.pyx: the extern PetscInt is the header's type; empty strata reshape cleanly; the kd-tree index order matches the old closure slice.
• redistribute_by_assignment: all three callers take the new 4-tuple.
• CI reach: check_test_coverage.py reports every test file reachable; fault_fixtures.py is not collected.

Merge conditions

• Split-level fault multigrid, the thin-sheet solver, and sheet extrusion #889 merged; this PR retargeted to development.
• tier A level 1 serial and tests/parallel at np 2 and 4 as in the PR body; CI green.

[github-actions]

🔍 Architectural Review Checklist

Thank you for submitting an architectural review! Reviewers should validate:

Design & Architecture

• Design rationale is clear and well-justified
• Trade-offs are documented with alternatives considered
• System architecture is comprehensible
• Integration points are clearly identified

Implementation

• Implementation matches documented design
• Code quality meets project standards
• Breaking changes are identified and justified
• Backward compatibility is properly addressed

Testing & Validation

• Testing strategy is adequate for the changes
• Test coverage is sufficient
• Edge cases are properly covered
• Performance impact has been assessed

Documentation

• Known limitations are clearly documented
• Benefits are quantified with metrics
• User-facing changes are documented
• Migration guide provided (if needed)

---

Review Process: See CODE-REVIEW-PROCESS.md

Approval: This PR merge = Review formally approved

[github-actions]

✅ Test Suite: success

[github-actions]

✅ Test Suite: success