Repository navigation
SLCN trace-back gives a partition-dependent answer (1.6% at np=8 on a 128x128 level set) #682
Description
Symptom
The semi-Lagrangian advection (AdvDiffusionSLCN) gives a different answer at different rank counts. On the LeVeque swirling flow carrying a conservative level set (uw.systems.LevelSetSolver(..., advection="slcn"), StructuredQuadBox 128 by 128, P2, Courant 0.5, 20 steps, reinitialisation every fifth step, no mass correction), the enclosed volume after 20 steps is
| ranks | volume |
|---|---|
| 1 | 0.0706776688 |
| 2 | 0.0706771414 |
| 4 | 0.0706771438 |
| 8 | 0.0695739287 |
The same pipeline with the Eulerian transport (advection="supg") gives 0.0707207616 at every rank count, to ten digits, so the mesh, the level-set wrapper, the reinitialisation and the integral are partition-independent; the difference is in the trace-back. At 256 by 256 the shift at four and eight ranks is in the sixth digit (0.0706884035 / 0.0706891414 / 0.0706891222).
Reading
A 1e-5 change at two and four ranks is consistent with departure points near partition boundaries being evaluated on a slightly different path (ghost-cell interpolation against owner-cell interpolation); a 1.6% change at eight ranks on a mesh with about 2,000 cells per rank is not, and suggests departure points that leave the local partition being handled wrongly once the partitions are small relative to the step (0.5 cells per step here, so the feet should stay within the ghost layer). Related: #611 (empty-rank path) and the np=4 behaviour recorded around #596.
Reproduction
~/+Simulations/supg_vs_slcn_657/parallel/ls_parallel_timing.py on the feature/levelset-supg branch (#677):
mpirun -n 8 python ls_parallel_timing.py -uw_res 128 -uw_steps 20 -uw_advection slcn
prints the volume; compare with -n 1. The supg run is the control.
Underworld development team with AI support from Claude Code
Activity
Level set: mass correction automatic by transport, clip after every s…
The monotone clamp is applied where a point is evaluated; the Navier-…
lmoresi commentedon Sep 27, 2026
Cause found and fixed on PR #800 (28314aa). Two parts:
global_evaluateappliedmonotone="clamp"on the rank that asked for a departure point, bounding it by that rank's nearest nodes; a departure point evaluated on another rank was bounded by the wrong neighbourhood. The bound now runs on the rank that evaluates the point.Isolated with plain SLCN on the same swirl (128^2 quad, P2, 20 steps): clamp np 1 / 8 was 0.0707250985 / 0.0697297410, without the clamp 0.0707616737 / 0.0707616744. The level-set reproduction from this issue on #800 (LevelSetSolver copied in from #677):
SUPG control unchanged (0.0707207616 at every rank count). This issue closes when #800 reaches development.
Underworld development team with AI support from Claude Code