Skip to content

Refuse a re-declaration that would change what an expression container holds - #780

Open
lmoresi wants to merge 8 commits into
developmentfrom
bugfix/expression-redeclaration-explicit
Open

lmoresi wants to merge 8 commits into
developmentfrom
bugfix/expression-redeclaration-explicit

Conversation

@lmoresi

@lmoresi lmoresi commented Sep 22, 2026

Copy link
Copy Markdown
Member

Stacked on #775 — merge that first, then this retargets to development. Split out of #775 because it is a behaviour change users will notice, and it deserves its own review rather than riding along with a snapshot fix.

What was happening

An expression's identity is its name: uw.expression(r"\eta", ...) returns the existing container, which is what lets a formula written early keep seeing later edits to its contents. That much is deliberate.

But __init__ then ran on that returned object and overwrote its contents and its description from the arguments. So a line that reads like a declaration silently changed every formula already written against the name:

a = uw.expression(r'\eta', 1.0, 'first use')
b = uw.expression(r'\eta', 99.0, 'second, unrelated use')
a is b        # True
a.sym         # 99.0  — and the description is replaced too

The rule now

  • Same value → passes. Declaring the same thing twice changes nothing and no formula can tell. A factory that rebuilds an unmutated problem in one process is doing exactly that, and refusing it would be noise.
  • Different value → raises, naming the two things the caller might have meant (name.sym = value to change contents, uw.expression(name) to fetch). The existing contents are left untouched.
  • Anything that cannot be compared counts as different, so the loud path is the default — a container whose contents we cannot reason about is the last one to overwrite quietly.

Blast radius, measured before choosing

Surveyed 576 tests (bands 00–03) with the refusal instrumented to record rather than raise:

re-declarations total 15
with the same value 6
with a different value 9, across 5 names
in library code 0

All nine were test factories rebuilding the same named problem in one process, where the silent overwrite was load-bearing.

The most instructive is \eta_{w302}, whose container still held

2.0*k_n302 + 0.05*{T302}(N.x, N.y)**2

from the first build — referencing the first build's mesh variable. The overwrite is what rebound it to the second build's variable, and nothing said so. That is the hazard in one line: not a stale value, a stale reference, invisible.

Those three factories now declare-or-assign through a small local helper, which makes the rebinding the visible act it always was:

def _param(name, value, description):
    try:
        return uw.expression(name, value, description)
    except ValueError:
        existing = uw.expression(name)
        existing.sym = value
        return existing

Tests

tests/test_00*|01*|02*|03*py: 576 passed. Two contract tests added — a name fetched twice is one container captured once, and re-declaring with a value raises while leaving the contents untouched.

Still open, not addressed here

_expr_names is a class attribute, so it outlives the model: after reset_default_model() a fresh model re-uses containers from the previous one, carrying their values until something sets them. Separate decision — clear the registry on reset, or namespace it per model.

Underworld development team with AI support from Claude Code

An expression's identity is its NAME: uw.expression(r"\eta", ...) returns the
existing container so a formula written early keeps seeing later edits. But
__init__ then ran on that returned object and overwrote its contents and its
description from the arguments - so a line that reads like a declaration
silently changed every formula already written against the name.

Re-declaration now:

  * with the SAME value, passes. Declaring the same thing twice changes nothing
    and no formula can tell; a factory that rebuilds an unmutated problem in one
    process is doing exactly that, and refusing it would be noise.
  * with a DIFFERENT value, raises, naming the two things the caller might have
    meant: `name.sym = value` to change the contents, uw.expression(name) to
    fetch. The existing contents are left untouched.

Anything that cannot be compared counts as different, so the loud path is the
default: a container whose contents we cannot reason about is the last one to
overwrite quietly.

Blast radius, measured before choosing rather than after. Surveying 576 tests
(bands 00-03) recorded 15 re-declarations in total: 6 with the same value and 9
with a different one, across five names - and NO library code among them. All
nine were test factories that rebuild the same named problem in one process,
where the silent overwrite was load-bearing. The most instructive is
\eta_{w302}, whose container still held `2.0*k_n302 + 0.05*{T302}(N.x, N.y)**2`
from the FIRST build: the overwrite is what rebound it to the second build's
mesh variable, and nothing said so.

Those three factories now declare-or-assign through a small local helper, which
makes the rebinding the visible act it always was.

tests/test_00*|01*|02*|03*py: 576 passed.

Underworld development team with AI support from Claude Code
@lmoresi
lmoresi changed the base branch from bugfix/snapshot-captures-expressions to development September 23, 2026 13:33
…two tests

that specified the old behaviour

Two things this branch got wrong, both found by CI running the FULL suite.

First, the blast radius I reported was measured on bands 00-03 and was wrong.
CI showed 42 failures across 11 files, not 6, and library code DOES re-declare -
the derivative cache ({T}_{,1}(N.x, N.y), six hits) and a solver form
(\mathbf{f}_0(\mathbf{u}), ten). A partial band is not a blast radius.

Second, and the actual cause: the top collisions were generic names -
t_\textrm{now} twenty times, L fourteen, t and \eta eight each, x six. Those are
not one program re-declaring. They are UNRELATED TESTS in one pytest process
sharing a container, because UWexpression._expr_names is a CLASS attribute and
reset_default_model() only replaced _default_model and the strict-units flag. A
reset that leaves the containers standing is not a reset: the next model looking
up a familiar name inherits the previous one's contents, in a test session and
in an interactive one alike.

reset_default_model() now calls reset_expression_registry(), which clears both
the persistent and the ephemeral registries. Live objects are unaffected - a
solver or a formula holds the container itself, not its name. What resets is
LOOKUP, so the next uw.expression(r"\eta", ...) builds a fresh container instead
of finding the old one, which is what "a new model" should mean.
uw_object._obj_count is deliberately left monotonic: resetting it would hand out
instance numbers that live objects already hold.

That took the bands from 42 failures to 2, with zero re-declaration errors
remaining.

The last two were TestExpressionSilentUpdate, which asserted the OPPOSITE of
this branch - "Recreating an expression with the same name should silently
update it", called "natural Python behavior", with the loop pattern blessed
explicitly. So the behaviour removed here was specified and tested, and this is
the reversal of a decision rather than the fixing of an oversight. The class is
renamed and its tests inverted, with the docstring recording that the old
contract existed and why it changed; the loop case is kept, written as the
assignment it always was underneath.

tests/test_0565 14 passed; bands 05-08 1420 passed 2->0 failures; bands 00-03
576 passed.

Underworld development team with AI support from Claude Code
Copilot AI lite review requested due to automatic review settings September 23, 2026 17:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

lmoresi and others added 2 commits October 12, 2026 15:53
Merging development turned this branch's refusal red on
test_1056_navier_stokes_supg_api::test_tau_shapes_construct_and_step,
which builds two Navier-Stokes solvers back to back, one per tau shape.
The second construction raised:

    expression '\mathbf{f}_0\left( \mathbf{u} \right)' already exists
    and holds ...

The refusal is right and the defect is the name. F0 and F1 declared
'\mathbf{f}_0\left( \mathbf{u} \right)' and
'\mathbf{F}_1\left( \mathbf{u} \right)' with no instance tag, where
every peer in the same file tags: \rho_{tag}, \mathbf{a}_{tag},
\mathbf{u}^{n-1}_{tag}. A UWexpression's identity is its NAME, so one
container was shared by every solver of the class, and a second solver
rebound it to its own instance's symbols under any formula already
written against the first solver's residual. Before this branch that
happened silently; the refusal is what made it visible.

Both names now carry self.instance_number.

The sibling solver does not have this: advection_diffusion_eulerian
declares F0/F1 through the Template descriptor, whose __get__ updates
.sym IN PLACE on an existing container and so never re-declares. Only
this solver hand-rolls the declaration in a property, which is why only
it collided. Converting it to Template is the larger tidy-up and is left
alone here -- #800 rewrites much of this file.

test_1056 7 passed; with test_0610, test_0016 and test_0017, 24 passed.

Underworld development team with AI support from Claude Code

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E87Q7KrpapxeQiLD1RiNXv
@lmoresi

lmoresi commented Oct 12, 2026

Copy link
Copy Markdown
Member Author

The refusal found a real defect: two untagged solver template names

Merging development turned this branch red on one test:

tests/test_1056_navier_stokes_supg_api.py::test_tau_shapes_construct_and_step
ValueError: expression '\mathbf{f}_0\left( \mathbf{u} \right)' already exists and holds ...

That test builds two Navier-Stokes solvers back to back, one per τ shape.
The refusal is correct and the defect is the name.

F0 and F1 declared

r"\mathbf{f}_0\left( \mathbf{u} \right)"
r"\mathbf{F}_1\left( \mathbf{u} \right)"

with no instance tag, where every peer in the same file tags itself —
\rho_{tag}, \mathbf{a}_{tag}, \mathbf{u}^{n-1}_{tag}. An expression's
identity is its NAME, so those two were one container shared by every
Navier-Stokes solver in the process
. A second solver rebound them to its
own instance's symbols underneath any formula already written against the
first solver's residual. Before this branch that happened silently. The
refusal is what made it visible — which is the argument for the branch, not
against it.

Fixed in fa9da964: both names now carry self.instance_number.

Why only this solver

The sibling does not have the bug. advection_diffusion_eulerian declares
its F0/F1 through the Template descriptor, and Template.__get__
updates .sym in place on the existing container rather than declaring
again:

expr._sym = updated_value   # preserves object identity

So it never re-declares and never meets the refusal. Only
navier_stokes_eulerian hand-rolls the declaration in a plain property,
calling public_expression afresh on every access. Converting it to
Template is the right tidy-up and the real fix for the asymmetry, but it is
a larger change and #800 rewrites much of that file, so it is left for
afterwards rather than done here.

Tests

Rebuilt from clean after merging development:

  • test_1056_navier_stokes_supg_api — 7 passed (was 1 failed, 6 passed)
  • with test_0610, test_0016_solver_description_contract and
    test_0017_describe_and_render — 24 passed

Underworld development team with AI support from Claude Code

Brings the branch up to current development (100 commits), including the
test_1079 checkpoint-layout fix (#848).

Underworld development team with AI support from Claude Code
The re-declaration guard refuses a second construction whose contents
differ, which is right for the case it was written for: two unrelated
owners reaching the same name. But a solver's F0 and F1 are templates,
re-derived on every access from the parameters as they stand, so their
contents legitimately move as \Delta t and the Adams-Moulton coefficients
do. Construction was doing that work: __init__ writes _sym unconditionally,
so a second call silently updated the container in place, and these
properties have always relied on it.

template_expression() makes that explicit -- it writes through .sym, the
sanctioned path for changing what a container holds, and declares only
when the container is absent. Behaviour at these sites is what it was;
what changes is that the intent is now stated, and the guard stays strict
for every caller where a repeated name is an accident.

test_0024 built the same Poisson problem on two meshes under one set of
names, so g_{0024f} genuinely held different contents the second time.
It now frees the names between the two builds. Only lookup resets --
_expr_count keeps counting, so the shifted-counter premise the test turns
on is untouched, and the first problem's live objects are unaffected.

tests/test_0024_jit_graph_lowering.py and tests/test_1059_stress_transport.py:
44 passed, where the two files gave 3 failed / 26 passed before.

Leaves open: every F0/F1 name in solvers.py is untagged, and
\mathbf{F}_1\left( \mathbf{u} \right) is declared by four classes
(SNES_Tensor_Projection, SNES_AdvectionDiffusion, SNES_Diffusion,
SNES_NavierStokes), f_0 \left( \mathbf{u} \right) by three. That is one
container shared between solvers, which is the hazard this guard exists
to catch, and it wants its own branch and a probe.

Underworld development team with AI support from Claude Code
Our own adversarial pass on the previous commit, probed rather than read.

A refresh dropped the units the container was declared with. Writing
straight to .sym does not wrap, and only __init__ did: template_expression
turned 1.0 [metre] into a bare 2.0 with .units None, silently, since
nothing downstream asks whether a length is still a length. The wrapping
is now _wrap_with_units(), one definition called by both construction and
refresh -- a second copy is how the two drift apart again.

A refresh also discarded the description it was given, keeping the first
wording. describe() and the transcript print that string, so they would
have quoted a stale line.

template_expression cannot detect the case it is unsafe for: two owners
passing one untagged name silently rebind each other, which is the bug the
guard refuses. Stated as a precondition on the function, with the remedy
(tag the name with the owner).

tests/test_0027_expression_redeclaration_guard.py covers the guard's own
contract, which nothing did -- the refusal, the allowed identical
re-declaration, the in-place refresh, the units, the description, and what
a registry reset does and does not touch. level_1/tier_b, inside the
test_00[0-4]*py batch glob. 7 passed; the units case was shown failing on
the previous commit first.

Not fixed, recorded: template_expression consults only the persistent
registry, so a name held ephemerally would get a persistent twin.
Unreachable today -- templates are never ephemeral.

Adjudicated: #767 (the name-interning path overwrites the cached object's
value and description) is fixed by this branch, on its own reproducer --
both mechanisms refused, a.sym stays 0.01. Wants the fixed-in-PR label.
Still live and adjacent: #843, #771, #641, #765, #760.

Underworld development team with AI support from Claude Code
@lmoresi

lmoresi commented Oct 28, 2026

Copy link
Copy Markdown
Member Author

Adversarial review, and what the three failures actually were

Rebased onto development (now carrying #847, #848, #849, #677 and the five
merged on 2026-10-27) and pushed. The PR diff against development is 11 files.

The three CI failures were all this guard firing, not pre-existing breakage

An earlier note on this branch said they reproduce on development too. That
was wrong, and #848's run on development's content was a clean SUCCESS across
the suite, so test_0024 and test_1059 pass there. Both failures are the new
guard refusing a re-declaration, and they are two different problems:

  • test_0024 — the test and the guard genuinely disagree. Its _poisson
    factory builds the same problem on two meshes under one set of hardcoded
    names, and g_{0024f}'s contents name the mesh's own variable, so on mesh2
    they are different contents under the same name. Fixed by freeing the names
    (reset_expression_registry()) between the two problems, which says that
    outright. Only LOOKUP resets — _expr_count keeps counting, so the counter
    shift this test turns on is untouched.
  • test_1059 — the library re-declares. F0/F1 are @property and call
    public_expression on every access while the timestep and the Adams-Moulton
    coefficients move, so on development the second access was a silent update
    in place. That behaviour is load-bearing, and this PR converts it to an error.

The fix, and why it is narrow

template_expression(name, sym, ...) declares when the container does not
exist and otherwise writes through .sym — the sanctioned path for changing
what a container holds. Two call sites converted, both in
navier_stokes_eulerian.py. We counted the blast radius rather than inferring
it: 18 F0/F1-shaped property sites, 9 of the 16 non-NS ones carrying
time-dependent terms. We chose the narrow fix over tagging all of solvers.py
because the two changes fail in different ways and should not share a CI run.

Left open deliberately, recorded in 06c6db0a's message:
\mathbf{F}_1\left( \mathbf{u} \right) is declared untagged by four solver
classes (SNES_Tensor_Projection, SNES_AdvectionDiffusion, SNES_Diffusion,
SNES_NavierStokes) and f_0 \left( \mathbf{u} \right) by three. Every
solvers.py F0/F1 name is untagged. We will not claim that produces wrong
answers without a probe; it wants its own branch.

The review broke our own commit

Four defects in template_expression, in work already called done:

  1. units silently dropped on the refresh path — 1.0 [metre] became a bare
    2.0 and .units became None, because writing straight to .sym does not
    wrap. Fixed by factoring _wrap_with_units() into one definition shared by
    __init__ and the refresh, so construction and refresh cannot drift.
  2. description discarded — asked 'second', kept 'first'. Fixed.
  3. the ephemeral registry not consulted — recorded as unreachable today.
  4. the helper sanctioned the sharing the guard refuses — documented as an
    owner-unique-name precondition, since a name two owners both reach is
    silently rebound by it.

(1) and (2) are silent-data bugs that no amount of reading found; the probe did.

Evidence

tests/test_0027_expression_redeclaration_guard.py is new, because nothing
covered #780's guard, which is the PR's entire point. 7 tests, level_1 /
tier_b, inside the test_00[0-4]*py CI batch glob, with an autouse fixture
resetting the registry around each test since names are process-wide. One of
them — test_a_template_refresh_keeps_the_units_it_was_declared_with — is the
test that would have caught defect (1).

Local regression after the review fixes, since _wrap_with_units() now sits on
every construction path that passes units, not just the refresh:

#780 territory, 8 files   135 passed                          (16:54)
units suite, 12 files      96 passed, 2 skipped, 5 xfailed    (the xfails are
                                                               pre-existing
                                                               UnitAwareExpression
                                                               bugs, unrelated)
guard tests                 7 passed
----------------------------------------------------------------------
                          238 passed, 2 skipped, 5 xfailed, 0 failed

Issue adjudication in this territory

@lmoresi

lmoresi commented Oct 28, 2026

Copy link
Copy Markdown
Member Author

Correction: the solvers.py name sharing we left open is not a defect

Our review comment above said:

\mathbf{F}_1\left( \mathbf{u} \right) is declared untagged by four solver
classes [...] We will not claim that produces wrong answers without a probe;
it wants its own branch.

We have now run that probe, and it refutes the concern. No branch is needed and
nothing should be filed.

solvers.py does not call the interning constructor. It defines its own
factory at line 80:

def expression(*args, **kwargs):
    """UWexpression factory with per-instance unique symbol names. ..."""
    return public_expression(*args, _unique_name_generation=True, **kwargs)

so every F0/F1 body in that file goes through _unique_name_generation=True
and gets a distinct container despite the identical LaTeX name. This is
deliberate and documented in
docs/developer/design/SYMBOL_DISAMBIGUATION_2025-12.md.

Probe, on development's behaviour (no guard present):

solver path, same LaTeX name '\mathbf{F}_1\left( \mathbf{u} \right)':
  same object : False
  a holds     : flux_A      b holds: flux_B
  registry keys for that symbol: []

public path, same name:
  same object : True        c.sym now 2.0   (the silent overwrite #780 refuses)

And with two of the four classes actually instantiated —
SNES_Diffusion and SNES_AdvectionDiffusion on one mesh, one process:

A.F0 name = 'f_0 \left( \mathbf{u} \right)'          A.F1 is B.F1 : False
B.F0 name = 'f_0 \left( \mathbf{u} \right)'          A.F0 is B.F0 : False
A.F1 after touching B.F1 unchanged : True

The names are identical in the source and the containers are not shared, so
there is no rebinding to go wrong. That is also why this PR's guard never fires
on them, which we had not explained before.

Two notes on our own method here, since both were errors worth recording. Our
first attempt used uw.systems.Poisson, which is SNES_Poisson and not one of
the four classes, so it could not have shown the effect either way. And it
tested "did A pick up B's diffusivity" with '7' in str(expr) — a bare
substring that matches any 7 anywhere in the expression and would have read as
a positive on an unrelated coefficient. Neither finding would have been sound;
the result above comes from object identity and the registry contents instead.

@lmoresi

lmoresi commented Oct 28, 2026

Copy link
Copy Markdown
Member Author

CI is green: test = SUCCESS, 80 minutes, on run 40637277528. MERGEABLE, blocked only by the ruleset's approval requirement.

The three failures this branch carried are gone, and they were the three the
review above attributed to the guard rather than to pre-existing breakage —
test_0024 (the test and the guard genuinely disagreed, fixed by freeing the
names between two problems that reuse them) and test_1059 (the library
re-declared, because F0/F1 re-derive on every access while the
Adams-Moulton coefficients move, fixed by template_expression).

Nothing outstanding on our side. The local evidence stands as reported: 238
passed across #780's territory, the units suite and the new guard tests, with
the 5 xfails pre-existing UnitAwareExpression bugs.

One note for whenever this merges: it ran 80 minutes because the branch
predates #863, which shards the batches. A re-trigger after #863 lands would
put the same suite at roughly 29 minutes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants