Skip to content

fix(#0140): stamp app_version into the record contribute() returns - #260

Merged
nilsmechtel merged 3 commits into
mainfrom
fix/0140-contribute-app-version
Nov 6, 2026
Merged

nilsmechtel merged 3 commits into
mainfrom
fix/0140-contribute-app-version

Conversation

@nilsmechtel

Copy link
Copy Markdown
Collaborator

The dict contribute() returns is the only record a depositing site keeps, and it could not be dated. It carried schema_version but no app_version, so a client holding a saved response cannot tell a field that is null because this release could not compute it from one that is null because an older release did not have the field at all. That is precisely the ambiguity app_version was added to remove, and the one record it was missing from is the one that leaves the worker.

schema_version does not substitute for it, and pool.py says so on the line above the field: "0.27.6 moved every l2_from_base value without changing the record's shape." Two responses of identical shape can hold values computed by different releases.

The change

app_version is resolved once into a local in contribute() and used by both the stored metadata.json and the returned dict, so the two records cannot date the same contribution differently — by construction, not by assertion. Empty-is-unset is preserved (an exported-but-empty var would otherwise stamp "" and read as a known version).

Purely additive: a new key on a return dict, colliding with no existing name and duplicating no value. No stored record changes.

How it was found, and what in the report does NOT hold

Reported by the paper campaign auditing 92 saved deposit responses that came back in five shapes, with 4 nulls and 4 absent keys. They framed it as "the weights digest is written by at least three paths that disagree". Measured against origin/main (2708230), that framing does not hold and is not what this PR fixes:

  • An AST walk over every return <dict> in entry.py finds exactly one site carrying weights_sha256 (entry.py:1562). There are not three writers; the five shapes are five app versions of one writer. weights_bytes_on_server and the n_bytes/n_train_images shapes appear nowhere in the tree.
  • A null digest is unreachable on this path: runtime_cellpose.py:533 sets it from pool.sha256_file(ckpt), which returns h.hexdigest() or raises.
  • The digest is stored twice, independently of the return value — in metadata.json (where build_contribution_metadata takes weights_sha256: str, required) and in the index stub at entry.py:1540. So a real federation reading the pool finds it; the campaign's 8 gaps were gaps in their own saved responses.

What survives is the datability gap above, which is real and is what this changes. The reader-side guard (refusing merge members with an absent or null digest) is #139, on its own branch.

Verification

Tests drive the real contribute() coroutine with collaborators stubbed — the pattern test_merge_serialisation.py already uses — and compare the returned value against the metadata bytes actually PUT, not against a second read of the environment, which would pass even if the two writers diverged. No GPU, no torch, no live Hypha.

docker run --rm --shm-size 4g -v $PWD/apps/model-finetune:/src:ro \
  ghcr.io/aicell-lab/model-finetune:0.20.1-cellpose \
  bash -c 'cp -r /src /tmp/mf && cd /tmp/mf && pip install -q pytest pytest-asyncio && \
    python -m pytest -q -rs -o asyncio_mode=auto'
# 95 passed, 1 skipped   (the 1 skip is the pre-existing MODEL_FINETUNE_CORPUS_DIR fixture gate)

Mutation control: deleting the one added line from the return dict turns 4 of the 5 new tests red (4 failed, 1 passed). The survivor is the digest pin, which does not touch app_version — so the new tests discriminate rather than pass vacuously.

Why this is a DRAFT

No version edit yet, per the repo's bump-last rule. app-version-check.yml requires apps/model-finetune/manifest.yaml's version to already exist in the published artifact before this can go non-draft, so the bump + upload happen immediately before gh pr ready — with the floor re-measured against the live artifact at that moment, not against origin/main. Holding until the paper campaign's merge chain is quiet.

🤖 Generated with Claude Code

The dict `contribute` returns is the only record a depositing site keeps, and
it carried `schema_version` but no `app_version` — so a saved response could
not be dated. `schema_version` does not cover this: 0.27.6 moved every
`l2_from_base` value without changing the record's shape, so two responses of
one shape can hold values computed by different releases. The stored
metadata.json has carried `app_version` since 0.27.14 (#116); the copy that
leaves the worker did not.

Resolved once into a local and used by both writers, so the two records cannot
date the same contribution differently by construction rather than by
assertion.

Found by the paper campaign auditing 92 saved responses in five undatable
shapes. Their report framed it as three disagreeing write paths with nullable
digests; that part does not hold — an AST walk finds exactly one return site
carrying weights_sha256, and the digest comes from sha256_file, which returns
a hexdigest or raises. #139 covers the reader-side guard.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nilsmechtel and others added 2 commits November 6, 2026 21:02
Clearance-time bump for PR #260, measured at this moment rather than read off a
row: origin/main's apps/model-finetune/manifest.yaml is 0.30.0 and the published
bioimage-io/model-finetune artifact's newest committed version is also 0.30.0
(55 versions), so 0.30.1 is strictly greater than both baselines the gates
compare against.

Patch rather than minor because the change is additive to a returned record and
alters no existing behaviour: contribute() now carries app_version in the value
the client keeps, single-sourced with the copy already written to the stored
metadata (#140). A caller that ignored the field is unaffected.

Refs: #140
@nilsmechtel
nilsmechtel marked this pull request as ready for review November 6, 2026 20:04
@nilsmechtel
nilsmechtel merged commit 6aa588e into main Nov 6, 2026
6 checks passed
@nilsmechtel
nilsmechtel deleted the fix/0140-contribute-app-version branch November 6, 2026 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant