Repository navigation
fix(#0141): publish the contribution weights digest on the pool wire state - #262
Draft
nilsmechtel wants to merge 2 commits into
Draft
nilsmechtel wants to merge 2 commits into
nilsmechtel wants to merge 2 commits into
Conversation
…tate `pool_wire_state` projected a 7-key contribution stub out of a 9-key stored one and dropped `weights_sha256` on the way out, so no client could read a digest the server had already computed, stored and was holding in the dict being iterated. `contribute()` returns the digest exactly once and nothing else exposes it. A deposit whose response is lost — measured at 4 of 92 in the round-2 soup campaign, on a 1,218,615,707-byte PUT — therefore left the client unable to tell a landed upload from one it had to retry, and a retry duplicates the member and silently weights that site 2:1 in the uniform mean. One key closes it. `.get()`, not a subscript, matching every sibling in this projection: this is the campaign page's view, where one absent field must not take the whole view down. The merge path subscripts instead, because there the harm is souping unchecked bytes (#139). Scope is the digest alone. Three other stored keys also never reach a client — `weights_path`, `metadata_path`, `architecture_signature` — and none of them answers a question a contributor cannot otherwise answer: the paths are server-side addressing, and the signature is implied by admission. (The reporter also listed `n_bytes` and `uploaded_at` as dropped; both reach the client renamed, as `bytes_out` and `received_at`. That name collision is #114, not an absence.) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…docstring The digest reached the wire in f9ba6b8, but only `pool_wire_state`'s docstring said so and that one ships nowhere. `get_pool_state` is the `@bioengine.method` a client inspects, and `bioengine/_app/decorators.py:129` publishes `f.__doc__` as the service schema's description — so a contributor holding a 504'd deposit, the reader this field exists for, had no way to learn it can be asked for. Docstring only, no behaviour change. Closes clause 3 of #141. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
get_pool_statehad every contribution's weights digest in the dict it wasiterating over and dropped it on the way out.
pool.pool_wire_statebuilt a 7-keycontribution stub from a 9-key stored one, so no client could read a digest the
server had already computed, stored and was holding.
Backlog #141. Reported by the round-2 model-soup campaign (
aged-mink-nvaylz),as the third and only surviving version of a report whose first two versions they
retracted themselves.
Why one missing key mattered
contribute()returns the digest exactly once, and nothing else on the appexposes it. So a deposit whose response is lost has no second way to ask.
Measured in the campaign: 4 of 92 deposits (4.3%) lost their
contribute()response to a 504 on a 1,218,615,707-byte PUT. In each case the pool gained the
member and the merge afterwards succeeded and reported nothing. The client was
left unable to answer "did my upload land, and is it the file I sent?" — and
retrying was not a safe fallback either, because a duplicate member silently
weights that site 2:1 in the uniform mean. With the digest readable, a lost
response costs a re-hash and a comparison.
What changed
One key on the projection, plus the docstring saying why it is published:
.get(), not a subscript, matching every sibling in this projection. This is thecampaign page's view, where one absent field must not take the whole view down.
The merge path subscripts instead, because there the harm is souping unchecked
bytes — that is #139 / PR #261, a different surface.
Scope: the digest alone
Three other stored keys also never reach a client —
weights_path,metadata_path,architecture_signature— and none answers a question acontributor cannot otherwise answer. The paths are server-side addressing a client
never uses; the signature is implied by admission, since a contribution in this
pool matched the pool's signature or
append_contributionwould have refused it.Widening a published surface further needs a reason, and there isn't one.
One correction to the report, measured here. It lists six dropped keys,
including
n_bytesanduploaded_at. Both of those do reach the client,renamed —
n_bytes→bytes_out,uploaded_at→received_at. The genuinelydropped set is four, not six. This matters because the report's rationale for
widening the fix ("a client that cannot see them cannot audit transport or
ordering") does not hold, and because
bytes_outas the wire name for acontribution's
n_bytesis #114 — a name colliding withmerge.transport.bytes_out— not an absence.Verification
Run in
ghcr.io/aicell-lab/bioengine-worker:0.16.58-dev1withpytestandpytest-asyncioinstalled (without the plugin, 6@pytest.mark.asynciocasesreport as failures unrun, in both arms).
origin/main+2 exactly, zero failures, identical skips. The branch also gives the five fixture
contributions distinct digests, and the 73 pre-existing tests all still pass, so
that fixture change broke nothing.
Two mutation controls, marker uniqueness asserted (count 1) before each edit:
.get()→ bare subscriptThe second is the one that earns its place: it shows the absent-digest test
discriminates the
.get()-vs-subscript choice specifically, rather than justre-testing presence. And because the fixture's five digests are all distinct,
the presence test cannot pass against a hardcoded value or one leaked from a
sibling row.
Not in this PR
No version bump yet.
apps/model-finetuneis gated byapp-version-check.yml,which requires the version to be already published before
gh pr ready, sobumping reserves a published version — and six
apps/model-finetunePRs arealready open (#237, #239, #255, #256, #260, #261) with backlog #127 paused on the
landing order. Nothing uploaded or deployed: a paper campaign is live on the
production instance.
This PR edits the same seven lines as #114. That issue is paused on a
published-schema choice (document / emit alongside / rename and move
SCHEMA_VERSION) forcontribution.bytes_out. This change needs no such choice —adding a key is purely additive. If #114's decision lands first, this one key can
ride along in the same commit; if this lands first, #114 rebases over one added
line. Either order works, but two concurrent branches on those lines will conflict.
🤖 Generated with Claude Code