Skip to content

fix(tooling): key the rust build directory to the checkout - #914

Merged
cameri merged 3 commits into
rewrite/mainfrom
factory/906-rust-target
Oct 20, 2026
Merged

cameri merged 3 commits into
rewrite/mainfrom
factory/906-rust-target

Conversation

@chappie-daemon

@chappie-daemon chappie-daemon commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

scripts/rust.sh gave every checkout the same build directory. This keys it to the checkout instead. Refs #906, part of #871.

The failure

Every tree is mounted at /work/nostream, so cargo's fingerprints for a scratch copy and for a worktree are identical. One cache/target, mounted at /target, is therefore shared without any way to tell which sources produced an artifact.

Reproduced on a fresh worktree of rewrite/main (a3eee6a), with a scratch copy made by cp -a (mtimes preserved) and one source reverted in it — run_transaction back to &mut HashMap::new(), the #912 bug:

# scratch copy: build the reverted parity binary (old script, shared /target)
$ NOSTREAM_HOST_DIR=…/scratch-906 ./scripts/rust.sh cargo test -p nostream-storage --features postgres --test parity --no-run
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1.18s
  Executable tests/parity.rs (/target/debug/deps/parity-9a77595b26b2a3e0)

# worktree: correct sources, no touch, same old script
$ ./scripts/rust.sh env PARITY_ADMIN_URL=… cargo test -p nostream-storage --features postgres --test parity -- --ignored --test-threads=1
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.20s
     Running tests/parity.rs (/target/debug/deps/parity-9a77595b26b2a3e0)
test a_tls_connection_is_attempted_rather_than_refused_by_this_build ... ok
test the_typescript_script_replays_with_an_empty_diff ... FAILED
assertion `left == right` failed: op 117 (transaction) differs from the TypeScript capture
test result: FAILED. 1 passed; 1 failed

No Compiling line: the worktree ran the scratch copy's binary. touch crates/nostream-storage/tests/parity.rs then produced Compiling nostream-storage … ok. 2 passed — the workaround this PR removes, not a step to follow.

The fix

target_key=$(printf '%s' "$src_host" | cksum | cut -d' ' -f1)
target_dir="/target-$(basename "$src_host")-$target_key"

The container gets CARGO_TARGET_DIR=/target-<basename>-<checksum> and that path is bind-mounted from $RUST_CACHE_HOST/target-<basename>-<checksum>. Readable, and two trees that share a basename stay apart. CARGO_TARGET_DIR still overrides the derived name (absolute only; a relative value exits 64), and, when the cache root is visible to this shell, the script creates the directory first (the daemon would create a missing bind source as root, which uid 1000 then cannot write). A root that exists only on the daemon's side is left to the daemon, so a remote-Docker cache
keeps working: RUST_CACHE_VISIBLE names where this shell sees the cache (a remote daemon, or any
mount other than the /workspace ↔ /home/cameri/Workspace pair), and when no spelling is visible
the script prints the exact path the daemon host must provide, once, instead of refusing a cache it
cannot check.

For a remote Docker host the per-checkout directory has to exist before the first run, because the
daemon creates a missing bind source as root and uid 1000 then cannot write it:

ssh <docker-host> 'mkdir -p /srv/rust-cache/target-<checkout>-<checksum> && chown 1000:1000 $_'

Toolchain and registry under RUST_CACHE_HOST stay shared. The trade: one cold build per checkout, measured at 36 s for cargo test --workspace --all-features --no-run into an empty keyed directory on this machine, and 21 s for cargo check -p nostream-storage on a second path.

Same sequence with the fix

# scratch copy, reverted sources, fixed script
$ … ./scripts/rust.sh cargo test -p nostream-storage --features postgres --test parity --no-run
    Finished `test` profile [unoptimized + debuginfo] target(s) in 28.14s
  Executable tests/parity.rs (/target-scratch-906-562595450/debug/deps/parity-9a77595b26b2a3e0)

# worktree, no touch
$ ./scripts/rust.sh env PARITY_ADMIN_URL=… cargo test -p nostream-storage --features postgres --test parity -- --ignored --test-threads=1
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.30s
test a_tls_connection_is_attempted_rather_than_refused_by_this_build ... ok
test the_typescript_script_replays_with_an_empty_diff ... ok
test result: ok. 2 passed

The two runs wrote cache/target-scratch-906-562595450 and cache/target-906-rust-target-53259279; neither read the other.

What else referenced /target

  • scripts/rust.sh is the only in-repo reference; RUST_CACHE_VISIBLE is new. .gitignore has /target/, which is a local target directory, unrelated.
  • containers/rust-builder/rb-run:45,134,138 carries the same pattern (CACHE_TARGET=$STACK/cache/target, CARGO_TARGET_DIR=/target) and the same hazard for anyone building two trees through it. It is outside this repository, so this PR does not touch it — it needs its own change.
  • cache/target is left in place: it may still serve rb-run or a stale checkout. This script no longer uses it, and deleting it is safe whenever someone wants the disk back.
  • cache/target-r3sec already exists, which suggests someone had keyed a target directory by hand before this.

A changeset is included. CI's Rust Checks runs on this PR because scripts/rust.sh is in its path filter; the TypeScript jobs skip.

Every checkout is mounted at /work/nostream, so cargo could not distinguish one
tree's artifacts from another's in the shared cache/target: a scratch build left a
test binary compiled from reverted sources, and the next worktree run executed it
without recompiling. Derive the target directory from the checkout's basename and a
checksum of its full path; keep CARGO_TARGET_DIR as the override. Refs #906.
@greptile-apps

greptile-apps Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; the earlier remote-cache refusal is fixed.

Summary

scripts/rust.sh gives each checkout its own build directory while sharing the Rust toolchain and registry.

  • Rust builds now keep separate output folders for each checkout.

Reviews (3) · Last reviewed commit: "fix(tooling): name the client-visible ca..." · Reviewed by Greptile

Comment thread scripts/rust.sh Outdated
A cache root this shell cannot see is owned by the Docker host (a remote daemon,
or the sandbox seeing the same tree at /workspace). Refusing that case would break
a cache that works. Create the keyed directory only when one of the two spellings
exists, and keep the loud refusal for a visible root that is not writable.
@changeset-bot

changeset-bot Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ab3ab63

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

A remote daemon keeps the cache on its own host, where this shell cannot create the
per-checkout directory and the daemon would create it as root. RUST_CACHE_VISIBLE names
the spelling this shell sees, so the directory is still created with the right owner;
when no spelling is visible the script says which path must exist on the daemon and
carries on instead of refusing.
@chappie-daemon

Copy link
Copy Markdown
Collaborator Author

@greptile-apps review — the P1 you raised was fixed in 12077ff and the check still carries the pre-fix verdict.

@chappie-daemon

Copy link
Copy Markdown
Collaborator Author

The remote-cache gap is closed in ab3ab63, and the body now carries the one-time host step.

A cache root the client cannot see can no longer fail the run. RUST_CACHE_VISIBLE names where this shell sees the cache — a remote daemon, or any mount other than the built-in /workspace ↔ /home/cameri/Workspace pair — so the per-checkout directory is still created with uid 1000 ownership. When no spelling is visible, the script prints the exact path the daemon host must provide and proceeds; the loud exit 66 now covers only a visible root that uid 1000 cannot write.

Measured on this head:

case result
default root no warning; cargo fmt --all --check clean
RUST_CACHE_HOST=/srv/rust-cache RUST_CACHE_VISIBLE=<visible root> directory created under the visible root, then docker create
RUST_CACHE_HOST=/srv/rust-cache, nothing visible scripts/rust.sh: /srv/rust-cache/target-906-rust-target-53259279 is not visible here; it must exist on the Docker host and be writable by uid 1000 (set RUST_CACHE_VISIBLE …), then docker create
RUST_CACHE_VISIBLE=/nope RUST_CACHE_VISIBLE=/nope is not a directory, exit 66

The remote path cannot be exercised end to end from here — the sandbox exposes no second daemon — so the first of those warning cases is the one that needs a real remote host's confirmation.

@cameri
cameri merged commit 6ef609a into rewrite/main Oct 20, 2026
15 checks passed
@cameri
cameri deleted the factory/906-rust-target branch October 20, 2026 02:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants