Repository navigation
refactor: pruner engine, deploy matrix tags and shared test harness - #33
Merged
Merged
Conversation
azlekov
added this pull request to stack #34
October 19, 2026 19:54
collect_units forked a find|awk pair plus stat subshells per evictable unit, tens of thousands of times on Gradle/bun caches. It now runs one find -printf per dir rule and one awk that sums size and newest file atime per unit (units sit at the glob's depth and find lists a directory before its contents), taking bun's package.json atime from the same stream. STORE_TABLE gets an opts column (shared, key=<file>) replacing the hard-coded actions exclusion and the bun unit_key override; the remaining per-store behaviour (rustup default protection, toolcache .complete markers, bun symlinks) sits in one store_hook dispatcher. Adds tests/bench/pruner-100k.sh, a timing of one pass over ~100k synthetic units. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… matrix jq
The matrix job extracts the Dockerfile versions once and exposes runner_tag as an output;
the build and merge jobs of deploy.yml and the build job of build.yml no longer repeat the
extraction. deploy.yml's matrix entries now carry their own tag list (merge) and the tag
baked into the image (build), replacing the per-line enable=${{ matrix.variant == ... }}
and the image-identity metadata step.
The variant/target/platform/cache-scope table moves to .github/jq/matrix.jq, shared by
build.yml and deploy.yml via jq -L; the tag rules are in .github/jq/tags.jq. Generated
tags and PR matrix are identical to the previous ones.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…tests tests/harness.sh holds the pure harness (scratch dir with exit trap, fail/gone/kept/finish, at_exit); tests/lib.sh keeps only the pruner fixtures on top of it, so tests/pruner-*.sh are unchanged. slots.sh and image-smoke.sh source the harness instead of keeping their own fail counters. image-smoke.sh runs its checks with docker exec in one idle container instead of a docker run per check. CI skips harness.sh and lib.sh in the test loop and shellchecks tests/bench. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
azlekov
force-pushed
the
refactor/pruner-engine
branch
from
October 19, 2026 19:55
2586dea to
7e0c21a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three cleanups deferred from #31 (/simplify review), one commit each, stacked on
feat/runner-slots(base this PR on it, or retarget tomainonce #31 merges).collect_unitsno longer forks per unit.find -printfper dir rule + one awk that aggregates by the glob's depth (a unit's size, newest file atime and bun'spackage.jsonatime are summed as the lines stream by;findlists a directory before its contents).STORE_TABLEgets anoptscolumn (shared,key=<file>) replacing the hard-codedactionsexclusion and bun'sunit_key; the rest of the per-store behaviour (rustup default protection, toolcache.completemarkers, bun symlinks) is onestore_hookdispatcher. The awk program is mawk-compatible (Ubuntu's default awk).%P\t%y\t%A@\t%T@\t%s\t%d:%T@is bun's old fallback key (unit mtime when there is no package.json),%dis the depth.matrixjob and exposed as outputs. deploy.yml's matrix jq emits each entry's own tag list (merge) and image tag (build), replacing the per-lineenable=${{ matrix.variant == ... }}and the identity metadata step. The variant/target/platform/cache-scope table is.github/jq/matrix.jq, shared by build.yml and deploy.yml viajq -L; the tag rules are.github/jq/tags.jq.tests/harness.sh(scratch dir + trap, fail/gone/kept/finish,at_exit) andtests/lib.sh(pruner fixtures on top, sotests/pruner-*.share unchanged).slots.shandimage-smoke.shsource the harness;image-smoke.shruns its checks withdocker execin one idle container instead of ~13docker runs.Benchmark: one pruner pass over a synthetic 100k-unit cache
tests/bench/pruner-100k.sh(50k bun packages, 30k Gradle module versions, 20k npm cache files; budget at 90%, so ~10% is evicted). ubuntu:24.04 container, Docker on Apple silicon (arm64):The remaining ~10 s is the evict loop (
rm -rf+ hooks per evicted unit) and the sort; not touched here.Verification
tests/pruner-*.shandtests/slots.shpass unchanged (ubuntu:24.04);tests/image-smoke.shpasses on local std/slim/ndk images and fails with exit 1 on a wrong JDK, leaving no container behind.collect_unitson a fixture with every store (hidden dirs, symlinks, names with spaces, rustup default, no-package.json bun unit): the same 34 records (key, size, store, stop, path).enable=conditions evaluated per entry): 20 build entries and 10 merge entries identical, including each image tag. build.yml's PR matrix is identical (key order aside).actionlintandshellcheck -xclean.🤖 Generated with Claude Code