Skip to content

refactor(pruner): table-driven LRU engine with per-slot budget (13 stores) - #30

Closed
azlekov wants to merge 2 commits into
mainfrom
refactor/pruner-store-table
Closed

azlekov wants to merge 2 commits into
mainfrom
refactor/pruner-store-table

Conversation

@azlekov

@azlekov azlekov commented Oct 19, 2026

Copy link
Copy Markdown

Summary

Rewrites the cache-capping half of pruner.sh around one generic lru_entries engine driven by a STORE_TABLE. prefetch_* and docker prune behaviour are unchanged, apart from a tighter until= under disk pressure.

Before After
LRU functions 3 (lru_cap, lru_cap_bun, lru_cap_mise_installs) 1 (lru_entries) + per-store hooks
Stores bounded 5 (gradle, bun, mise-cache, hk, mise-data) 13 (+ npm, pnpm, cargo, rustup, konan, pub, toolcache; action archive cache via the same engine)
Budget 20 GiB per dir, per runner per slot: min(CACHE_BUDGET_BYTES=60 GiB, CACHE_BUDGET_PCT=40 % of fs) / slots, one global LRU across the slot's stores
Gradle eviction single files (metadata can point at deleted artifacts, #17.5) whole <g>/<m>/<v>, transforms, wrapper dists, jars dirs; metadata untouched
Job safety none (caches under running jobs) per-slot flock -n .prune.lock; slot skipped while busy < 6 h old, stale busy pruned
Schedule fixed 24 h sleep wakes every PRUNE_CHECK_SECONDS (3600); full pass when free < PRUNE_FREE_PCT (15) or PRUNE_INTERVAL_SECONDS elapsed; docker prune until=2h instead of 24h under pressure
Logging per dir per slot, with elapsed seconds

Protected: bun bin/ and @t@ piles, cargo bin/ and registry index, rustup default toolchain (from settings.toml, short names match <name>-<triple>). Toolcache <ver> is removed with its <arch>.complete markers (markers first). CACHE_CAP_BYTES stays as a deprecated alias of CACHE_BUDGET_BYTES (now a total over slots, logged).

Depends on W3

Needs the compose side from W3 (contract C1): RUNNER_DATA_DIR mounted at the same path into the pruner service and passed as env, with slots at $RUNNER_DATA_DIR/slots/<id>/{cache/<store>,busy,.prune.lock}. Without it the pruner finds no slots and only trims the action archive cache. Merge together with, or right after, W3's PR. docker-compose.yml, hooks, README and workflows are untouched here.

Tests

docker run --rm -v "$PWD:/w" -w /w ubuntu:24.04 bash tests/<t>.sh — all pass, plus shellcheck -x pruner.sh tests/*.sh clean:
pruner-bun (ported to the slot layout), -gradle, -cargo, -pnpm, -rustup, -toolcache, -busy (fresh/stale/locked), -budget (split over 2 slots); shared helpers in tests/lib.sh.

Not verified against a real runner volume.

🤖 Generated with Claude Code

azlekov and others added 2 commits October 19, 2026 20:04
…over 13 stores

Replace lru_cap / lru_cap_bun / lru_cap_mise_installs with a single lru_entries
engine driven by STORE_TABLE (bun npm pnpm gradle cargo rustup konan mise-data
mise-cache hk pub toolcache + the action archive cache). Whole directories are
evicted where partial entries are unusable (bun packages, gradle versions,
toolchains, tool versions), files elsewhere, with global LRU across a slot's
stores and a budget of min(CACHE_BUDGET_BYTES, CACHE_BUDGET_PCT% of the fs) per
slot. Slots under $RUNNER_DATA_DIR/slots are pruned under .prune.lock and
skipped while a fresh busy marker exists. The loop checks df every
PRUNE_CHECK_SECONDS and prunes fully when free space is low or the interval
elapsed, with a tighter docker prune under pressure.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
pnpm's store lives in cache/pnpm/store and its metadata cache in
cache/pnpm/cache (both symlink targets). Evict them together, emptying the
directory but keeping store/ and cache/ so the container symlinks stay valid.
Also test that control slots without a cache/ dir are skipped.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@azlekov

azlekov commented Oct 19, 2026

Copy link
Copy Markdown
Author

folded into the combined refactor PR

@azlekov azlekov closed this Oct 19, 2026
@azlekov
azlekov deleted the refactor/pruner-store-table branch October 19, 2026 21:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant