Skip to content

Reclaim artifact storage after each prebuilt run - #10

Merged
danielhanchen merged 1 commit into
masterfrom
reclaim-artifact-storage
Aug 11, 2026
Merged

Reclaim artifact storage after each prebuilt run#10
danielhanchen merged 1 commit into
masterfrom
reclaim-artifact-storage

Conversation

@danielhanchen

Copy link
Copy Markdown
Member

Ports the reclaim job from unslothai/llama.cpp unsloth-prebuilt.yml, which this pipeline was copied from without it, and sets retention-days on the upload steps that had none.

Why

Nothing in this repo ever deleted a build artifact. Measured 2026-08-11: 168 live artifacts, 13.49 GiB across 27 runs, whose binaries were already published as release assets. llama.cpp has cleaned up after itself for a while; this repo inherited the pipeline before that job existed.

Two independent gaps caused it:

  1. No cleanup job. llama.cpp deletes a runs artifacts once they are provably on the release; this repo had zero artifact DELETE calls in any workflow.
  2. retention-days on 2 of 5 upload steps. The rest inherited the 90-day default, which is why artifacts from early July were still alive and set to expire in October.

What this adds

reclaim job (needs: [resolve, assemble], if: always()), ported with its gates intact:

  • Published runs delete only artifacts whose name matches a release asset. Build children upload sd-<tag>-bin-<label> and assemble publishes <name>.zip, so the match is exact and fixed-string. Anything unmatched is KEPT.
  • Only runs/$GITHUB_RUN_ID/artifacts is ever listed, so the job cannot reach a concurrent builds artifacts.
  • Runs that publish nothing delete their own bundles outright. always() rather than gating on published is deliberate: a publish:false dispatch and a cancelled run are exactly the cases that leaked in llama.cpp before.
  • continue-on-error: true throughout, so cleanup can never fail a published release.

assemble.outputs.published - set only after gh release edit --draft=false lands, so reclaim can tell "these are now release assets" from "nothing will ever read these".

retention-days: 7 added to the 3 upload steps that lacked it, matching the two that already set it.

Note on the full-set fallback artifact

unsloth-sd-prebuilt-<tag> (the "Upload full set (artifact fallback)" step) does not match any release asset name, so the published path keeps it - it was the single largest artifact at 1.38 GiB. That is llama.cpps conservative rule: never delete something that is not provably on the release. With retention-days: 7 it now ages out in a week instead of 90 days. If it is genuinely redundant once the release is up, deleting it explicitly would be a cheap follow-up, but that is a behaviour change I did not want to make silently.

Verification

  • unsloth-sd-prebuilt.yml parses as YAML; job graph is resolve, build-unix, build-linux-cuda, build-windows, assemble, reclaim.
  • reclaim has permissions: actions: write, contents: read and both steps.
  • All 5 upload steps now carry retention-days.
  • Asset naming confirmed against the live master-813-bfbef5b-uff6b225 release: every bundle is <artifact name>.zip.

Test plan

  • Dispatch with publish:false and confirm the unpublished path deletes the runs artifacts
  • Let a scheduled publishing run complete and confirm the 5 bundles are deleted and the manifest/fallback kept
  • Confirm the job summary table renders

@danielhanchen
danielhanchen merged commit 2d2351f into master Aug 11, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant