[dongxuelian2]

Rechunking with a previous build can lose components or select one twice when multiple architectures or versions share a package name. Prior layer annotations persist names, while the object mapping uses unique component identifiers. This covers a reproducible cause of the assertions in #1885; the original downstream image builds have not been rerun.

Reuse the previous bins instead of abandoning the whole layout. Keep every current component of a same-name group in its prior bin when the annotations identify one bin, consuming repeated annotations only once. When a name spans prior bins, move only that name's components into the existing additions bin because the old annotations cannot recover the architecture/version split. Preserve unrelated package groups and the existing capacity fallback. No annotation format or dependencies change.

Regression tests assert the exact reused layout across eight duplicate-name scenarios, component identity/count/total size, and complete object movement with an empty remainder. Linux validation on the identical source blob:

• Original PR production function with the updated layout-preservation regression: expected failure.
• Focused chunking suite: 11 passed.
• make validate: passed, including formatting, feature/build checks, Clippy and documentation.
• Complete workspace: 495 passed, 0 failed, 4 pre-existing ignored tests/doctests, including 29 passing OSTree integration tests. The Ubuntu runner used explicit OSTree/skopeo runtime dependencies and a private mount namespace with Fedora-compatible Bash /bin/sh.
• Local formatting and diff checks: passed.

Validation run: https://github.com/dongxuelian2/bootc/actions/runs/39885851743

The new upstream CentOS 9 package job is currently red in the existing RPM reproducibility assertion: consecutive builds produce different bootc RPM hashes with identical SOURCE_DATE_EPOCH and version. RPM compilation completed; the later unit tests were skipped. The exact differing payload is unavailable. A targeted retry was denied with HTTP 403 (“Must have admin rights to Repository”), so maintainer action is needed for that check. Failing package job. This is separate from the successful isolated validation above.

Only components whose names span prior layers lose their prior placement. This can increase downloads for their affected layers, but unrelated prior bins remain reusable. Exact matching of those split components would require richer persistent identifiers.

Refs #1885.

Generated-by: AI

The code, regression tests and description were generated with AI assistance, validated automatically and independently reviewed by another AI agent. No separate human code review or manual downstream-image test is claimed. The contributor explicitly requested the required DCO sign-off and supplied their real name.

[TheMDev]

@dongxuelian2 Personally not a fan of this proposed fix. The way I read it is if the current components or prior manifest annotations contain duplicate names, rpm-ostree abandons the entire previous packing layout and computes a fresh layout which is no different than just running without the --previous-build flag

[dongxuelian2]

@TheMDev Agreed. I replaced the whole-layout fallback in 6dcf2c3 with targeted reuse:

• If a name occurs in one prior bin, all current components with that name stay in that bin, and repeated annotations consume them only once.
• If a name spans prior bins, only that name's components move to the existing additions bin. The name-only annotations cannot identify the old architecture/version split; unrelated groups retain their prior bins.

The regression now checks exact reused layouts for eight added/removed/retained and cross-bin scenarios, alongside component identity/count/size and full object-movement checks. With the old production function, the new layout-preservation test fails as expected. On the identical candidate source, all 11 chunking tests, make validate, and the full workspace pass (495 passed, 0 failed, 4 existing ignored tests/doctests, including 29 passing OSTree integration tests): Linux validation.

The original downstream image reproducer has not been rerun. The PR description now describes the targeted behavior and its remaining name-only metadata limitation. AI assistance and the requested DCO sign-off are retained.

CI update: the new upstream CentOS 9 package job built the RPMs but failed the existing check_package_reproducibility assertion in crates/xtask/src/buildsys.rs:59: two consecutive builds of the same bootc RPM yielded different SHA256 hashes despite identical SOURCE_DATE_EPOCH and package version. Compilation did not fail; the later unit-test step was skipped. The differing RPM payloads were not retained, so the exact source of the nondeterminism is not established. The isolated validation above remains successful. I attempted a targeted job retry, but GitHub returned HTTP 403, “Must have admin rights to Repository”; a maintainer must rerun or investigate that packaging check. The PR changes only chunking.rs and does not change the build-system assertion.

[TheMDev]

LGTM

[cgwalters]

Just a reminder that this code should be considered deprecated in favor of https://github.com/coreos/chunkah

[Copilot]

🔵 Needs a closer look

Required human review and maintainer resolution of the reported CentOS packaging-check failure remain outstanding.

No open issues 🎉

What changed in this PR

Fixes prior-build chunk packing when multiple components share a package name, addressing a reproducible cause of the assertions reported in #1885.

Changes:

• Keeps all same-name components in their unambiguous prior bin and consumes repeated annotations only once.
• Moves names spanning prior bins into the additions bin while preserving unrelated groups and the existing capacity fallback.
• Adds regression tests for eight packing layouts and complete object movement.

File	Description
crates/​ostree-ext/​src/​chunking.rs	Preserves duplicate-name components during prior-layout reuse and adds regression coverage.

_{Review effort: Balanced}

[cgwalters-bot]

Triage of cgwalters's comment: claimed; tracked in cgwalters-forge/tracker#895.

Generated-by: https://github.com/cgwalters/#llms

[TheMDev]

Just a reminder that this code should be considered deprecated in favor of https://github.com/coreos/chunkah

I'm aware however unfortunately chunkah still does not support chunking from a previous build coreos/chunkah#39

[cgwalters]

I only spent a few minutes looking at this, it looks plausible. But that said I think we're missing deeper integration tests here.

Some of this code is also a thing we could probably model more formally with Verus perhaps?

[cgwalters]

Sorry, I've now hopefully fully cut off this bot spam when I comment