Skip to content

refactor(ci): replace inline fixture-collector bash with a typed, tested script - #3

Merged
JPHutchins merged 4 commits into
mainfrom
refactor/collect-fixtures-script
Jun 3, 2026
Merged

refactor(ci): replace inline fixture-collector bash with a typed, tested script#3
JPHutchins merged 4 commits into
mainfrom
refactor/collect-fixtures-script

Conversation

@JPHutchins

Copy link
Copy Markdown
Collaborator

What

Replaces the ~90-line inline bash in the Collect fixtures step with a single-file, dependency-free Python script (.github/scripts/collect_fixtures.py) run by uv (PEP 723), plus a new check-script CI job that gates it on ruff, ruff format --check, mypy --strict, and pytest (36 tests). The build job now needs: check-script.

Why

The step was untyped, untested string-munging (find + sed + grep + jq). The rewrite is a pure functional core (parsing, transport/group classification, canonical naming, manifest shaping) with a thin IO shell (listing, copies, manifest write), and an exhaustively-matched tagged-union artifact model (Exe | MergedHex | Hex | Elf).

Behaviour

Manifest output is unchanged for all real Twister output — verified by an end-to-end uv run against a synthetic twister-out (the emitted qemu_cmd is byte-identical to the old jq form) and by the unit/integration tests. Latent bash bugs fixed in passing, none observable on real inputs:

  • merged.hex silently overriding a co-present .exe
  • run pointing at whichever path last set $artifact instead of the executable
  • the QEMU load flag chosen by sniffing the filename suffix rather than the image kind
  • a build-dir name without exactly one dot silently mis-sliced into config/target
  • silently taking the first of several merged_*.hex

The first two are mutually-exclusive in practice (native_sim vs sysbuild), so the change is behaviour-preserving; the rewrite just makes them structurally impossible.

Also — out-of-date action pins

  • astral-sh/setup-uv@v8 (new)
  • softprops/action-gh-release@v2@v3 (Node 20 → 24 runtime bump; safe on ubuntu-24.04)
  • checkout / setup-python / upload-artifact / download-artifact are already at their current majors — left as-is.

Test plan

  • ruff check and ruff format --check clean
  • mypy --strict clean (script + tests)
  • pytest 36/36 (incl. doctests + an end-to-end main())
  • uv run end-to-end manifest matches the old jq output
  • CI: check-script + both build legs (native_sim, arm) green on this PR

🤖 Generated with Claude Code

The "Collect fixtures" step was ~90 lines of inline bash (find + sed +
grep + jq) with no type checking or tests. Extract it to a single-file,
dependency-free Python script run by `uv` (PEP 723), gated by a new
`check-script` job: ruff, ruff format --check, mypy --strict, pytest.
The build job now `needs: check-script`.

The script keeps a pure functional core (parsing, transport/group
classification, canonical naming, manifest shaping) with a thin IO shell
(directory listing, copies, manifest write). Artifact kinds are a tagged
union (Exe | MergedHex | Hex | Elf) chosen by exhaustive match, so the
launch command, mcuboot flag, and qemu load flag all derive from one
selected image rather than from re-sniffing strings.

Manifest output is unchanged for all real Twister output (validated by an
end-to-end `uv run` plus 36 unit/integration tests). Latent bash bugs are
fixed in passing, none observable on real inputs:
- merged.hex no longer silently overrides a co-present .exe
- `run` derives from the Exe artifact, not from whichever branch set it
- qemu load flag is chosen by image kind, not by sniffing the filename
- a build-dir name without exactly one dot now errors, not mis-slices
- multiple merged_*.hex now errors instead of silently taking the first

Also bump out-of-date action pins: astral-sh/setup-uv@v8 (new), and
softprops/action-gh-release@v2 -> v3 (a Node 20 -> 24 runtime bump, safe
on ubuntu-24.04).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings June 3, 2026 00:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the CI “Collect fixtures” logic by replacing a large inline bash script in the build workflow with a typed, dependency-free Python script (PEP 723, run via uv), and adds a dedicated CI job to lint, type-check, and test that script before the build runs.

Changes:

  • Add .github/scripts/collect_fixtures.py to collect Twister outputs into canonically named fixtures and emit a sorted JSON manifest.
  • Add .github/scripts/test_collect_fixtures.py plus .github/scripts/pyproject.toml to enforce ruff, mypy --strict, and pytest coverage for the script.
  • Update .github/workflows/build.yaml to introduce a check-script gating job, run the collector via uv, and bump the release action pin.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
.github/workflows/build.yaml Adds check-script, gates build on it, switches fixture collection to uv run ...collect_fixtures.py, and updates the release action pin.
.github/scripts/collect_fixtures.py New typed fixture collector script implementing naming, artifact selection, manifest shaping, and IO.
.github/scripts/test_collect_fixtures.py New unit/integration tests defining the behavioral contract for the collector.
.github/scripts/pyproject.toml Tooling config for ruff/mypy/pytest for the script + tests.
.github/scripts/.gitignore Ignores local Python tooling caches for the scripts directory.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/build.yaml Outdated
Comment thread .github/scripts/collect_fixtures.py
JPHutchins and others added 2 commits June 2, 2026 17:10
astral-sh/setup-uv publishes moving major tags only up to v7; v8 ships
as v8.0.0 / v8.1.0 with no v8 alias, so `@v8` failed to resolve on the
runner. Pin the exact latest release.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses PR review feedback:
- check-script ran pytest on the runner's default interpreter (3.12 on
  ubuntu-24.04); pin UV_PYTHON=3.13 so the gate tests the same runtime
  the script targets (requires-python >=3.13) and the build leg uses.
- select_artifact now raises when a build dir carries both a native_sim
  .exe and a merged MCUboot image -- different target classes that never
  coexist -- instead of silently preferring merged. Legitimate sysbuild
  dirs (merged.hex + zephyr.hex + zephyr.elf together) are unaffected and
  still resolve via the merged > exe > hex > elf ladder.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@JPHutchins

Copy link
Copy Markdown
Collaborator Author

Thanks — addressed both in f7c75c7:

  1. Python pin in check-script — good catch. The gate ran uvx pytest on the runner's default interpreter (3.12 on ubuntu-24.04), not the 3.13 the script targets. Set UV_PYTHON: '3.13' on the job so ruff/mypy/pytest all run under the same runtime as the build leg.

  2. .exe vs merged.hex — fair, the wording overstated it. I narrowed the fix to the real contradiction: select_artifact now raises when a dir has both a native_sim .exe and a merged.hex (host vs flashed-MCU target classes that never coexist). I deliberately did not raise on "multiple image kinds" generally, because real sysbuild dirs legitimately carry merged.hex + zephyr.hex + zephyr.elf together — those still resolve via the merged > exe > hex > elf ladder. Added a regression test for each case.

Resolve the build.yaml collect-step conflict: keep the typed-script
invocation (uv run collect_fixtures.py) and take main's mps2/an385 matrix
addition. main's two new commits add a runnable mps2/an385 serial-recovery
fixture whose bootable image is MCUboot itself (mcuboot/zephyr/zephyr.elf),
with the app shipped as a .signed.bin.

Port that into the typed collector: select_artifact gains a lowest-priority
mcuboot/zephyr/zephyr.elf fallback (after zephyr.elf), matching main's merged
bash precedence -- without it the new fixture would be silently dropped from
the manifest. serial_recovery.nrf52840dk keeps MERGED_HEX_FILES, so it still
resolves to merged.hex (unchanged).

Verified: ruff, mypy --strict, pytest (40, +3 recovery-path tests), and a
uv-run smoke test reproducing main's manifest entry for the mps2 recovery
fixture (qemu_cmd byte-identical, mcuboot=false, serial_recovery=true).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@JPHutchins
JPHutchins merged commit f8a64ad into main Jun 3, 2026
4 checks passed
@JPHutchins
JPHutchins deleted the refactor/collect-fixtures-script branch June 3, 2026 04:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants