Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/workflows/auto-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,15 +59,20 @@ jobs:
env:
IDJSON: ${{ steps.identity.outputs.identity-json }}
WD: ${{ inputs.working-directory }}
SHA: ${{ github.sha }}
run: |
set -euo pipefail
prefix="$(printf '%s' "$IDJSON" | jq -r '.tag.prefix // "v"')"
vfile="$(printf '%s' "$IDJSON" | jq -r '.version_file // "version.txt"')"
version="$(tr -d ' \t\n\r' < "$WD/$vfile")"
# workflow_dispatch has no head_commit payload. Read the subject from
# the exact commit we will tag, for both push and manual runs.
subject="$(git log -1 --format=%s "$SHA" --)"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This repo states it is the automation source of truth for callers (README.md:3-6), so behavioral contracts of a reusable workflow need to live in versioned files. The diff makes workflow_dispatch-driven callers a supported path (subject now resolved from github.sha rather than the push-only payload), but the only place that contract is written down is the PR description: "Manual runs evaluate the selected commit, not all changes since the last release, and are not a force-release mechanism." The workflow's own header comment still reads "on push to main" (lines 3-5), and README.md says nothing about manual runs. A future agent or caller reading the file will conclude manual dispatch is unsupported, or will assume a manual run releases everything accumulated since the last tag — the path gate on a manual run only sees the single selected commit (github.event.before is empty, so gate.sh takes the diff-tree --root branch). Fix: extend the header comment to say the workflow serves both push and manual (workflow_dispatch in the caller) runs, and that a manual run evaluates only the selected commit against both gates and is not a force-release.

Reply inline to this comment.

{
echo "prefix=$prefix"
echo "version-file=$vfile"
echo "version=$version"
printf 'commit-subject=%s\n' "$subject"
} >> "$GITHUB_OUTPUT"

- id: gate
Expand All @@ -81,7 +86,7 @@ jobs:
run-number: ${{ github.run_number }}
before-sha: ${{ github.event.before }}
after-sha: ${{ github.sha }}
commit-message: ${{ github.event.head_commit.message }}
commit-message: ${{ steps.meta.outputs.commit-subject }}

- name: Tag
if: steps.gate.outputs.should-release == 'true'
Expand Down
7 changes: 7 additions & 0 deletions .github/workflows/test-actions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,13 @@ jobs:
- uses: actions/checkout@v4
- run: bash actions/auto-release/test_gate.sh
shell: bash
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install "PyYAML==6.0.2"
shell: bash
- run: python actions/auto-release/test_workflow.py
shell: bash

# darwin-gate portable halves (artifact sanity + the functional probe) via a
# stub binary. check-macho needs the macOS toolchain → exercised in a real #8
Expand Down
126 changes: 126 additions & 0 deletions actions/auto-release/test_workflow.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
"""Exercise the actual workflow shell blocks without minting release tags."""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sibling suite documents how to run itself (test_gate.sh:2: "run locally and in CI") and needs nothing but bash. This file adds the repo's first Python/third-party dependency, and the only record of that requirement is the pin inside .github/workflows/test-actions.yml (pip install "PyYAML==6.0.2"). Someone running python actions/auto-release/test_workflow.py locally, as the PR's own validation section instructs, gets a bare ModuleNotFoundError with no pointer to the fix. Fix: extend the module docstring to one more line, e.g. "Requires PyYAML (pip install PyYAML); run with python actions/auto-release/test_workflow.py."

Reply inline to this comment.


import json
import os
from pathlib import Path
import subprocess
import tempfile
import unittest

import yaml

ROOT = Path(__file__).resolve().parents[2]
WORKFLOW = yaml.safe_load((ROOT / ".github/workflows/auto-release.yml").read_text())
STEPS = WORKFLOW["jobs"]["auto-release"]["steps"]
META = next(step for step in STEPS if step.get("id") == "meta")
GATE = next(step for step in STEPS if step.get("id") == "gate")
TAG = next(step for step in STEPS if step.get("name") == "Tag")
ACTION = yaml.safe_load((ROOT / "actions/auto-release/action.yml").read_text())


class ManualReleaseTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.repo = Path(self.temp.name)
self.git("init", "-q")
self.git("config", "user.name", "Release Test")
self.git("config", "user.email", "release-test@example.com")
(self.repo / "version.txt").write_text("1.2\n")
self.git("add", "version.txt")

def git(self, *args):
return subprocess.check_output(
["git", *args], cwd=self.repo, text=True, stderr=subprocess.STDOUT
).strip()

def commit(self, message):
self.git("-c", "commit.gpgsign=false", "commit", "-qm", message)
return self.git("rev-parse", "HEAD")

def run_script(self, script, **env):
output = self.repo / "output"
output.write_text("")
result = subprocess.run(
["bash", "-e", "-o", "pipefail", "-c", script],
cwd=self.repo,
env={**os.environ, "GITHUB_OUTPUT": str(output), **env},
text=True,
capture_output=True,
)
values = dict(line.split("=", 1) for line in output.read_text().splitlines())

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dict(line.split("=", 1) for line in ...) raises ValueError: dictionary update sequence element #N has length 1 when a step writes a line without =. That is precisely the failure mode the body-leak guard in test_manual_release_and_dry_run is meant to catch ("Body must not enter the output file."), so the most important regression this file guards against surfaces as an opaque unittest error in a helper rather than a named assertion failure.

Fix: parse defensively and assert, e.g. split into pairs = [line.split("=", 1) for line in lines], self.assertTrue(all(len(p) == 2 for p in pairs), output.read_text()), then build the dict. Cheap, and it makes a leaked multi-line subject report itself directly.

Reply inline to this comment.

return result, values

def meta(self, sha):
return self.run_script(
META["run"], SHA=sha, WD=".",
IDJSON=json.dumps({"tag": {"prefix": "v"}, "version_file": "version.txt"}),
)

def test_workflow_wiring(self):
self.assertEqual(META["env"]["SHA"], "${{ github.sha }}")
self.assertEqual(GATE["with"]["after-sha"], "${{ github.sha }}")
self.assertEqual(
GATE["with"]["commit-message"], "${{ steps.meta.outputs.commit-subject }}"
)
self.assertEqual(TAG["if"], "steps.gate.outputs.should-release == 'true'")

def test_manual_release_and_dry_run(self):
sha = self.commit("feat: ship it\n\nBody must not enter the output file.")
result, meta = self.meta(sha)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(meta["commit-subject"], "feat: ship it")
result, gate = self.run_script(
ACTION["runs"]["steps"][0]["run"],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow steps are resolved by identity (next(step for step in STEPS if step.get("id") == "meta"), lines 15-17), but the composite action's gate step is resolved positionally as ACTION["runs"]["steps"][0]["run"] here. Inserting any step ahead of the gate in actions/auto-release/action.yml — a pre-flight check, for instance — would make this test execute the wrong shell block, and the failure would point at the test rather than at the edit. Fix: hoist a module-level GATE_STEP = next(s for s in ACTION["runs"]["steps"] if s.get("id") == "gate") and use GATE_STEP["run"], matching the lookup idiom already used for the workflow steps.

Reply inline to this comment.

ACTION_PATH=str(ROOT / "actions/auto-release"),
RELEASE_PATHS="**.go,go.mod,go.sum,version.txt", TOOL_PATHS="",
VERSION_FILE=meta["version-file"], TAG_PREFIX=meta["prefix"],
VERSION=meta["version"], RUN="9", BEFORE="", AFTER=sha,
COMMIT_MESSAGE=meta["commit-subject"],
)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertEqual(gate, {"tag": "v1.2.9", "should-release": "true"})
result, _ = self.run_script(
TAG["run"], TAG=gate["tag"], DRY_RUN="true", TAG_TOKEN="",
REPO="example/test", SHA=sha,
)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn(f"dry-run: would create tag v1.2.9 at {sha}", result.stdout)
self.assertEqual(self.git("tag", "--list"), "")

def test_exact_sha_not_latest_head(self):
sha = self.commit("fix(scope)!: ship the selected commit")
self.git("-c", "commit.gpgsign=false", "commit", "--allow-empty", "-qm", "docs: newer HEAD")
result, meta = self.meta(sha)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(meta["commit-subject"], "fix(scope)!: ship the selected commit")

def test_subjects_still_use_conventional_commit_gate(self):
for subject, expected in [("docs: update guide", 1), ("not conventional", 2)]:
with self.subTest(subject=subject):
self.git("add", "version.txt")
self.git("-c", "commit.gpgsign=false", "commit", "--allow-empty", "-qm", subject)
result, meta = self.meta(self.git("rev-parse", "HEAD"))
self.assertEqual(result.returncode, 0, result.stderr)
result = subprocess.run(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test calls actions/conventional-commit/check.sh directly instead of feeding the meta-produced subject through the composite gate script, so the non-release/invalid-subject cases never exercise the code path the PR is fixing.

The rc mapping of check.sh itself is already covered by actions/auto-release/test_gate.sh (docs skip -> 1, invalid -> 2), so the only new value here is that meta emits a subject that the gate consumes correctly. That translation step — commit-subject -> COMMIT_MESSAGE -> rc -> should-release — is exactly where the reported production failure occurred (empty COMMIT_MESSAGE produced conventional-commit check failed (rc=2) and a failed job). As written, a regression in the action's rc handling (e.g. treating rc=1 as a hard error, or rc=2 as a skip) would leave all six tests green.

Fix: run ACTION["runs"]["steps"][0]["run"] the way test_manual_release_and_dry_run does, passing COMMIT_MESSAGE=meta["commit-subject"], and assert the outcomes: for docs: update guide expect returncode 0 with should-release == "false", and for not conventional expect a nonzero returncode with no should-release key (or whatever the intended contract is). That keeps the assertion on the workflow's observable behavior rather than on a helper script already under test elsewhere.

Reply inline to this comment.

["bash", str(ROOT / "actions/conventional-commit/check.sh"),
"release-gate", meta["commit-subject"]], capture_output=True,
)
self.assertEqual(result.returncode, expected)

def test_shell_characters_remain_data(self):
subject = 'fix: preserve $(touch injected) `touch injected` "quotes"'
result, meta = self.meta(self.commit(subject))
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(meta["commit-subject"], subject)
self.assertFalse((self.repo / "injected").exists())

def test_missing_sha_fails_without_subject_output(self):
self.commit("feat: initial")
result, meta = self.meta("0" * 40)
self.assertNotEqual(result.returncode, 0)
self.assertNotIn("commit-subject", meta)


if __name__ == "__main__":
unittest.main()
Loading