Fix DefaultCopilotVersion toolcache bypass caused by compat.json drift

[Copilot]

DefaultCopilotVersion (1.0.75) exceeded the compat.json max-agent (1.0.56), which caused install_copilot_cli.sh to skip compat resolution entirely when passed an explicit version—meaning the 1.0.56 runner toolcache entry was always rejected via exact-match, forcing a network download on every agentic job.

Root causes

• compat.json max-agent was stale relative to DefaultCopilotVersion
• The install script unconditionally skipped compat resolution when $VERSION was set, leaving COMPAT_MATCHED_MIN/MAX_AGENT empty
• find_cached_copilot_bin used exact-match-only when a version was explicit (continue past the range check), so a compatible toolcache entry was always rejected
• GH_AW_COMPILED_VERSION was never emitted in agent/detection job envs, so the install script had no compiled_version to resolve a compat window even if it wanted to

Changes

.github/aw/compat.json

• max-agent: 1.0.56 → 1.0.75

actions/setup/sh/install_copilot_cli.sh

• Accept MAJOR.MINOR.PATCH (with or without v) in GH_AW_COMPILED_VERSION validation — c.version in the compiler is "0.83.1" not "v0.83.1"
• Run compat resolution even when $VERSION is explicit, populating COMPAT_MATCHED_MIN/MAX_AGENT for range-based toolcache matching
• In find_cached_copilot_bin: when exact match fails but a compat range is available, fall through to the range check instead of skipping — lets cached 1.0.56 satisfy an explicit request for 1.0.75 when both are within the compat window

# Before: exact match only when VERSION is explicit
echo "  Skipping candidate (version mismatch: want ${requested_version_normalized}, got ${candidate_version_normalized})" >&2
continue   # range check below is dead code

# After: fall through to range check when compat window is available
if [ -z "$min_version" ] && [ -z "$max_version" ]; then
  echo "  Skipping candidate (version mismatch ...)" >&2
  continue
fi
echo "  No exact match; checking compat range ${min_version}..${max_version}" >&2
# fall through

pkg/workflow/compiler_main_job_helpers.go / threat_detection_job.go

• Emit GH_AW_COMPILED_VERSION: <version> in job-level env for released builds, enabling the install script to perform compat resolution and populate the range for toolcache matching

pkg/constants/version_constants_test.go

• TestDefaultCopilotVersionWithinCompatWindow: reads .github/aw/compat.json, finds the open compat row, and asserts min-agent ≤ DefaultCopilotVersion ≤ max-agent — CI gate to prevent this class of drift from recurring

• Fixes DefaultCopilotVersion drifts past compat.json max-agent, forcing a network install on every job and disabling the toolcache path #48358

---

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 12.3 AIC · ⌖ 5.94 AIC · ⊞ 7.2K · ◷
_{Comment /souschef to run again}

[Copilot]

Pull request overview

Fixes Copilot CLI toolcache bypasses caused by compatibility-window drift and missing compiler-version context.

Changes:

• Updates the supported Copilot CLI range.
• Enables compatibility-range matching for cached installations.
• Emits compiler versions and adds drift-prevention tests.

Show a summary per file

File	Description
.github/aw/compat.json	Extends the Copilot compatibility window.
actions/setup/sh/install_copilot_cli.sh	Adds explicit-version range fallback.
pkg/constants/version_constants_test.go	Guards against version drift.
pkg/workflow/compiler_main_job_helpers.go	Exposes compiler version to agent jobs.
pkg/workflow/threat_detection_job.go	Exposes compiler version to detection jobs.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 5/5 changed files
• Comments generated: 3
• Review effort level: Medium

[github-actions]

✅ Design Decision Gate 🏗️ completed the design decision gate check.

[github-actions]

✅ Test Quality Sentinel completed test quality analysis.

[github-actions]

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

[github-actions]

✅ PR Code Quality Reviewer completed the code quality review.

[github-actions]

Test Quality Sentinel Report - Score: 100/100 Excellent - 2 design tests, 0 violations. Both tests are behavioral_contract/high_value/design_test. Verdict: PASSED (0% implementation tests, threshold 30%)

🧪 Test quality analysis by Test Quality Sentinel · sonnet46 · 37 AIC · ⌖ 6.06 AIC · ⊞ 8.1K · ◷
_{Comment /review to run again}

[github-actions]

✅ Test Quality Sentinel: 100/100. 0% implementation tests (threshold: 30%). Both tests are strong behavioral contracts enforcing design invariants (compat.json window and Playwright cooldown). No violations.

[github-actions]

Review: Fix DefaultCopilotVersion toolcache bypass

The fix correctly addresses the root cause (stale compat.json max-agent + exact-match-only toolcache lookup). Overall approach is sound. Three pre-existing Copilot review comments cover the notable non-blocking issues:

1. Explicit version becomes a soft hint — any workflow pinning engine.version: X will now silently accept a compat-range match when GH_AW_COMPILED_VERSION is present.
2. TTL exemption still applies to explicit requests — a stale toolcache entry won't be expired when requested_version != latest.
3. open bool default in test — Go JSON unmarshaling maps an omitted open field to false; the test would miss any future open row relying on the schema default.

No blocking issues found. The CI gate test (TestDefaultCopilotVersionWithinCompatWindow) is a good addition to prevent drift from recurring.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · sonnet46 · 33.8 AIC · ⌖ 5.68 AIC · ⊞ 5K

[github-actions]

🏗️ Design Decision Gate — ADR Required

This PR makes significant changes to core business logic (125 new lines in pkg/ directories) but does not have a linked Architecture Decision Record (ADR).

📄 Draft ADR committed: docs/adr/48593-enable-compat-range-toolcache-for-explicit-copilot-version.md — review and complete it before merging.

🔒 This PR cannot merge until an ADR is linked in the PR body.

📋 What to do next

1. Review the draft ADR committed to your branch — it was generated from the PR diff
2. Complete the missing sections — add context the AI could not infer, refine the decision rationale, and list real alternatives you considered
3. Commit the finalized ADR to docs/adr/ on your branch
4. Reference the ADR in this PR body by adding a line such as:

   ADR: ADR-48593: Enable Compat-Range Toolcache Matching for Explicit Copilot CLI Version Requests

Once an ADR is linked in the PR body, this gate will re-run and verify the implementation matches the decision.

❓ Why ADRs Matter

"AI made me procrastinate on key design decisions. Because refactoring was cheap, I could always say 'I'll deal with this later.' Deferring decisions corroded my ability to think clearly."

ADRs create a searchable, permanent record of why the codebase looks the way it does. Future contributors (and your future self) will thank you.

📋 Michael Nygard ADR Format Reference

An ADR must contain these four sections to be considered complete:

• Context — What is the problem? What forces are at play?
• Decision — What did you decide? Why?
• Alternatives Considered — What else could have been done?
• Consequences — What are the trade-offs (positive and negative)?

All ADRs are stored in docs/adr/ as Markdown files numbered by PR number (e.g., 48593-enable-compat-range-toolcache-for-explicit-copilot-version.md for PR #48593).

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · sonnet46 · 59.2 AIC · ⌖ 13.1 AIC · ⊞ 8.5K · ◷
_{Comment /review to run again}

[github-actions]

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd. The fix correctly addresses all four root causes identified in the PR description and adds a solid CI gate. A few follow-up items worth addressing before merge.

📋 Key Themes & Highlights

Key Themes

• open bool decode (existing comment #3665567402): json.Unmarshal maps omitted open → false; the test skips rows where open == false, so a schema-valid row without an explicit open: true is silently ignored — the CI gate would pass even if the version drifted outside that row's range
• Soft-hint concern (existing comment #3665567324): the fallthrough from exact-match to range check means any workflow pinned to a specific explicit version can silently install a different (compatible) version; the behaviour change is intentional but deserves an integration test
• TTL exemption gap (existing comment #3665567363): the TTL bypass for explicit versions is not updated to account for the new range-match path
• Wildcard max-agent (new comment): semverCmp will panic-via-Fatal if max-agent is ever "*"; add a guard
• IsRelease() nil check (new comment): if env == nil in buildMainJobEnv is dead code; worth a comment to explain the dev/release asymmetry

Positive Highlights

• ✅ All four root causes are identified and addressed in a single coherent change
• ✅ TestDefaultCopilotVersionWithinCompatWindow is an excellent CI gate — exactly the right kind of regression prevention
• ✅ parseSemver / semverCmp are clean, self-contained, and easy to test in isolation
• ✅ Shell changes preserve backward-compatibility: when no compat window is available, exact-match still applies

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 · 44.9 AIC · ⌖ 4.93 AIC · ⊞ 6.7K
_{Comment /matt to run again}

[github-actions]

Verdict: REQUEST_CHANGES

The fix correctly addresses the reported toolcache bypass, but the new explicit-version compat-range fallthrough introduces a regression that silently disables cache freshness checks.

💡 Themes and highlights

Blocking:

• install_copilot_cli.sh: the fallthrough that lets an explicit version match a compat-range candidate bypasses the TTL-expiry check entirely, because that check is guarded by requested_version = "latest". This means an explicit-version install can now silently reuse an arbitrarily old cached binary within the compat window with no freshness check at all — the opposite of the intended fix's rigor.

Non-blocking / lower priority:

• version_constants_test.go: parseSemver rejects any prerelease-suffixed version (e.g. 1.0.75-beta.1) despite the compat.json schema explicitly permitting one, which would turn a schema-valid config into a confusing test failure rather than a real compat violation.
• A prior reviewer already flagged the Open bool JSON-decode default mismatch (schema says open defaults to true when omitted, but the Go struct defaults to false) — worth fixing alongside the above.
• The compat.json bump and GH_AW_COMPILED_VERSION env-injection changes themselves look correct and are consistent between compiler_main_job_helpers.go and threat_detection_job.go.

Recommend extending the TTL check to also apply on the explicit-version + range-match path before merging.

🔎 Code quality review by PR Code Quality Reviewer · sonnet50 · 73.8 AIC · ⌖ 4.4 AIC · ⊞ 7.4K
_{Comment /review to run again}

Comments that could not be inline-anchored

actions/setup/sh/install_copilot_cli.sh:431

This fallthrough lets an explicit-version request match a stale cached binary within the compat range, but the TTL-expiry check below only fires when requested_version = "latest" — so explicit-version installs now silently accept an expired cache entry with zero freshness check.

pkg/constants/version_constants_test.go:179

parseSemver hard-fails on any prerelease-suffixed version (e.g. 1.0.75-beta.1), even though compat.schema.json's semver definition explicitly permits a -prerelease suffix.

done

[pelikhan]

@copilot run pr-finisher skill

[pelikhan]

@copilot review changes against the spec.

[Copilot]

@copilot review changes against the spec.

Reviewed against the spec and fixed the remaining gaps in bb52032: only release-build compiler defaults opt into --compat-range; explicit pins (including the default’s numeric value) and development builds remain exact-match; default provenance now survives inline detection-job generation; TTL and production shell behavior have regression coverage. Final validation passed.

[pelikhan]

@copilot since it's open max-agent should be *