Skip to content

Warn when Codex uses auto under AWF API proxy - #7818

Merged
lpcox merged 3 commits into
mainfrom
copilot/awf-fix-codex-auto-model-recognition
Aug 28, 2026
Merged

Warn when Codex uses auto under AWF API proxy#7818
lpcox merged 3 commits into
mainfrom
copilot/awf-fix-codex-auto-model-recognition

Conversation

Copilot AI commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Codex auto model resolution depends on ChatGPT-authenticated remote model/plugin metadata, but AWF routes Codex through API-key-based proxy auth. Even with chatgpt.com allowed, auto can fail with The requested model is not supported.

  • Agent startup warning

    • Detects codex + GH_AW_MODEL_AGENT_CODEX=auto when AWF API proxy routing is active.
    • Emits an actionable warning before the sandboxed command runs.
  • Docs

    • Updates the Codex API proxy example to use an explicit model.
    • Adds troubleshooting guidance for Unknown model auto is used / model_not_supported failures.
  • Coverage

    • Extends entrypoint shell tests to verify the warning only appears for Codex auto under API proxy routing.

Example:

npx @openai/codex --model gpt-5-codex -p "write a hello world function"

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix codex engine model recognition issue in AWF sandbox Warn when Codex uses auto under AWF API proxy Aug 28, 2026
Copilot AI requested a review from lpcox August 28, 2026 04:17
@lpcox
lpcox marked this pull request as ready for review August 28, 2026 04:25
Copilot AI balanced review requested due to automatic review settings August 28, 2026 04:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an actionable warning and guidance for Codex auto failures under AWF API-key routing.

Changes:

  • Detects Codex auto during agent startup.
  • Documents explicit-model configuration.
  • Adds shell-test coverage for warning behavior.
Show a summary per file
File Description
containers/agent/entrypoint.sh Adds the startup warning.
tests/entrypoint-phase-functions.test.sh Tests warning output.
docs/api-proxy-sidecar.md Updates the Codex example.
docs/troubleshooting.md Adds failure guidance.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Suppressed comments (2)

containers/agent/entrypoint.sh:428

  • The suggested model is rejected by this same API proxy: containers/api-proxy/guards/retired-model-guard.js:18-20 marks gpt-5-codex as retired and returns gpt-5.3-codex as its replacement. Following this warning would trade the auto failure for a guaranteed 400 response.
    echo "[entrypoint][WARN] Set an explicit Codex model in workflow frontmatter (for example, model: gpt-5-codex) before running under AWF."

docs/api-proxy-sidecar.md:104

  • The recommended explicit model is retired and blocked by AWF's API proxy (containers/api-proxy/guards/retired-model-guard.js:18-20), so this guidance leads users directly to another model error. Recommend the guard's supported replacement instead.
When Codex runs through the AWF API proxy, use an explicit model such as
`gpt-5-codex`. Codex's `auto` model alias depends on ChatGPT-authenticated
  • Files reviewed: 4/4 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread containers/agent/entrypoint.sh Outdated
Comment on lines +414 to +415
if [ -z "${AWF_API_PROXY_IP:-}" ]; then
return
Comment thread docs/api-proxy-sidecar.md Outdated
Comment thread docs/troubleshooting.md Outdated
@lpcox

lpcox commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

@copilot address review feedback

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot AI commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

@copilot address review feedback

Addressed in commit 493f66f: the warning now requires OPENAI_BASE_URL (with a negative fixture for sidecar-only configuration), and the docs use the supported gpt-5.3-codex model.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@github-actions

Copy link
Copy Markdown
Contributor

Documentation Preview

Documentation build failed for this PR. View logs.

Built from commit 7820f65

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Security Guard completed successfully!

PR #7818 security review complete: No security-weakening changes found. The new warn_codex_auto_model() function in containers/agent/entrypoint.sh adds informational diagnostics only — it reads environment variables, performs string matching, and logs warnings. No firewall rules, capabilities, ACLs, egress allowlists, or validation logic are weakened or expanded. The change is safe and passes security review.

Generated by Security Guard for #7818

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Docker Sbx has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Docker Sbx

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Contribution Check completed successfully!

PR review complete: the change includes code, docs, and a targeted shell test for the new Codex auto-model warning, and the PR description references the related issue. No contribution-guideline issues found that require comment.

Generated by Contribution Check for #7818

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Smoke Gemini reports failed. Facets need polishing...

💎 Faceted by Smoke Gemini

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Smoke Claude passed

Generated by Smoke Claude for #7818

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot Engine — PASS ✅

Overall: PASS

cc @lpcox

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct) Mode ✅ PASS

  • GitHub MCP: PR listing verified (2 recent merged PRs retrieved)
  • GitHub.com: HTTP 200 connectivity confirmed
  • File Write/Read: Smoke test file exists and readable
  • BYOK Inference: Running in direct BYOK mode via api-proxy → api.githubcopilot.com

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) with api-proxy sidecar handling real credentials.

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions github-actions Bot added smoke-copilot-byok smoke-copilot-network-isolation Copilot network-isolation egress smoke test labels Aug 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

EGRESS_RESULT allow=pass deny=pass

✅ Allowed domain (github.com) reachable — allowed=200
✅ Non-allowed domain (example.com) blocked — CONNECT tunnel 403

Overall: PASS

cc @lpcox

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke test: FAIL
Merged PR review: ❌ tool unavailable
Safe GitHub PR query: ❌ tool unavailable
GitHub page title check: ✅
Smoke file write/read: ✅
AWF build (npm ci && npm run build): ✅
Discussion lookup/comment: ❌ tool/write unavailable
Overall: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke test: GitHub Actions services connectivity

  • Redis PING: ❌ (DNS resolution failure for host.docker.internal)
  • Postgres pg_isready: ❌ (no response)
  • Postgres SELECT 1: ❌ (DNS resolution failure)

Overall: FAILhost.docker.internal could not be resolved from within the AWF sandbox.

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Docker Sbx Validation

  • ✅ GitHub MCP connectivity (verified via list_pull_requests)
  • ✅ GitHub.com connectivity (HTTP 200)
  • ✅ File write/read (smoke-test-docker-sbx-33188263505.txt)

Overall: PASS

cc @lpcox

📰 BREAKING: Report filed by Smoke Docker Sbx
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

OTel Tracing Smoke Test Results

  • Scenario 1 (Module Loading): otel.js loaded successfully, isEnabled: true, exports startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled, plus internal helpers.
  • Scenario 2 (Test Suite): 3 suites / 68 tests passed (otel.test.js, otel-fanout.test.js, otel-workload-identity.test.js).
  • Scenario 3 (Env Var Forwarding): env-passthrough.ts forwards GITHUB_AW_OTEL_TRACE_ID and GITHUB_AW_OTEL_PARENT_SPAN_ID; api-proxy-env-config.ts forwards OTEL vars to the api-proxy container.
  • Scenario 4 (Token Tracker Integration): onUsage callback present in token-tracker-http.js as the OTEL hook point.
  • Scenario 5 (OTel Diagnostics): otel.jsonl contains an exported span (gh-aw.agent.setup) with valid trace/span IDs and workflow attributes — export confirmed working during this run.

All 5 scenarios pass. No regressions detected in OTel tracing integration.

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine Validation

Check Status
API ✅ PASS
gh CLI ✅ PASS
File ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #7818 · claude · haiku45 · 32.7 AIC · ⊞ 4.5K ·
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Chroot Version Comparison Results

Runtime Host Version Chroot Version Match?
Python Python 3.12.14 Python 3.12.14 ✅ YES
Node.js v24.19.0 v2.97.0 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Result: Not all tests passed — Node.js version mismatch detected inside the chroot environment (host reports v24.19.0, chroot reports v2.97.0). smoke-chroot label not applied.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A (run succeeded) ✅ PASS
.NET json-parse N/A (run succeeded) ✅ PASS
Go color ok ✅ PASS
Go env ok ✅ PASS
Go uuid ok ✅ PASS
Java gson pass ✅ PASS
Java caffeine pass ✅ PASS
Node.js clsx All tests passed ✅ PASS
Node.js execa All tests passed ✅ PASS
Node.js p-limit All tests passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — PASS

Note: Java's default ~/.m2/repository was not writable (/home/runner/.m2 owned by root); builds succeeded after redirecting the local repo with -Dmaven.repo.local=/tmp/gh-aw/agent/m2repo (unrelated to firewall/proxy behavior — proxy settings from settings.xml were applied correctly).

Generated by Build Test Suite for #7818 · copilot · auto · 42.6 AIC · ⊞ 12K ·
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

@Copilot @lpcox

Smoke Test Results:

  • [Test Coverage] docker-manager.ts: ✅
  • Log the resolved filesystem.allowWrite boundary before the Cloud Hypervisor guest boots: ✅
  • GitHub.com connectivity: ✅
  • File write/read: ✅
  • BYOK inference: ✅

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)

Overall: PASS

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test Results:
• GitHub MCP: ❌
• GitHub.com: ✅
• File I/O: ✅
• BYOK inference: ✅

Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) via Entra
Overall: FAIL
cc: @lpcox

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)
Add label ready-for-aw to run again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[awf] agent: codex engine "auto" model not recognized when running inside AWF sandbox

3 participants