Skip to content

[agentic-token-optimizer] Optimize Daily Rendering Scripts Verifier: cut ~40 % of turns spent on environment probing #48677

Description

@github-actions

Target workflow

Daily Rendering Scripts Verifier (.github/workflows/daily-rendering-scripts-verifier.md) — selected as the highest-AIC workflow not optimized in the last 14 days and not self-targeting (excludes "Token"-named workflows). No source: provenance frontmatter, so it is safe to edit directly in this repo.

Analysis period + runs analyzed

Cost profile

Metric Value
Total AIC (8 snapshot days) 1,970.99
Runs counted in snapshots 5
Avg AIC/run ~394
Avg turns/run (3 sampled runs) 74 (63–85)
Avg cost/run (sampled) $4.48
Cache-read ratio ~97–98 % (healthy — cache is not the waste driver)
Recent reliability 3 of last 3 daily runs (2026-07-26 → 2026-07-28) failed outright before the agent could do any work

Ranked recommendations

1. Fix the blocking model-pricing failure (highest priority — currently 100% failure rate)

  • Estimated AIC savings: full run cost recovery (~$4–5/run currently wasted on 4 failed API retries with 0 tokens billed, but burns ~40s wall-clock and blocks the daily check entirely for 3 straight days)
  • Evidence: all 3 most recent scheduled runs (§30342547715, §30251308561, §30194656716) fail immediately with API Error: 400 Model "claude-opus-5" has no AI credits pricing and no default pricing is configured. This is an environment/model-pool config issue upstream of the prompt, but it means the workflow has produced zero useful audits since 2026-07-25. Flag to whoever owns apiProxy.defaultAiCreditsPricing / the claude model-pool alias so claude-opus-5 gets a fallback price or is excluded from the pool used by engine: claude.

2. Remove blind node-availability probing loop (~8–10 turns/run wasted)

  • Estimated AIC savings: ~10-12% per run
  • Concrete action: Every successful sampled run repeats the same discovery sequence — node -e "console.log(...)" (denied, "requires approval"), retried with dangerouslyDisableSandbox (denied again), then a full-path node invocation, then falling back to npm run test:js via vitest. Add a short note to Phase 4 stating plainly: "Bare node invocations are permission-gated in this sandbox and will always be denied — do not attempt them. Use npm run test:js -- --run <spec> (vitest) as the only working harness, per cached prior-run notes." This removes 3–4 failed-then-retried tool calls per run.
  • Evidence: run §30150974376 logs 10 separate node/path-probe commands, all failing until the agent falls back to npm run test:js; the same dead-end pattern recurs in the other 2 sampled runs (3–4 occurrences each).

3. Consolidate repeated git status --porcelain checks (3–4 duplicate calls/run)

  • Estimated AIC savings: ~5%
  • Concrete action: Phase 7 asks the agent to verify a clean working tree before finishing; in practice it calls git status --porcelain 3–4 times with slightly different flags (-C <path>, cd && git status, piped | head). Instruct: "Run git -C ${{ github.workspace }} status --porcelain exactly once after cleanup; do not repeat this check."
  • Evidence: §30150974376 has 4 near-identical git-status invocations; the other 2 sampled runs have 3 each.

4. Add explicit non-interactive polling guidance for the logs/audit MCP tools (5–6 sleep+tail calls/run)

  • Estimated AIC savings: ~8%
  • Concrete action: Phase 1's logs tool call is followed in every run by manual sleep N; tail ... polling loops against background task output files (the tool call itself appears to run async/backgrounded and 60s-time out per cache notes surfaced in the model's own summary). Add to Phase 1: "If the logs tool call appears to hang or time out, it is still completing the download server-side — wait once with a single sleep 60, then read the file_path result directly; do not poll repeatedly."
  • Evidence: 4–6 sleep; tail polling commands appear in every sampled run; one run's own final summary self-reports: "The agenticworkflows logs MCP tool 60s-times-out on every call, but still downloads artifacts server-side... readable afterward."

Optional structural optimization: shared setup note

Phases 1, 4, and 7 each restate ad hoc environment caveats (bash-cannot-access-/tmp, node-is-gated, MCP tool timeouts) inline and inconsistently. Recommend consolidating these into one ## Known Environment Constraints section placed right after ## Phase 0: Setup, listing:

  • bash cannot access /tmp/ — use Read/Write tools for /tmp/gh-aw/... paths
  • bare node execution is permission-gated (always denied) — use npm run test:js instead
  • the logs MCP tool may appear to hang/time out but completes server-side — wait once, don't poll repeatedly

This avoids restating these three caveats in three different phases and gives the agent one authoritative reference, estimated 5–8% additional turn reduction from reduced re-discovery.

Caveats

  • Sample size is 3 full run logs plus 8 snapshot-days; wider sampling could refine the exact turn-savings percentages.
  • Recommendation rejig docs #1 (model pricing) is an infra/config issue, not a prompt change — flagging here because it currently blocks all audits and dominates recent reliability data, but it may need to be actioned outside this workflow's .md file.
  • No inline sub-agent candidates identified: this workflow's phases are tightly sequential (each phase's output feeds the next — engine/output-file from Phase 2 is required by Phase 4 and 5), so it does not meet the independence bar for extraction.

References:

Generated by Agentic Workflow AIC Usage Optimizer · sonnet50 · 85 AIC · ⊞ 10K ·

  • expires on Aug 4, 2026, 7:53 AM UTC-08:00

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions