Skip to content

[agentic-token-optimizer] Daily XKCD Comic — Stop bash/curl fallback retries, downgrade to Haiku #354

Description

@github-actions

Target Workflow

Daily XKCD Comic (daily-xkcd.md) — selected as highest-AIC eligible workflow (all top-5 have been optimized within 14 days; XKCD was optimized 9 days ago and has the oldest optimization date among candidates with meaningful AIC).

Analysis Period

2026-07-24 → 2026-07-30 · 7 runs · All conclusions: success · Model: claude-sonnet-4.6

Spend Profile

Metric Value
Total AIC (7 runs) 279.5
Avg AIC / run 39.9
Min / Max AIC 24.4 / 61.0
Token-only AIC (avg) ~13 / run
Action minutes (avg) ~6 min / run
Avg turns 1–7 (high variance)
Cache efficiency Cross-run: 0% · Within-run: 86–98% (turns 2+)

Root Cause: bash/curl Retries Drive 2.5× AIC Variance

Observability flagged execution drift: runs vary from 0 to 7 turns. The agent-stdio logs reveal exactly why:

4 of 7 runs (57%) wasted 2–3 turns attempting bash curl or wget to fetch the XKCD JSON, which failed silently, before falling back to web_fetch or the pre-fetched file:

● Read latest comic info and day of year (shell)
✗ Fetch comic 705 (shell)           ← curl failed
✗ Fetch comic 705 via Python (shell) ← python fetch failed
✗ Try wget (shell)                   ← wget failed
● Check available xkcd files (shell) ← fallback
● create_discussion (MCP: safeoutputs)

3 of 7 runs (43%) used web_fetch directly and completed in 3 turns:

● Read latest comic and day of year (shell)
● create_discussion (MCP: safeoutputs)

Per-turn token usage confirms the cost difference:

Per-run breakdown
Run AIC Token AIC Turns Pattern
§30084387498 24.9 11.0 3 web_fetch direct
§30348954792 24.4 10.5 3 web_fetch direct
§30441984755 24.4 10.5 3 web_fetch direct
§30258070585 42.5 13.5 ~4 2× curl fail
§30197087279 51.0 14.3 ~4 2× curl fail
§30153237685 51.3 14.6 ~4 3× curl/wget fail
§30532789260 61.0 16.7 7 3× curl/wget fail

The bash tool is configured with curl, wget (via bash shell), and python3, all of which the agent sometimes reaches for instead of web_fetch. Because the xkcd.com endpoint appears to be accessible only through the web_fetch proxy (not direct curl from the runner), each attempt fails and burns a full turn.


Ranked Recommendations

1. Mandate web_fetch for all XKCD API calls — estimated ~15 AIC savings/run

Action: Replace the current Step 4 instruction:

4. Fetch the chosen comic: use `web_fetch` on `(xkcd.com/redacted)

with an explicit prohibition:

4. Fetch the chosen comic using **`web_fetch` only** — never `bash`, `curl`, `wget`, or Python — 
   on `(xkcd.com/redacted) and read the JSON fields: `num`, `title`, `img`, `alt`.

Also update Step 2 to use the pre-fetched file instead of fetching again:

2. Read `/tmp/gh-aw/agent/xkcd/latest.json` (already fetched by the pre-step). 
   Check if `title` or `alt` touches on code, math, ML, AI, or tech culture — if yes, use it directly.
   Do NOT re-fetch the latest comic via web_fetch or bash.

Evidence: 4/7 runs had curl/wget failures adding 2–4 extra turns, wasting 17–37 AIC each. Affected runs average 52 AIC vs 24 AIC for clean runs. Fixing this would align all runs with the low-AIC baseline. Savings = (0.57 × (52 − 24)) ≈ 16 AIC/run (conservative: 15).

References: §30532789260 · §30153237685 · §30197087279


2. Downgrade model from Sonnet to Haiku — estimated ~10 AIC savings/run

Action: Add model: claude-haiku-4.5 to daily-xkcd.md frontmatter.

The task is a textbook small-model fit:

  • Read a pre-fetched JSON file (trivial)
  • Fetch one JSON endpoint via web_fetch (trivial)
  • Classify title/alt text against ~8 topic keywords (simple classification)
  • Fill a 5-line markdown template (template formatting)

No cross-referencing of heterogeneous sources, no strategic synthesis. The "Why it resonates" sentence is punchy by design — a 1-sentence creative output well within Haiku's capability.

Haiku costs ~3× less per token than Sonnet. Current token-only AIC averages ~13/run. Haiku would reduce this to ~4.5/run. Faster inference also shortens run duration by an estimated 1–2 minutes, saving ~2–4 action-minutes AIC.

Estimated savings: 8.5 (tokens) + ~2.5 (action minutes) ≈ 10 AIC/run.


3. Reduce relevance retry limit from 3 to 2 — estimated ~3 AIC savings/run

Action: Change the instruction in Step 6:

6. If the chosen comic feels irrelevant, pick the next number from the list and try again (max 2 attempts).

The curated list is pre-vetted for developer/AI relevance. A 3rd retry is rarely necessary and adds one full web_fetch + reasoning turn. With Recommendation 1 in place, each retry turn costs ~1–2 AIC (Haiku) instead of ~3–4 AIC (Sonnet).

Evidence: Run 30532789260 performed 3 failed fetches before completing. Capping at 2 eliminates the worst-case third attempt.


Tool Usage Summary

Tool Configured Used in successful path Recommendation
web_fetch ✅ (efficient runs) Keep — required
bash (curl) ✗ Fails for xkcd.com Keep for pre-step; add prompt warning
bash (jq, date, echo) ✅ (date/day-of-year) Keep
bash (expr) Unused Consider removing
bash (wget) ✗ Fails for xkcd.com Consider removing

Caveats

  • Token/AIC breakdown is approximate; action-minutes costs are inferred from total vs token-only AIC deltas.
  • Model downgrade savings assume Haiku output quality is sufficient for the "Why it resonates" sentence. Recommend monitoring first 5 runs after change.
  • 7 runs is a small sample; the 3/7 vs 4/7 curl-fail split may shift.

Generated by Agentic Workflow AIC Usage Optimizer · 699.4 AIC · ⊞ 21.6K ·

  • expires on Aug 6, 2026, 3:19 PM UTC

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions