[jehrr]

Fixes from the 2026-10-14 audit of main @ 62a92f8.

Problems and resulting behaviour

Finding	Before	After
F1 writers truncate	write_json/csv/run_meta/last_attempt opened with "w"; a failed write left a 5-byte invalid out.json over a 703-byte good one	temp file in the target's dir, fsync, os.replace, unlink on error; new file 0o666 & ~umask, existing file keeps its mode
F2 CSV formulas	=HYPERLINK(...), -5, @SUM written raw	strings (after list joining) starting = + - @ TAB CR LF get ' in CSV only; numbers and JSON untouched; csv_cells_escaped in the sidecar, merged under caller extra
F3 solve budget	stubbed solver, 3-exit pool: 3 solves for one page	page_flow.SolveBudget (1/page) asked before every purchase, all 3 engines, both solver call sites each; not reset on rotation: 1 solve
F5 status dropped	Amazon's 404 page (measured: /dp/B0DEADDEAD, HTTP 404, 2.3 KB) carries the throttle page's "automated access" comment, so it was classified throttled → retried → exit 3 blocked	Playwright/pyppeteer pass goto() status; new terminal not_found state (404/410 or the page's own alt="Sorry! We couldn't find that page), not retried, not rotated, no solve, stop_reason: not_found, exit 5
F4 credential scan	clone under …/venv/r: 0 files scanned, nothing credential-shaped	exclusions on path.relative_to(REPO).parts; 0 files scanned is now a failure
F8 braced placeholders	ws://{login}-zone-… read as configured	any {...} reads as unset; warning names the variable only
CLI	negative --proxy-block-retries made the per-exit loop run 0 times; scraper_api_client accepted negative --retries/--retry-delay (negative delay → time.sleep traceback after a billed attempt)	argparse error naming the flag, exit 2
F7 flags	README: "asserts they take the same flags" (they don't)	README lists engine-only flags; suite checks them in both directions
F9/F11	references to documents outside the repo in code, .gitignore, CHANGELOG; donor repo named in captcha_solver.py; stale --mode variations comment; unused UNIQUE_BY_SKU_MODES	removed / restated; suite guard against private citations (tokens assembled from pieces)

Also removed an accidentally duplicated "CI" bullet in [Unreleased].

Evidence

• Offline suite green in a core-only venv and in a selenium and a pyppeteer venv (each pip check clean, that engine's group ran). Playwright venv (no browser download): green, including the new behavioural checks that drive _fetch_one_page with a faked page and a counting solver (1 solve over 3 exits; 404 → not_found, no solve, also with no pool).
• ci_checks.py --all green (35 files scanned).
• Planted-fault controls, each in a fresh copy with PYTHONDONTWRITEBYTECODE=1, each RED on the named check (18/18): writer opening its target directly; flat 0644; missing chmod; CSV escape removed; escape before list join; extra overwritten by writer stats; Playwright amazon solve unguarded (behavioural check bought >1); budget created per exit (static check; behavioural check crashes); Selenium reCAPTCHA unguarded; pyppeteer status dropped; not_found check removed; scan on absolute parts; braced placeholder accepted; negative block retries accepted; scraper_api negative retries accepted; private citation in README; section-sign reference in code; undocumented engine flag.
• One free live request pair to amazon.com (no proxy, no key) to measure the 404 page; a good /dp/ and a search page classify ok with the new marker.

Canary (owner request 2026-10-15)

Before: canary.yml already runs ungated, daily, from a bare GitHub runner with no secret and fails on exit 3/4/5/6, on zero rows and on parser-level floors (rows, price share, field coverage, pages, status). Last 10 scheduled runs green; the 2026-10-14 artifact holds 77 rows, 3/3 pages, complete. No repo secret except CLAUDE_CODE_OAUTH_TOKEN; AMAZON_PROXY is optional and not set.
After: only comments changed — two comments (canary.yml variation job, weekly.yml header) recommended a green ::notice:: skip if runners were ever refused; they now say to require AMAZON_PROXY and fail with an ::error:: naming it. Dispatched on this branch: https://github.com/2scraper/amazon-scraper/actions/runs/38694465697 — success, live-run 73 rows / 3 pages / complete, variation canary 1 row complete (csv_cells_escaped: 0 in both sidecars).

Not changed

• Release notes: v0.1.3 contained one private section reference; only those characters were removed with gh release edit, the rest is byte-identical. v0.1.0–v0.1.2 had none. Released CHANGELOG sections: only citation removals.
• Version / tag / release (pyproject says 0.2.0 vs tag v0.1.3: owner decision).
• Shared fetch loop / per-state policy as data (F6), schema/sinks: out of scope.
• pyppeteer/selenium missing --fingerprint/--locale: documented, not added (owner decision).
• Not-found on non-English marketplaces through Selenium (no status there; only the English page was measured).
• Canary policy unchanged.

🤖 Generated with Claude Code