Skip to content

test: fail on rejected Prismic requests and keep credentials out of traces - #293

Merged
angeloashmore merged 3 commits into
masterfrom
test/e2e-infra-fixes
Nov 9, 2026
Merged

angeloashmore merged 3 commits into
masterfrom
test/e2e-infra-fixes

Conversation

@lihbr

@lihbr lihbr commented Nov 9, 2026 •

Copy link
Copy Markdown
Member
  • res.ok is a method on Playwright's APIResponse, so assert(res.ok) never failed. Now res.ok().
  • Traces leaked the test account's credentials (from the sign-in request) into CI artifacts. Tests that log in are no longer traced. Rotate the password of angelo.ashmore+test@prismic.io, then update the E2E_PRISMIC_PASSWORD secret.

Also updates the toolbar test selector: the toolbar now loads shell.html instead of iframe.html.

Created with Claude Opus 5.5 in Claude Code.


Note

Medium Risk
Changes fix silent test passes on failed API calls and reduce credential leakage in CI, but alter e2e auth assertions and CI debugging artifacts.

Overview
E2E infrastructure now fails when Prismic API calls return non-OK responses by calling Playwright’s APIResponse.ok() (and status() where needed) instead of asserting the truthy ok property, which never triggered failures. Wroom login also asserts a prismic-auth session cookie after sign-in, and repository teardown tolerates a 404 on the retry delete path when the first attempt returns 500.

Credential and CI artifact exposure is reduced by dropping the global trace: on-first-retry setting from playwright.config.ts and removing the workflow step that uploaded playwright-report/ artifacts. Toolbar coverage is aligned with the current embed by targeting shell.html instead of iframe.html in the iframe locator.

Operators should rotate the E2E Prismic account password and refresh E2E_PRISMIC_PASSWORD if traces may have captured sign-in payloads.

Reviewed by Cursor Bugbot for commit 2bcbd37. Bugbot is set up for automated code reviews on this repo. Configure here.

lihbr and others added 2 commits November 9, 2026 12:44
…traces

Playwright's `APIResponse.ok` is a method, so `assert(res.ok)` never
failed. A rejected sign-in looped until the test timed out.

Traces record request bodies, so setup, teardown and the toolbar tests,
which log in to Prismic, are no longer traced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The toolbar script now loads `shell.html` instead of `iframe.html`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nobody uses the CI traces or the uploaded Playwright report. When CI
fails, we reproduce the failure locally.

Remove the global trace setting, the per-project and per-spec
`trace: "off"` opt-outs, and the step that uploads `playwright-report/`.
Without traces, the e2e account credentials cannot get into public CI
artifacts through a missed per-spec opt-out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L1dGD7TcFZH71ay5zFABEv

Copy link
Copy Markdown
Member

Let's just disable traces on CI altogether. We don't look at them, and if something fails, we can check it locally.


Generated by Claude Code

@angeloashmore
angeloashmore merged commit cd0e8b4 into master Nov 9, 2026
14 checks passed
@angeloashmore
angeloashmore deleted the test/e2e-infra-fixes branch November 9, 2026 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants