Skip to content

FE-2021: Stop the Brunch provider admission tests failing on response timing - #10092

Merged
lunelson merged 1 commit into
mainfrom
claude/fe-2021-untimed-provider-admission-fixtures
Oct 19, 2026
Merged

lunelson merged 1 commit into
mainfrom
claude/fe-2021-untimed-provider-admission-fixtures

Conversation

@claude

@claude claude Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Requested by Lu Nelson · Slack thread

🌟 Purpose

Unit (@apps/brunch-agent) fails intermittently on main and on unrelated branches. Bilal Mahmoud reported it from this job, where "leaves unrelated provider use and its streaming behavior untouched" failed on a durationMs: 0 the other result lacked. "preserves admitted text, Unicode arguments, ids, usage and finish reason" fails the same way.

Both tests stream one fake response twice and compare the results. Since the upgrade to Pi 1.2, a stream adds durationMs to the final message only when the message's timestamp is no older than the stream. The fake response takes its timestamp when the test builds it, so each result gains or lacks the field depending on the millisecond its stream starts in. Lu Nelson worked this out in the thread.

This PR gives the shared response in both tests a preset durationMs: 0. A stream leaves a message that already has a duration alone, so both results carry the same value.

🔍 Changes

  • Looped 30 times locally on the base commit, the first test failed 22 times and the second 18 times. With this change both passed all 30 runs, and so did the whole file.
  • FE-1828: Add interview length controls to Brunch #10053 changes the first test's response differently, with timestamp: 0. Either fix works, but that branch will conflict with this one on that test.

@vercel

vercel Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hash Ready Ready Preview Oct 19, 2026 1:44pm UTC
petrinaut Ready Ready Preview Oct 19, 2026 1:44pm UTC
2 Skipped Deployments
Project Deployment Actions Updated
hashdotdesign-tokens Ignored Ignored Preview Oct 19, 2026 1:44pm UTC
petrinaut-docs Skipped Skipped Oct 19, 2026 1:44pm UTC

Request Review

@cursor

cursor Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

PR Summary

Low Risk
Test-only changes in provider-admission.test.ts; no production behavior modified.

Overview
Fixes flaky provider-admission unit tests that compare two streams of the same fake assistant message. After Pi 1.2, the stream layer can attach durationMs to the final result based on message timestamp vs stream start, so back-to-back collects of the same preset response could disagree on that field.

Both affected tests now build the shared response with an explicit durationMs: 0, so the stream leaves timing alone and the two results stay aligned. Comments document why the preset is there.

Reviewed by Cursor Bugbot for commit d9b4464. Bugbot is set up for automated code reviews on this repo. Configure here.

@lunelson
lunelson added this pull request to the merge queue Oct 19, 2026
Merged via the queue into main with commit 94eb1d6 Oct 19, 2026
69 checks passed
@lunelson
lunelson deleted the claude/fe-2021-untimed-provider-admission-fixtures branch October 19, 2026 14:32

This branch was successfully deployed

2 active and 1 inactive deployments
Preview – hash — d9b44646 Deployed Oct 19, 2026 by vercel[bot]
Preview – petrinaut — d9b44646 Deployed Oct 19, 2026 by vercel[bot]
Preview – petrinaut-docs — d9b44646 Deployed Oct 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants