You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
FE-2021: Stop the Brunch provider admission tests failing on response timing - #10092
Unit (@apps/brunch-agent) fails intermittently on main and on unrelated branches. Bilal Mahmoud reported it from this job, where "leaves unrelated provider use and its streaming behavior untouched" failed on a durationMs: 0 the other result lacked. "preserves admitted text, Unicode arguments, ids, usage and finish reason" fails the same way.
Both tests stream one fake response twice and compare the results. Since the upgrade to Pi 1.2, a stream adds durationMs to the final message only when the message's timestamp is no older than the stream. The fake response takes its timestamp when the test builds it, so each result gains or lacks the field depending on the millisecond its stream starts in. Lu Nelson worked this out in the thread.
This PR gives the shared response in both tests a preset durationMs: 0. A stream leaves a message that already has a duration alone, so both results carry the same value.
🔍 Changes
Looped 30 times locally on the base commit, the first test failed 22 times and the second 18 times. With this change both passed all 30 runs, and so did the whole file.
Low Risk
Test-only changes in provider-admission.test.ts; no production behavior modified.
Overview
Fixes flaky provider-admission unit tests that compare two streams of the same fake assistant message. After Pi 1.2, the stream layer can attach durationMs to the final result based on message timestamp vs stream start, so back-to-back collects of the same preset response could disagree on that field.
Both affected tests now build the shared response with an explicit durationMs: 0, so the stream leaves timing alone and the two results stay aligned. Comments document why the preset is there.
Reviewed by Cursor Bugbot for commit d9b4464. Bugbot is set up for automated code reviews on this repo. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Lu Nelson · Slack thread
🌟 Purpose
Unit (@apps/brunch-agent)fails intermittently onmainand on unrelated branches. Bilal Mahmoud reported it from this job, where "leaves unrelated provider use and its streaming behavior untouched" failed on adurationMs: 0the other result lacked. "preserves admitted text, Unicode arguments, ids, usage and finish reason" fails the same way.Both tests stream one fake response twice and compare the results. Since the upgrade to Pi 1.2, a stream adds
durationMsto the final message only when the message'stimestampis no older than the stream. The fake response takes its timestamp when the test builds it, so each result gains or lacks the field depending on the millisecond its stream starts in. Lu Nelson worked this out in the thread.This PR gives the shared response in both tests a preset
durationMs: 0. A stream leaves a message that already has a duration alone, so both results carry the same value.🔍 Changes
timestamp: 0. Either fix works, but that branch will conflict with this one on that test.