feat(ccwidgets): add e2e tests for real time transcript and real time assist #729
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -46,7 +46,11 @@ const RealTimeTranscriptComponent: React.FC<RealTimeTranscriptComponentProps> = | |
| ) : null} | ||
| </Text> | ||
| ) : null} | ||
| <div | ||
| className="real-time-transcript__item" | ||
| data-testid="real-time-transcript:item" | ||
| data-speaker-role={entry.isCustomer ? 'customer' : 'agent'} | ||
|
Comment on lines
+51
to
+52
There was a problem hiding this comment.
The new E2E helpers depend on AGENTS.md reference: AGENTS.md:L68-L68 Useful? React with 👍 / 👎. |
||
| > | ||
| <div className="real-time-transcript__avatar-wrap"> | ||
| <Avatar | ||
| className="real-time-transcript__avatar-fallback" | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,124 @@ | ||
| import {Page, expect, Locator} from '@playwright/test'; | ||
| import {AWAIT_TIMEOUT, AI_ASSIST_SUGGESTION_TIMEOUT} from '../constants'; | ||
|
|
||
| /** | ||
| * Utility functions for exercising the AI Assistant widget (launcher, landing | ||
| * page, Real-Time Assist chat, and adaptive-card feedback controls) in e2e | ||
| * tests. | ||
| * | ||
| * The suggestion content itself is generated by a live backend AI pipeline | ||
| * during the test call, so these helpers deliberately avoid asserting on | ||
| * exact transcript/suggestion text and instead verify structure and state | ||
| * transitions (panel open, spinner clears, chat entries appear, feedback | ||
| * controls toggle). | ||
| * | ||
| * @packageDocumentation | ||
| */ | ||
|
|
||
| /** | ||
| * Enables the AI Assistant widget via its sample-app checkbox. | ||
| * Unlike most widgets, AI Assistant defaults to unchecked, so this must be | ||
| * called explicitly before the widget will render. | ||
| * @param page - The Playwright page object | ||
| */ | ||
| export async function enableAIAssistantWidget(page: Page): Promise<void> { | ||
| const checkbox = page.getByTestId('samples:widget-aiAssistant'); | ||
| const isChecked = await checkbox.isChecked().catch(() => false); | ||
| if (!isChecked) { | ||
| await checkbox.check({timeout: AWAIT_TIMEOUT}); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Opens the AI Assistant panel by clicking its launcher button. | ||
| * No-ops if the panel is already open (launcher only renders when closed). | ||
| * @param page - The Playwright page object | ||
| */ | ||
| export async function openAIAssistant(page: Page): Promise<void> { | ||
| const launcher = page.getByTestId('ai-assistant:launcher'); | ||
| if (await launcher.isVisible().catch(() => false)) { | ||
| await launcher.click({timeout: AWAIT_TIMEOUT}); | ||
| } | ||
| await expect(page.getByTestId('ai-assistant:panel')).toBeVisible({timeout: AWAIT_TIMEOUT}); | ||
| } | ||
|
|
||
| /** | ||
| * Closes the AI Assistant panel via its header close button. | ||
| * @param page - The Playwright page object | ||
| */ | ||
| export async function closeAIAssistant(page: Page): Promise<void> { | ||
| const closeButton = page.getByTestId('ai-assistant:header-close'); | ||
| if (await closeButton.isVisible().catch(() => false)) { | ||
| await closeButton.click({timeout: AWAIT_TIMEOUT}); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Returns true while the landing page (feature disabled / no active | ||
| * interaction) is shown instead of the Real-Time Assist chat. | ||
| * @param page - The Playwright page object | ||
| */ | ||
| export async function isShowingLanding(page: Page): Promise<boolean> { | ||
| return page | ||
| .getByTestId('ai-assistant:landing') | ||
| .isVisible() | ||
| .catch(() => false); | ||
| } | ||
|
|
||
| /** | ||
| * Clicks "Get Suggestions" and waits for the request to settle: either the | ||
| * chat/context-form appears (success) or the inline error message appears | ||
| * (failure). Does not throw on failure - callers assert the outcome. | ||
| * @param page - The Playwright page object | ||
| */ | ||
| export async function requestRealTimeAssistSuggestions(page: Page): Promise<void> { | ||
| const getSuggestionsButton = page.getByTestId('ai-assistant:get-suggestions'); | ||
| await expect(getSuggestionsButton).toBeVisible({timeout: AWAIT_TIMEOUT}); | ||
| await getSuggestionsButton.click({timeout: AWAIT_TIMEOUT}); | ||
|
|
||
| // The button is replaced by a spinner while the request is in flight; | ||
| // wait for the request to settle one way or another. | ||
| await Promise.race([ | ||
| page.getByTestId('ai-assistant:context-form').waitFor({state: 'visible', timeout: AI_ASSIST_SUGGESTION_TIMEOUT}), | ||
| page.getByTestId('ai-assistant:error').waitFor({state: 'visible', timeout: AI_ASSIST_SUGGESTION_TIMEOUT}), | ||
| ]).catch(() => {}); | ||
| } | ||
|
|
||
| /** | ||
| * Waits for at least one assistant suggestion (adaptive card or greeting) to | ||
| * appear in the Real-Time Assist chat. | ||
| * @param page - The Playwright page object | ||
| * @returns Locator for the first assistant chat entry | ||
| */ | ||
| export async function waitForFirstSuggestion(page: Page): Promise<Locator> { | ||
| const assistantEntry = page.getByTestId('ai-assistant:chat-assistant').first(); | ||
| await assistantEntry.waitFor({state: 'visible', timeout: AI_ASSIST_SUGGESTION_TIMEOUT}); | ||
| return assistantEntry; | ||
| } | ||
|
|
||
| /** | ||
| * Clicks the like/dislike feedback control (identified by its accessible | ||
| * name) on the first rendered suggestion card and returns its locator so the | ||
| * caller can assert on the resulting `data-active` state. | ||
| * @param page - The Playwright page object | ||
| * @param kind - Which control to click | ||
| */ | ||
| export async function clickSuggestionFeedback(page: Page, kind: 'like' | 'dislike'): Promise<Locator> { | ||
| const label = kind === 'like' ? 'Like suggestion' : 'Dislike suggestion'; | ||
| const control = page.getByLabel(label).first(); | ||
| await expect(control).toBeVisible({timeout: AWAIT_TIMEOUT}); | ||
| await control.click({timeout: AWAIT_TIMEOUT}); | ||
| return control; | ||
| } | ||
|
|
||
| /** | ||
| * Clicks the copy control on the first rendered suggestion card. | ||
| * @param page - The Playwright page object | ||
| * @returns Locator for the copy control | ||
| */ | ||
| export async function clickSuggestionCopy(page: Page): Promise<Locator> { | ||
| const control = page.getByLabel('Copy suggestion').first(); | ||
| await expect(control).toBeVisible({timeout: AWAIT_TIMEOUT}); | ||
| await control.click({timeout: AWAIT_TIMEOUT}); | ||
| return control; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,143 @@ | ||
| import {Page, Locator} from '@playwright/test'; | ||
| import {TRANSCRIPT_ENTRY_TIMEOUT} from '../constants'; | ||
|
|
||
| /** | ||
| * Utility functions for exercising the Real-Time Transcript widget in e2e | ||
| * tests. | ||
| * | ||
| * The live speech-to-text pipeline (driven by the call's dummy audio) is | ||
| * non-deterministic, so `waitForTranscriptEntry` only verifies that *some* | ||
| * real transcription eventually arrives (an integration smoke check). | ||
| * | ||
| * For precise, deterministic verification of how the widget renders | ||
| * transcript content - in particular the word-by-word progressive rendering | ||
| * of a single utterance as multiple `REAL_TIME_TRANSCRIPTION` events arrive | ||
| * for the same `messageId` - use `dispatchRealtimeTranscriptionEvent` to | ||
| * inject a known mock event directly into the store, the same way the SDK's | ||
| * real event handler does. This requires the sample app's debug hook | ||
| * `window.store` (see `widgets-samples/cc/samples-cc-react-app/src/App.tsx`) | ||
| * and a currently active call (the transcript panel only mounts while | ||
| * `store.currentTask` is set). | ||
| * | ||
| * @packageDocumentation | ||
| */ | ||
|
|
||
| /** | ||
| * Waits for the Real-Time Transcript panel to be visible. | ||
| * The panel only renders while `store.currentTask` exists (an active call). | ||
| * @param page - The Playwright page object | ||
| * @param timeout - Optional timeout override in ms | ||
| */ | ||
| export async function waitForRealTimeTranscriptPanel(page: Page, timeout: number = TRANSCRIPT_ENTRY_TIMEOUT) { | ||
| const root = page.getByTestId('real-time-transcript:root'); | ||
| await root.waitFor({state: 'visible', timeout}); | ||
| return root; | ||
| } | ||
|
|
||
| export type TranscriptSpeakerRole = 'agent' | 'customer'; | ||
|
|
||
| /** | ||
| * Waits for a transcript entry to appear in the live transcript feed and | ||
| * returns it. | ||
| * @param page - The Playwright page object | ||
| * @param role - Optional speaker leg to filter by (`data-speaker-role="agent"` | ||
| * for the agent leg, `"customer"` for the caller leg). When omitted, waits | ||
| * for the first entry of either role. | ||
| * @returns Locator for the first matching transcript item | ||
| */ | ||
| export async function waitForTranscriptEntry(page: Page, role?: TranscriptSpeakerRole): Promise<Locator> { | ||
| const selector = role | ||
| ? `[data-testid="real-time-transcript:item"][data-speaker-role="${role}"]` | ||
| : '[data-testid="real-time-transcript:item"]'; | ||
| const item = page.locator(selector).first(); | ||
| await item.waitFor({state: 'visible', timeout: TRANSCRIPT_ENTRY_TIMEOUT}); | ||
| return item; | ||
| } | ||
|
|
||
| /** SDK role values recognized by `getTranscriptSpeaker` in `task/src/helper.ts`. */ | ||
| export type TranscriptionEventRole = 'agent' | 'caller'; | ||
|
|
||
| /** | ||
| * A single, consistent mock "conversation" used to deterministically verify | ||
| * real-time transcript rendering. Sentences start with a distinctive, | ||
| * clearly-synthetic token so they can never collide with whatever the live | ||
| * speech-to-text pipeline happens to transcribe from the call's dummy audio | ||
| * in the background. | ||
| */ | ||
| export const MOCK_TRANSCRIPT_CONVERSATION: Record< | ||
| 'agent' | 'customer', | ||
| {messageId: string; role: TranscriptionEventRole; sentence: string} | ||
| > = { | ||
| agent: { | ||
| messageId: 'e2e-mock-agent-message-1', | ||
| role: 'agent', | ||
| sentence: 'AgentE2EMock thank you for calling support how can I help you today', | ||
| }, | ||
| customer: { | ||
| messageId: 'e2e-mock-customer-message-1', | ||
| role: 'caller', | ||
| sentence: 'CustomerE2EMock hi I am having trouble logging into my account', | ||
| }, | ||
| }; | ||
|
|
||
| /** | ||
| * Injects a single `REAL_TIME_TRANSCRIPTION` event by calling the store's | ||
| * real event handler directly (`window.store.handleRealtimeTranscription`) - | ||
| * the exact same method the SDK's live event listener invokes. Requires an | ||
| * active call so `store.currentTask` is set and the transcript panel is | ||
| * mounted. | ||
| * @param page - The Playwright page object (must be the agent's page) | ||
| * @param event - The mock transcription payload fields to send | ||
| */ | ||
| export async function dispatchRealtimeTranscriptionEvent( | ||
| page: Page, | ||
| event: {role: TranscriptionEventRole; content: string; isFinal: boolean; messageId: string; utteranceId?: string} | ||
| ): Promise<void> { | ||
| await page.evaluate((evt) => { | ||
| const injectedStore = ( | ||
| window as unknown as { | ||
| store?: {handleRealtimeTranscription?: (payload: unknown) => void}; | ||
| } | ||
| ).store; | ||
|
|
||
| if (!injectedStore?.handleRealtimeTranscription) { | ||
| throw new Error( | ||
| 'window.store.handleRealtimeTranscription is not available - cannot inject a mock transcription event' | ||
| ); | ||
| } | ||
|
|
||
| injectedStore.handleRealtimeTranscription({ | ||
| agentId: 'e2e-agent', | ||
| orgId: 'e2e-org', | ||
| notifType: 'REAL_TIME_TRANSCRIPTION', | ||
| notifDetails: {actionEvent: 'REAL_TIME_TRANSCRIPTION'}, | ||
| data: { | ||
| content: evt.content, | ||
| conversationId: 'e2e-conversation', | ||
| isFinal: evt.isFinal, | ||
| messageId: evt.messageId, | ||
| orgId: 'e2e-org', | ||
| publishTimestamp: Date.now(), | ||
| role: evt.role, | ||
| trackingId: 'e2e-tracking', | ||
| utteranceId: evt.utteranceId || evt.messageId, | ||
| }, | ||
| }); | ||
| }, event); | ||
| } | ||
|
|
||
| /** | ||
| * Locates the transcript message element for a mock utterance injected via | ||
| * `dispatchRealtimeTranscriptionEvent`/`MOCK_TRANSCRIPT_CONVERSATION`, | ||
| * identified by its distinctive leading token (e.g. `"AgentE2EMock"`) so it | ||
| * stays unambiguous even alongside unrelated live-transcribed entries. | ||
| * @param page - The Playwright page object | ||
| * @param role - Which leg's mock entry to locate | ||
| * @param leadingToken - The first word of the mock sentence for that role | ||
| */ | ||
| export function locateMockTranscriptMessage(page: Page, role: TranscriptSpeakerRole, leadingToken: string): Locator { | ||
| return page | ||
| .locator(`[data-testid="real-time-transcript:item"][data-speaker-role="${role}"]`) | ||
| .filter({hasText: leadingToken}) | ||
| .locator('.real-time-transcript__message'); | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Replacing
test:e2ewith only the two suffixed scripts leaves no root script namedtest:e2e, while the modified Playwright README still usesyarn test:e2efor every all-suite, single-suite, project, UI, debug, and headed invocation. Those documented local commands now fail during script resolution before Playwright starts; retain a compatible alias or update the documented callers to usetest:e2e:cc.AGENTS.md reference: AGENTS.md:L78-L78
Useful? React with 👍 / 👎.