Skip to content

Commit e69e67e

Browse files
committed
Fix 5 shared test failures in auth, chat-customization-css, and chat-slots
Two root causes: 1. Tests used messages ("Hello", "Hi", "hello", "Say something short") that don't match any aimock fixture. With --proxy-only mode, unmatched requests fall through to real OpenAI which rejects the mock API key (sk-mock-local-dev) with 502/401. Replaced all test messages with exact d5-all.json fixture entries: "Say hello in one short sentence", "Tell me a one-line joke", "Give me a fun fact". 2. The "second assistant turn" test in chat-slots sent its second message immediately after the first assistant bubble appeared. The assistant message becomes visible on the first streaming chunk, but the chat input stays disabled until the full stream ends (aimock streams at 60ms/8-char-chunk). Added a text-stabilization poll between turns to wait for streaming to finish before sending the next message. All tests copied identically to both LGP and LGT. Verified 16/16 pass on both ports (3100 and 3101) across multiple runs.
1 parent 54717e6 commit e69e67e

6 files changed

Lines changed: 72 additions & 26 deletions

File tree

showcase/integrations/langgraph-python/tests/e2e/auth.spec.ts

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ test.describe("Authentication", () => {
5858
await expect(page.getByPlaceholder("Type a message")).toBeVisible();
5959

6060
const input = page.getByPlaceholder("Type a message");
61-
await input.fill("Hello");
61+
await input.fill("Say hello in one short sentence");
6262
await input.press("Enter");
6363

6464
await expect(
@@ -109,7 +109,7 @@ test.describe("Authentication", () => {
109109
// After sign-out, the next send must surface the rejection — not
110110
// produce an assistant response and not white-screen the page.
111111
const input = page.getByPlaceholder("Type a message");
112-
await input.fill("Hello again");
112+
await input.fill("Tell me a one-line joke");
113113
await input.press("Enter");
114114

115115
const errorSurface = page.locator('[data-testid="auth-demo-error"]');
@@ -144,7 +144,7 @@ test.describe("Authentication", () => {
144144

145145
// Chat is responsive again.
146146
const input = page.getByPlaceholder("Type a message");
147-
await input.fill("Hello again");
147+
await input.fill("Give me a fun fact");
148148
await input.press("Enter");
149149
await expect(
150150
page.locator('[data-testid="copilot-assistant-message"]').first(),

showcase/integrations/langgraph-python/tests/e2e/chat-customization-css.spec.ts

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -66,10 +66,12 @@ test.describe("Chat Customization (CSS)", () => {
6666
test("user bubble uses transparent background with ember left-border after sending a message", async ({
6767
page,
6868
}) => {
69-
// "hello" matches an existing aimock fixture (content response), so this
70-
// exercises a full round-trip through the themed chat and lets us assert
71-
// on the rendered user bubble's computed styles.
72-
await page.getByPlaceholder("Type a message").fill("hello");
69+
// Use a message that matches an aimock fixture to get a deterministic
70+
// response. Lowercase "hello" doesn't reliably match — "Say hello in
71+
// one short sentence" is an exact d5-all.json fixture entry.
72+
await page
73+
.getByPlaceholder("Type a message")
74+
.fill("Say hello in one short sentence");
7375
await page.locator('[data-testid="copilot-send-button"]').first().click();
7476

7577
const userMsg = page
@@ -87,7 +89,9 @@ test.describe("Chat Customization (CSS)", () => {
8789
test("assistant bubble uses transparent background after round-trip", async ({
8890
page,
8991
}) => {
90-
await page.getByPlaceholder("Type a message").fill("hello");
92+
await page
93+
.getByPlaceholder("Type a message")
94+
.fill("Tell me a one-line joke");
9195
await page.locator('[data-testid="copilot-send-button"]').first().click();
9296

9397
const assistant = page

showcase/integrations/langgraph-python/tests/e2e/chat-slots.spec.ts

Lines changed: 24 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -69,9 +69,9 @@ test.describe("Chat Slots", () => {
6969
// Type and send via the send button — Enter-on-textarea was intermittently
7070
// dropping the submit on this deployment. We assert the disclaimer +
7171
// custom assistant wrapper appear once we transition out of the welcome
72-
// state.
72+
// state. Use exact aimock fixture messages to ensure deterministic responses.
7373
const input = page.getByPlaceholder("Type a message");
74-
await input.fill("Hello");
74+
await input.fill("Say hello in one short sentence");
7575
await page.locator('[data-testid="copilot-send-button"]').first().click();
7676

7777
// Assistant replies and is wrapped in the custom slot.
@@ -94,15 +94,34 @@ test.describe("Chat Slots", () => {
9494
const sendBtn = () =>
9595
page.locator('[data-testid="copilot-send-button"]').first();
9696

97-
// Turn 1
98-
await input.fill("Hi");
97+
// Turn 1 — use exact aimock fixture messages for deterministic responses.
98+
await input.fill("Say hello in one short sentence");
9999
await sendBtn().click();
100100
await expect(page.locator(SLOT_LABEL_ASSISTANT).first()).toBeVisible({
101101
timeout: 45000,
102102
});
103103

104+
// Wait for the first turn's stream to fully complete. The assistant
105+
// message becomes visible as soon as the first chunk arrives, but the
106+
// chat input stays in "responding" state until the full stream ends.
107+
// Rather than race with that, wait for the assistant text to stabilize
108+
// (no new content for 2 seconds).
109+
const firstAssistant = page.locator(SLOT_LABEL_ASSISTANT).first();
110+
let previousText = "";
111+
await expect
112+
.poll(
113+
async () => {
114+
const text = (await firstAssistant.textContent()) ?? "";
115+
const stable = text.length > 0 && text === previousText;
116+
previousText = text;
117+
return stable;
118+
},
119+
{ timeout: 30000, intervals: [2000] },
120+
)
121+
.toBe(true);
122+
104123
// Turn 2 — the slot should wrap every assistant turn, not just the first.
105-
await input.fill("Say something short");
124+
await input.fill("Give me a fun fact");
106125
await sendBtn().click();
107126

108127
// Expect at least two custom-wrapped assistant messages.

showcase/integrations/langgraph-typescript/tests/e2e/auth.spec.ts

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ test.describe("Authentication", () => {
5858
await expect(page.getByPlaceholder("Type a message")).toBeVisible();
5959

6060
const input = page.getByPlaceholder("Type a message");
61-
await input.fill("Hello");
61+
await input.fill("Say hello in one short sentence");
6262
await input.press("Enter");
6363

6464
await expect(
@@ -109,7 +109,7 @@ test.describe("Authentication", () => {
109109
// After sign-out, the next send must surface the rejection — not
110110
// produce an assistant response and not white-screen the page.
111111
const input = page.getByPlaceholder("Type a message");
112-
await input.fill("Hello again");
112+
await input.fill("Tell me a one-line joke");
113113
await input.press("Enter");
114114

115115
const errorSurface = page.locator('[data-testid="auth-demo-error"]');
@@ -144,7 +144,7 @@ test.describe("Authentication", () => {
144144

145145
// Chat is responsive again.
146146
const input = page.getByPlaceholder("Type a message");
147-
await input.fill("Hello again");
147+
await input.fill("Give me a fun fact");
148148
await input.press("Enter");
149149
await expect(
150150
page.locator('[data-testid="copilot-assistant-message"]').first(),

showcase/integrations/langgraph-typescript/tests/e2e/chat-customization-css.spec.ts

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -66,10 +66,12 @@ test.describe("Chat Customization (CSS)", () => {
6666
test("user bubble uses transparent background with ember left-border after sending a message", async ({
6767
page,
6868
}) => {
69-
// "hello" matches an existing aimock fixture (content response), so this
70-
// exercises a full round-trip through the themed chat and lets us assert
71-
// on the rendered user bubble's computed styles.
72-
await page.getByPlaceholder("Type a message").fill("hello");
69+
// Use a message that matches an aimock fixture to get a deterministic
70+
// response. Lowercase "hello" doesn't reliably match — "Say hello in
71+
// one short sentence" is an exact d5-all.json fixture entry.
72+
await page
73+
.getByPlaceholder("Type a message")
74+
.fill("Say hello in one short sentence");
7375
await page.locator('[data-testid="copilot-send-button"]').first().click();
7476

7577
const userMsg = page
@@ -87,7 +89,9 @@ test.describe("Chat Customization (CSS)", () => {
8789
test("assistant bubble uses transparent background after round-trip", async ({
8890
page,
8991
}) => {
90-
await page.getByPlaceholder("Type a message").fill("hello");
92+
await page
93+
.getByPlaceholder("Type a message")
94+
.fill("Tell me a one-line joke");
9195
await page.locator('[data-testid="copilot-send-button"]').first().click();
9296

9397
const assistant = page

showcase/integrations/langgraph-typescript/tests/e2e/chat-slots.spec.ts

Lines changed: 24 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -69,9 +69,9 @@ test.describe("Chat Slots", () => {
6969
// Type and send via the send button — Enter-on-textarea was intermittently
7070
// dropping the submit on this deployment. We assert the disclaimer +
7171
// custom assistant wrapper appear once we transition out of the welcome
72-
// state.
72+
// state. Use exact aimock fixture messages to ensure deterministic responses.
7373
const input = page.getByPlaceholder("Type a message");
74-
await input.fill("Hello");
74+
await input.fill("Say hello in one short sentence");
7575
await page.locator('[data-testid="copilot-send-button"]').first().click();
7676

7777
// Assistant replies and is wrapped in the custom slot.
@@ -94,15 +94,34 @@ test.describe("Chat Slots", () => {
9494
const sendBtn = () =>
9595
page.locator('[data-testid="copilot-send-button"]').first();
9696

97-
// Turn 1
98-
await input.fill("Hi");
97+
// Turn 1 — use exact aimock fixture messages for deterministic responses.
98+
await input.fill("Say hello in one short sentence");
9999
await sendBtn().click();
100100
await expect(page.locator(SLOT_LABEL_ASSISTANT).first()).toBeVisible({
101101
timeout: 45000,
102102
});
103103

104+
// Wait for the first turn's stream to fully complete. The assistant
105+
// message becomes visible as soon as the first chunk arrives, but the
106+
// chat input stays in "responding" state until the full stream ends.
107+
// Rather than race with that, wait for the assistant text to stabilize
108+
// (no new content for 2 seconds).
109+
const firstAssistant = page.locator(SLOT_LABEL_ASSISTANT).first();
110+
let previousText = "";
111+
await expect
112+
.poll(
113+
async () => {
114+
const text = (await firstAssistant.textContent()) ?? "";
115+
const stable = text.length > 0 && text === previousText;
116+
previousText = text;
117+
return stable;
118+
},
119+
{ timeout: 30000, intervals: [2000] },
120+
)
121+
.toBe(true);
122+
104123
// Turn 2 — the slot should wrap every assistant turn, not just the first.
105-
await input.fill("Say something short");
124+
await input.fill("Give me a fun fact");
106125
await sendBtn().click();
107126

108127
// Expect at least two custom-wrapped assistant messages.

0 commit comments

Comments
 (0)