fix(showcase/harness): probe-side DOM-signal alignments — flips 5 more D4 cells to D5 (CopilotKit#4732)

## Summary

Six probe-side fixes landing **5 additional D5 flips** on the
langgraph-python column. None of the fixes touch demo or runtime code —
just the harness-side selectors / signal predicates so each probe
asserts on what the published langgraph-python image actually renders
today.

## D5 impact

Stacks on PR CopilotKit#4728 (merged). After this branch (live in PocketBase):

| Cell | Before this PR | After |
|---|---|---|
| `tool-rendering-default-catchall` | red | **green** |
| `beautiful-chat` | red | **green** |
| `reasoning-default` | red | **green** (rolled with reasoning-custom on
the same `d5:reasoning-display` row) |
| `frontend-tools` | regressed in earlier replay sweep | **green**
(toolCallId disambiguation in PR CopilotKit#4728) |

**Net dashboard count: 11 → 5 D4 cells in the langgraph-python column**
(including PR CopilotKit#4724 + CopilotKit#4728's flips).

## Per-probe changes

- **`d5-tool-rendering-default-catchall`** — add a fallback that scans
`[data-testid="copilot-assistant-message"]` bubbles for the literal tool
name + a `done`/`running` status label, alongside the strict testid
contract. Strict path (the `[data-testid="copilot-tool-render"]` wrapper
added in commit `ba60df5d3`) re-engages once `@copilotkit/react-core`
ships a release with it; until then the bubble-text scan is the only
stable hook on 1.56.5.

- **`d5-reasoning-display`** — published built-in
`CopilotChatReasoningMessage` carries no testid. Accept the verbatim
`Thought for` / `Thinking…` header text the slot emits, in addition to
the four pre-existing testid selectors used by the `reasoning-custom`
override.

- **`d5-gen-ui-headless-complete`** — the `SuggestionBar` renders
agent-generated chip phrasing alternately with the static
`useConfigureSuggestions` titles. Switch from `aria-label="Suggestion:
<Title>"` to a per-chip alias list (e.g. `["stock", "aapl"]`) that
matches BOTH forms via case-insensitive CSS attribute substring
(`button[aria-label*="..." i]`).

- **`d5-gen-ui-interrupt`** — widen the post-pick assertion to accept
any of (a) the `time-picker-picked` testid, (b) the visible `Booked`
badge text, or (c) the agent's `scheduled` / `confirmed` resume
continuation. The picked-state Card unmounts as soon as langgraph
resumes after `resolve`, so a 5s testid wait races the unmount; the OR
catches whichever signal lands first.

- **`_beautiful-chat-shared` (toggle-theme)** — two-track signal: pass
on EITHER the `html.dark` class flipping from its initial reading OR the
visible `Theme toggled` assistant content rendering. The class-flip is
the strongest signal but `useFrontendTool`'s dispatch occasionally drops
the handler call without dropping the agent's follow-up content message;
track (b) catches the "tool semantics reached the UI" state in the
meantime.

- **`_beautiful-chat-shared` (search-flights)** — swap the literal
`United Airlines` wait for the short brand label `United`, which appears
in BOTH the FlightCard a2ui surface (when it paints) and the assistant's
`$349 / $289` narration. Price literals stay as the canonical 2-flight
fingerprint.

## Conversation-runner adjustment

Reverse the partial-fix from PR CopilotKit#4724: try chat-input cascade resolution
AT BOOT first (and read baseline at boot too); only defer to
post-`preFill` on auth-shape demos where the cascade fails. Without
this, demos where `preFill` itself fires the message (chip clicks in
headless-complete) had baseline read AFTER the assistant bubble already
appeared and the settle waited indefinitely for further growth that
never came.

## Remaining 5 D4 cells

| Cell | Class |
|---|---|
| `agent-config` | Bucket A — runtime/streaming defect |
| `shared-state-streaming` | Bucket A — same defect class |
| `open-gen-ui-advanced` | Bucket A — same defect class |
| `gen-ui-headless-complete` | First 3 of 4 chip turns now pass; 4th
turn (`revenue` chart) still fails. Suggestion drift across 4 sequential
turns means the 4th chip's aria-label sometimes drifts outside both
alias forms. Worth narrowing once dev-debug-mode confirms what the
4th-turn chip rendered. |
| `gen-ui-interrupt` | Post-pick signal still elusive — `Booked` badge /
`scheduled` continuation are not landing in the rendered DOM after a
slot click in the local environment. Further investigation needed via
debug-mode trace. |

Bucket A is deferred per prior discussion; the other two would benefit
from `debug: true` on the runtime/provider to see exactly what events
fire after the slot click / 4th chip click.

## Test plan

- [x] Full d5 sweep against local replay confirms 5 → 5 red d5 keys
(down from 8 pre-PR; -3 net flips on this branch alone)
- [x] PocketBase reflects current state — 5 D4 cells visible on
dashboard
- [ ] CI's langgraph-python D5 cron passes the same 8 cells on a deploy
of this branch
- [ ] `dashboard.showcase.copilotkit.ai` shows 11 → 5 D4 transitions on
the langgraph-python column once Railway picks this up

Author: AlemTuzlak

showcase/aimock/d5-all.json

@@ -1552,6 +1552,15 @@
         "content": "Agent context is a typed payload published from the frontend on every turn through the useAgentContext hook. The payload is forwarded into the agent's runtime context (LangGraph 0.6 introduced the `context` channel as the supported relay for per-run frontend-supplied data; legacy `properties` flowed via `forwardedProps` and did not land in `RunnableConfig`). On the Python side, CopilotKitMiddleware reads the value off the runtime context, then routes it into the system-prompt builder so the model sees the user's tone, expertise, and length preferences before each call. The result is per-turn behavior change without a model swap."
       }
     },
+    {
+      "match": {
+        "userMessage": "Make the background a sunset gradient",
+        "toolCallId": "call_d5_change_background_sunset"
+      },
+      "response": {
+        "content": "Done — sunset gradient is live."
+      }
+    },
     {
       "match": {
         "userMessage": "Make the background a sunset gradient"
@@ -1560,6 +1569,7 @@
         "content": "Sunset gradient applied — warm orange-to-rose blend.",
         "toolCalls": [
           {
+            "id": "call_d5_change_background_sunset",
             "name": "change_background",
             "arguments": {
               "background": "linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ff6b6b 100%)"
@@ -1568,6 +1578,15 @@
         ]
       }
     },
+    {
+      "match": {
+        "userMessage": "deep green forest gradient",
+        "toolCallId": "call_d5_change_background_forest"
+      },
+      "response": {
+        "content": "Done — forest gradient is live."
+      }
+    },
     {
       "match": {
         "userMessage": "deep green forest gradient"
@@ -1576,6 +1595,7 @@
         "content": "Forest gradient applied — deep green hues.",
         "toolCalls": [
           {
+            "id": "call_d5_change_background_forest",
             "name": "change_background",
             "arguments": {
               "background": "linear-gradient(135deg, #0a3d2e 0%, #166534 50%, #059669 100%)"
@@ -1584,6 +1604,15 @@
         ]
       }
     },
+    {
+      "match": {
+        "userMessage": "navy → magenta cosmic gradient",
+        "toolCallId": "call_d5_change_background_cosmic"
+      },
+      "response": {
+        "content": "Done — cosmic gradient is live."
+      }
+    },
     {
       "match": {
         "userMessage": "navy → magenta cosmic gradient"
@@ -1592,6 +1621,7 @@
         "content": "Cosmic gradient applied — navy fading to magenta.",
         "toolCalls": [
           {
+            "id": "call_d5_change_background_cosmic",
             "name": "change_background",
             "arguments": {
               "background": "linear-gradient(135deg, #1e3a8a 0%, #6b21a8 50%, #9333ea 100%)"
@@ -1799,10 +1829,22 @@
                     "title": "Sales by region",
                     "description": "Q4 — share of total revenue",
                     "data": [
-                      { "label": "North America", "value": 540 },
-                      { "label": "EMEA", "value": 320 },
-                      { "label": "APAC", "value": 210 },
-                      { "label": "LATAM", "value": 90 }
+                      {
+                        "label": "North America",
+                        "value": 540
+                      },
+                      {
+                        "label": "EMEA",
+                        "value": 320
+                      },
+                      {
+                        "label": "APAC",
+                        "value": 210
+                      },
+                      {
+                        "label": "LATAM",
+                        "value": 90
+                      }
                     ]
                   }
                 }
@@ -1834,10 +1876,22 @@
                     "title": "Quarterly revenue",
                     "description": "FY24 — USD thousands",
                     "data": [
-                      { "label": "Q1", "value": 820 },
-                      { "label": "Q2", "value": 940 },
-                      { "label": "Q3", "value": 1080 },
-                      { "label": "Q4", "value": 1240 }
+                      {
+                        "label": "Q1",
+                        "value": 820
+                      },
+                      {
+                        "label": "Q2",
+                        "value": 940
+                      },
+                      {
+                        "label": "Q3",
+                        "value": 1080
+                      },
+                      {
+                        "label": "Q4",
+                        "value": 1240
+                      }
                     ]
                   }
                 }
@@ -1874,17 +1928,26 @@
                 {
                   "id": "status-api",
                   "type": "StatusBadge",
-                  "props": { "text": "API: healthy", "variant": "success" }
+                  "props": {
+                    "text": "API: healthy",
+                    "variant": "success"
+                  }
                 },
                 {
                   "id": "status-db",
                   "type": "StatusBadge",
-                  "props": { "text": "Database: healthy", "variant": "success" }
+                  "props": {
+                    "text": "Database: healthy",
+                    "variant": "success"
+                  }
                 },
                 {
                   "id": "status-workers",
                   "type": "StatusBadge",
-                  "props": { "text": "Workers: degraded", "variant": "warning" }
+                  "props": {
+                    "text": "Workers: degraded",
+                    "variant": "warning"
+                  }
                 }
               ],
               "data": {}

showcase/aimock/d5-recorded/.gitignore

@@ -0,0 +1,6 @@
+# aimock writes per-call fixture files to `recorded/` while it runs in
+# `--record` mode. The `record-d5-fixtures.mjs` orchestrator merges those
+# into the per-demo `<slug>.json` files at the parent level and deletes
+# the per-call files. Anything left in `recorded/` is recording-session
+# scratch and must not be committed.
+recorded/

showcase/docker-compose.record.yml

@@ -0,0 +1,48 @@
+# Record-mode override for aimock. Layer on top of docker-compose.local.yml
+# to flip aimock from replay-only into record+replay against real OpenAI /
+# Anthropic. Recorded fixtures land on the host at
+# `showcase/aimock/d5-recorded/recorded/` (one file per LLM call).
+#
+# Usage:
+#   docker compose -f showcase/docker-compose.local.yml \
+#                  -f showcase/docker-compose.record.yml \
+#                  --profile infra up -d aimock langgraph-python
+#
+# When recording is finished, switch back to base compose (no override) and
+# load any new fixtures from `d5-recorded/` via an additional `--fixtures`
+# entry on the aimock command.
+#
+# Authorization headers from the integration container are forwarded to the
+# real provider — no key needs to be set on aimock itself.
+services:
+  aimock:
+    volumes:
+      # RW mount so aimock can write `recorded/<provider>-<ts>-<uuid>.json`
+      # under this dir. Existing baseline fixtures stay :ro from the base
+      # compose; only the recording dir is writable.
+      - ./aimock/d5-recorded:/showcase-fixtures/d5-recorded:rw
+    command: [
+        "--port",
+        "4010",
+        "--host",
+        "0.0.0.0",
+        "--record",
+        "--provider-openai",
+        "https://api.openai.com",
+        "--provider-anthropic",
+        "https://api.anthropic.com",
+        # Load ONLY the recorded dir as the fixture source. The baseline
+        # d5-all.json / feature-parity.json fixtures are intentionally
+        # NOT loaded — if they were, aimock would short-circuit on
+        # already-matched prompts and never proxy them to the real
+        # provider, defeating the recording pass.
+        #
+        # smoke.json is also dropped here; the langgraph-python container
+        # doesn't use it during d5 probes, only the dedicated /api/smoke
+        # route which we aren't exercising.
+        #
+        # First --fixtures path is the recording destination base; aimock
+        # writes to `<base>/recorded/<provider>-<ts>-<uuid>.json`.
+        "--fixtures",
+        "/showcase-fixtures/d5-recorded",
+      ]

showcase/docker-compose.replay.yml

@@ -0,0 +1,32 @@
+# Replay-mode override for aimock — load existing baseline fixtures PLUS
+# the freshly-recorded per-demo files in `aimock/d5-recorded/*.json`. No
+# `--record` flag, no provider URLs: every request must match a loaded
+# fixture or aimock returns 404.
+#
+# Usage:
+#   docker compose -f showcase/docker-compose.local.yml \
+#                  -f showcase/docker-compose.replay.yml \
+#                  --profile infra up -d aimock langgraph-python
+services:
+  aimock:
+    volumes:
+      # Keep the recorded directory mounted (read-only is fine here — we're
+      # not recording in this mode) so aimock loads the per-demo files.
+      - ./aimock/d5-recorded:/showcase-fixtures/d5-recorded:ro
+    command: [
+        "--port",
+        "4010",
+        "--host",
+        "0.0.0.0",
+        # Recorded per-demo fixtures first so they take precedence over the
+        # baseline files when match criteria collide. aimock's matcher
+        # iterates fixtures in registration order.
+        "--fixtures",
+        "/showcase-fixtures/d5-recorded",
+        "--fixtures",
+        "/showcase-fixtures/d5-all.json",
+        "--fixtures",
+        "/showcase-fixtures/smoke.json",
+        "--fixtures",
+        "/showcase-fixtures/feature-parity.json",
+      ]

showcase/harness/fixtures/d5/frontend-tools.json

@@ -1,12 +1,24 @@
 {
-  "_comment": "D5 fixture for /demos/frontend-tools. Three per-pill prompts (Sunset/Forest/Cosmic) drive the `change_background` frontend tool. Each fixture emits a tool call with a gradient containing the pill family's signature hex tokens, so the probe can assert the gradient mutated to the right family. Substrings 'Make the background a sunset gradient', 'deep green forest gradient', 'navy → magenta cosmic gradient' are unique per pill.",
+  "_comment": "D5 fixture for /demos/frontend-tools. Three per-pill prompts (Sunset/Forest/Cosmic) drive the `change_background` frontend tool. Each pill has TWO entries: a first-turn (matches by userMessage substring AND no toolCall-id present in the most recent tool message) and a follow-up (matches by userMessage substring AND a static toolCallId). Both criteria are combined so a follow-up never cross-matches a different pill (the matcher iterates top-to-bottom and uses first-match-wins; a follow-up keyed only on toolCallId would re-match the next pill's first-turn request because the prior tool result is still the latest tool message in history). Substrings 'Make the background a sunset gradient', 'deep green forest gradient', 'navy → magenta cosmic gradient' are unique per pill. See showcase/harness/fixtures/d5/README.md (\"Mid-loop re-invocations after a tool call\").",
   "fixtures": [
     {
-      "match": { "userMessage": "Make the background a sunset gradient" },
+      "match": {
+        "userMessage": "Make the background a sunset gradient",
+        "toolCallId": "call_d5_change_background_sunset"
+      },
+      "response": {
+        "content": "Done — sunset gradient is live."
+      }
+    },
+    {
+      "match": {
+        "userMessage": "Make the background a sunset gradient"
+      },
       "response": {
         "content": "Sunset gradient applied — warm orange-to-rose blend.",
         "toolCalls": [
           {
+            "id": "call_d5_change_background_sunset",
             "name": "change_background",
             "arguments": {
               "background": "linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ff6b6b 100%)"
@@ -16,11 +28,23 @@
       }
     },
     {
-      "match": { "userMessage": "deep green forest gradient" },
+      "match": {
+        "userMessage": "deep green forest gradient",
+        "toolCallId": "call_d5_change_background_forest"
+      },
+      "response": {
+        "content": "Done — forest gradient is live."
+      }
+    },
+    {
+      "match": {
+        "userMessage": "deep green forest gradient"
+      },
       "response": {
         "content": "Forest gradient applied — deep green hues.",
         "toolCalls": [
           {
+            "id": "call_d5_change_background_forest",
             "name": "change_background",
             "arguments": {
               "background": "linear-gradient(135deg, #0a3d2e 0%, #166534 50%, #059669 100%)"
@@ -30,11 +54,23 @@
       }
     },
     {
-      "match": { "userMessage": "navy → magenta cosmic gradient" },
+      "match": {
+        "userMessage": "navy → magenta cosmic gradient",
+        "toolCallId": "call_d5_change_background_cosmic"
+      },
+      "response": {
+        "content": "Done — cosmic gradient is live."
+      }
+    },
+    {
+      "match": {
+        "userMessage": "navy → magenta cosmic gradient"
+      },
       "response": {
         "content": "Cosmic gradient applied — navy fading to magenta.",
         "toolCalls": [
           {
+            "id": "call_d5_change_background_cosmic",
             "name": "change_background",
             "arguments": {
               "background": "linear-gradient(135deg, #1e3a8a 0%, #6b21a8 50%, #9333ea 100%)"

showcase/harness/src/probes/helpers/conversation-runner.ts

@@ -274,18 +274,35 @@ export async function runConversation(
   // ever got a chance to dismiss it. Subsequent turns reuse the cached
   // selector so we don't re-probe per turn.
   let chatInputSelector: string | null = null;
-  // Initial assistant-message count is also computed lazily (after
-  // preFill) for the same reason: on idiomatic-shape auth demos the
-  // chat tree isn't mounted until preFill clicks sign-in, so reading
-  // before preFill would always observe 0 (correct here, but the read
-  // belongs alongside the resolution it pairs with).
+  // Try to resolve the chat input AT BOOT first — when it works
+  // (every demo except idiomatic-shape auth), capture the baseline
+  // assistant-message count BEFORE turn 1's preFill runs. Demos like
+  // /demos/headless-complete fire the user message inside preFill (a
+  // chip click), so reading the baseline AFTER preFill would observe
+  // the assistant's response already appended and the settle would
+  // wait forever for further growth that never comes. The deferred
+  // path is only used when boot-time resolution fails — that's
+  // specifically the auth shape, where the chat tree mounts later.
   let baselineCount = 0;
-  console.debug(
-    "[conversation-runner] initial baseline assistant message count",
-    {
-      baselineCount,
-    },
-  );
+  try {
+    chatInputSelector = await resolveChatInputSelector(
+      page,
+      opts.chatInputSelector,
+    );
+    console.debug(
+      "[conversation-runner] resolved chat input selector at boot",
+      { selector: chatInputSelector },
+    );
+    baselineCount = await readMessageCount(page);
+    console.debug(
+      "[conversation-runner] initial baseline assistant message count (boot)",
+      { baselineCount },
+    );
+  } catch {
+    console.debug(
+      "[conversation-runner] chat input cascade did not resolve at boot — deferring to post-preFill (auth shape)",
+    );
+  }
 
   for (let idx = 0; idx < total; idx++) {
     const turn = turns[idx]!;