Skip to content

Commit 1ce83b8

Browse files
tylerslatonclaude
andcommitted
fix(showcase): close 2 D5 fixture gaps + filter deprecated features from gold-standard view
Three fixes that follow up on PR CopilotKit#4743 to bring LGP closer to fully-green on the dashboard: 1. tool-rendering-reasoning-chain probe: was failing with `expected [data-testid="reasoning-block"] to mount within 30000ms`. Root cause: the demo's `<ReasoningBlock>` slot only mounts when a reasoning-role message lands in the transcript, which requires aimock to emit REASONING_MESSAGE_* events, which in turn requires the fixture's first-leg response to carry a `reasoning` field. The weather/Tokyo and SFO/JFK first-leg fixtures were missing it. Mirrors the convention documented in reasoning-display.json:2. Patched both source (harness/fixtures/d5/) and bundle (aimock/d5-all.json). 2. gen-ui-interrupt source fixture: the source fixture file was missing the resume-leg toolCallId entries that already existed in the bundle. Cosmetic mirror so re-bundling stays consistent. Same chip prompts + same toolCallIds as interrupt-headless.json (both probes share the same agent and aimock fixture set; the difference is the FRONTEND rendering — useInterrupt inline vs useHeadlessInterrupt separate-pane). 3. Dashboard gold-standard filter: 4 deprecated/legacy features (agentic-chat-reasoning, hitl, hitl-in-chat-booking, reasoning-default-render) used to render as X-marked rows in the LGP gold-standard dashboard view because LGP intentionally does NOT implement them — they were consolidated into the modern shape (reasoning-custom + reasoning-default; hitl-in-chat with useHumanInTheLoop). Other 17 integrations still serve those legacy demos, so we don't yank the features from feature-registry.json entirely. Instead: marked them `deprecated: true` and updated generate-registry.ts to skip emitting cells when a deprecated feature is unshipped for an integration. LGP cells: 43 → 39 (the 4 deprecated rows disappear). Other integrations: unchanged (audit trail preserved). Catalog total: 774 → 770. Tests: - 1588/1588 harness vitest passing - 19/19 generate-catalog + generate-registry tests passing (counts updated for the 4 dropped LGP cells + new deprecated- feature filter test) - validate-fixture-tool-surface clean (282 fixtures × 627 demos) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 1ad78c3 commit 1ce83b8

6 files changed

Lines changed: 121 additions & 27 deletions

File tree

showcase/aimock/d5-all.json

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -159,7 +159,8 @@
159159
"name": "search_flights",
160160
"arguments": "{\"origin\":\"SFO\",\"destination\":\"JFK\"}"
161161
}
162-
]
162+
],
163+
"reasoning": "The user wants flights from SFO to JFK. I'll call search_flights with those airports to look up options."
163164
}
164165
},
165166
{
@@ -1192,7 +1193,8 @@
11921193
"name": "get_weather",
11931194
"arguments": "{\"location\":\"Tokyo\"}"
11941195
}
1195-
]
1196+
],
1197+
"reasoning": "The user asked about Tokyo weather. I'll call get_weather with location='Tokyo' to get the current conditions."
11961198
}
11971199
},
11981200
{
@@ -1209,7 +1211,8 @@
12091211
"name": "get-weather",
12101212
"arguments": "{\"location\":\"Tokyo\"}"
12111213
}
1212-
]
1214+
],
1215+
"reasoning": "The user asked about Tokyo weather. I'll call get_weather with location='Tokyo' to get the current conditions."
12131216
}
12141217
},
12151218
{

showcase/harness/fixtures/d5/gen-ui-interrupt.json

Lines changed: 25 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,16 @@
11
{
2-
"_comment": "D5 fixture for /demos/gen-ui-interrupt. Two pill prompts (sales call / 1:1 with Alice). Each triggers the `schedule_meeting` tool which raises a LangGraph interrupt; the frontend renders the time picker, the user picks, and the agent resumes with a confirmation message.",
2+
"_comment": "D5 fixture for /demos/gen-ui-interrupt. Two pill prompts (sales call / 1:1 with Alice). Each pill is a 2-leg flow — first leg returns a `schedule_meeting` tool call (which triggers the Python agent's `interrupt(...)` suspension); second leg matches `toolCallId` after the user picks a slot and returns a confirmation narration. Both legs are SHARED with /demos/interrupt-headless.json (same agent, same chip prompts, different frontend rendering — `useInterrupt` inline vs `useHeadlessInterrupt` in a separate pane). The `call_d5_schedule_sales_001` / `_alice_001` toolCallIds match `interrupt-headless.json` and `showcase/aimock/d5-all.json` so all three sources stay in lockstep.",
33
"fixtures": [
44
{
5-
"match": { "userMessage": "intro call with the sales team" },
5+
"match": {
6+
"userMessage": "intro call with the sales team",
7+
"toolName": "schedule_meeting"
8+
},
69
"response": {
710
"content": "Sure — let me check available times.",
811
"toolCalls": [
912
{
13+
"id": "call_d5_schedule_sales_001",
1014
"name": "schedule_meeting",
1115
"arguments": {
1216
"topic": "Sales intro call",
@@ -21,11 +25,24 @@
2125
}
2226
},
2327
{
24-
"match": { "userMessage": "1:1 with Alice" },
28+
"match": {
29+
"userMessage": "intro call with the sales team",
30+
"toolCallId": "call_d5_schedule_sales_001"
31+
},
32+
"response": {
33+
"content": "Booked: Sales intro call confirmed for the slot you picked."
34+
}
35+
},
36+
{
37+
"match": {
38+
"userMessage": "1:1 with Alice",
39+
"toolName": "schedule_meeting"
40+
},
2541
"response": {
2642
"content": "Got it — pulling up next-week slots.",
2743
"toolCalls": [
2844
{
45+
"id": "call_d5_schedule_alice_001",
2946
"name": "schedule_meeting",
3047
"arguments": {
3148
"topic": "1:1 with Alice — Q2 goals",
@@ -40,15 +57,12 @@
4057
}
4158
},
4259
{
43-
"match": { "userMessage": "request the gen-ui interrupt" },
44-
"response": {
45-
"content": "Choose to continue."
46-
}
47-
},
48-
{
49-
"match": { "userMessage": "confirm the gen-ui choice" },
60+
"match": {
61+
"userMessage": "1:1 with Alice",
62+
"toolCallId": "call_d5_schedule_alice_001"
63+
},
5064
"response": {
51-
"content": "Gen-UI interrupt resolved."
65+
"content": "Scheduled: 1:1 with Alice locked in for the slot you picked."
5266
}
5367
}
5468
]

showcase/harness/fixtures/d5/tool-rendering-reasoning-chain.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"_comment": "D5 fixture for /demos/tool-rendering-reasoning-chain (LangGraph Python). The demo composes a reasoningMessage slot (<ReasoningBlock>) with per-tool renderers (WeatherCard / FlightListCard). Two chip prompts probed: 'What's the weather in Tokyo?' and 'Find flights from SFO to JFK.' — each is a 2-leg flow: first leg emits the tool call (get_weather / search_flights), second leg matches `hasToolResult: true` and emits the deterministic narration. The reasoning tokens themselves come from the agent's `init_chat_model(reasoning={effort:'low'})` — they're produced upstream of these fixtures (the LLM call returns the tool decision; reasoning lives in the same LLM payload). Mirrored verbatim into showcase/aimock/d5-all.json so re-bundling stays consistent.",
2+
"_comment": "D5 fixture for /demos/tool-rendering-reasoning-chain (LangGraph Python). The demo composes a reasoningMessage slot (<ReasoningBlock data-testid=\"reasoning-block\">) with per-tool renderers (WeatherCard / FlightListCard). Two chip prompts probed: 'What's the weather in Tokyo?' and 'Find flights from SFO to JFK.' — each is a 2-leg flow: first leg emits the tool call (get_weather / search_flights) AND a `reasoning` field so aimock emits REASONING_MESSAGE_* events (role='reasoning'); without the reasoning field the v2 <ReasoningBlock> never mounts (mirrors the convention documented in reasoning-display.json). Second leg matches `hasToolResult: true` and emits the deterministic narration. The agent uses `init_chat_model(... use_responses_api=True, reasoning={effort:'low'})` — Responses-API mode delivers reasoning summaries alongside tool calls. Mirrored verbatim into showcase/aimock/d5-all.json so re-bundling stays consistent.",
33
"fixtures": [
44
{
55
"match": {
@@ -8,6 +8,7 @@
88
"toolName": "get_weather"
99
},
1010
"response": {
11+
"reasoning": "The user asked about Tokyo weather. I'll call get_weather with location='Tokyo' to get the current conditions.",
1112
"toolCalls": [
1213
{
1314
"id": "call_d5_get_weather_001",
@@ -32,6 +33,7 @@
3233
"hasToolResult": false
3334
},
3435
"response": {
36+
"reasoning": "The user wants flights from SFO to JFK. I'll call search_flights with those airports to look up options.",
3537
"toolCalls": [
3638
{
3739
"id": "call_tr_flights_sfo_jfk_001",

showcase/scripts/__tests__/generate-catalog.test.ts

Lines changed: 52 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -109,15 +109,22 @@ describe("Catalog Generator", () => {
109109
(c: any) => c.manifestation === "starter",
110110
);
111111

112-
expect(integrated.length).toBe(774); // 43 features x 18 integrations
112+
// 43 features x 18 integrations = 774 nominal, but 4 deprecated
113+
// features (agentic-chat-reasoning, hitl, hitl-in-chat-booking,
114+
// reasoning-default-render) are filtered when an integration's
115+
// manifest doesn't declare them — see deprecatedFeatureIds in
116+
// generate-registry.ts. LGP doesn't declare any of the 4 (gold-
117+
// standard reference uses the modern shape), so 4 LGP cells are
118+
// dropped: 774 - 4 = 770.
119+
expect(integrated.length).toBe(770);
113120
expect(starters.length).toBe(0);
114-
expect(catalog.cells.length).toBe(774);
121+
expect(catalog.cells.length).toBe(770);
115122
// total_cells excludes docs-only features (currently 1 feature x 18 integrations = 18)
116-
expect(catalog.metadata.total_cells).toBe(756);
123+
expect(catalog.metadata.total_cells).toBe(752);
117124
expect(catalog.metadata.docs_only).toBe(18);
118125
});
119126

120-
it("LGP has 43 cells: 38 wired + 1 stub + 4 unshipped", () => {
127+
it("LGP has 39 cells: 38 wired + 1 stub (deprecated features filtered)", () => {
121128
runGenerator();
122129
const catalog = readCatalog();
123130

@@ -126,15 +133,50 @@ describe("Catalog Generator", () => {
126133
c.integration === "langgraph-python" &&
127134
c.manifestation === "integrated",
128135
);
129-
expect(lgpCells.length).toBe(43); // One cell per feature
136+
// 39 = 39 features in LGP manifest (the 4 deprecated features are
137+
// filtered out at catalog-generation time since LGP doesn't declare
138+
// them — see deprecatedFeatureIds in generate-registry.ts).
139+
expect(lgpCells.length).toBe(39);
130140

131141
const wired = lgpCells.filter((c: any) => c.status === "wired");
132142
const stub = lgpCells.filter((c: any) => c.status === "stub");
133143
const unshipped = lgpCells.filter((c: any) => c.status === "unshipped");
134144

135145
expect(wired.length).toBe(38);
136146
expect(stub.length).toBe(1);
137-
expect(unshipped.length).toBe(4);
147+
expect(unshipped.length).toBe(0);
148+
});
149+
150+
it("deprecated features are emitted only for integrations that declare them", () => {
151+
runGenerator();
152+
const catalog = readCatalog();
153+
154+
// 4 features marked deprecated:true in feature-registry.json. LGP
155+
// doesn't declare any of them (modern-shape reference); other
156+
// integrations that DO declare them still get a cell so the
157+
// cross-integration audit trail is preserved.
158+
const deprecated = [
159+
"agentic-chat-reasoning",
160+
"hitl",
161+
"hitl-in-chat-booking",
162+
"reasoning-default-render",
163+
];
164+
165+
for (const feat of deprecated) {
166+
const lgpCell = catalog.cells.find(
167+
(c: any) =>
168+
c.integration === "langgraph-python" && c.feature === feat,
169+
);
170+
expect(lgpCell).toBeUndefined();
171+
}
172+
173+
// ag2 declares agentic-chat-reasoning + hitl + reasoning-default-render
174+
// (per its manifest), so those cells exist.
175+
const ag2Acr = catalog.cells.find(
176+
(c: any) =>
177+
c.integration === "ag2" && c.feature === "agentic-chat-reasoning",
178+
);
179+
expect(ag2Acr).toBeDefined();
138180
});
139181

140182
it("stub detection: LGP/cli-start has stub status (demo exists, no route)", () => {
@@ -196,8 +238,10 @@ describe("Catalog Generator", () => {
196238
const catalog = readCatalog();
197239

198240
expect(catalog.metadata).toBeDefined();
199-
// total_cells excludes docs-only features
200-
expect(catalog.metadata.total_cells).toBe(756);
241+
// total_cells excludes docs-only features (and deprecated-feature cells
242+
// that were filtered during catalog generation when an integration
243+
// didn't declare them).
244+
expect(catalog.metadata.total_cells).toBe(752);
201245

202246
// Headline counts exclude docs-only cells; must sum to total_cells.
203247
expect(

showcase/scripts/generate-registry.ts

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -279,6 +279,7 @@ function generateCatalog(
279279
name: string;
280280
category: string;
281281
kind?: string;
282+
deprecated?: boolean;
282283
}>;
283284
categories: Array<{ id: string; name: string }>;
284285
},
@@ -314,6 +315,21 @@ function generateCatalog(
314315
.map((f) => f.id),
315316
);
316317

318+
// Deprecated features — consolidated/replaced patterns that LGP (the
319+
// gold-standard reference integration) intentionally does NOT implement,
320+
// but legacy integrations still serve. We only EMIT a cell for a
321+
// deprecated feature when the integration's manifest actively declares
322+
// it; integrations that don't claim it (notably the gold-standard
323+
// reference) get no row at all, so the dashboard's gold-standard view
324+
// doesn't render confusing X-marked rows for patterns LGP intentionally
325+
// skipped. The cross-integration audit trail is preserved on the
326+
// integrations that DO declare the feature.
327+
const deprecatedFeatureIds = new Set(
328+
featureRegistry.features
329+
.filter((f) => f.deprecated === true)
330+
.map((f) => f.id),
331+
);
332+
317333
// Step 1: Cross-join to produce integrated cells and collect wired features
318334
// and unsupported features per integration.
319335
const wiredFeaturesPerIntegration = new Map<string, Set<string>>();
@@ -328,6 +344,17 @@ function generateCatalog(
328344

329345
for (const featureId of allFeatureIds) {
330346
const status = determineCellStatus(featureId, integration);
347+
348+
// Deprecated-feature filter: skip emitting the cell entirely when
349+
// the feature is deprecated AND this integration's manifest does
350+
// NOT declare it. The cell would otherwise render as an
351+
// unshipped/X row that's misleading — it implies LGP is missing
352+
// something it intentionally doesn't have. Integrations that DO
353+
// declare the deprecated feature still get their row (audit trail).
354+
if (deprecatedFeatureIds.has(featureId) && status === "unshipped") {
355+
continue;
356+
}
357+
331358
if (status === "wired") {
332359
wiredFeatures.add(featureId);
333360
}

showcase/shared/feature-registry.json

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -153,7 +153,8 @@
153153
"description": "Time-picker card rendered inline via useHumanInTheLoop for a booking flow",
154154
"kind": "primary",
155155
"og_docs_url": "https://docs.copilotkit.ai/generative-ui/your-components/interactive",
156-
"shell_docs_path": "/generative-ui/your-components/interactive"
156+
"shell_docs_path": "/generative-ui/your-components/interactive",
157+
"deprecated": true
157158
},
158159
{
159160
"id": "hitl",
@@ -162,7 +163,8 @@
162163
"description": "Original HITL demo kept for backwards compatibility",
163164
"kind": "primary",
164165
"og_docs_url": "https://docs.copilotkit.ai/generative-ui/your-components/interactive",
165-
"shell_docs_path": "/generative-ui/your-components/interactive"
166+
"shell_docs_path": "/generative-ui/your-components/interactive",
167+
"deprecated": true
166168
},
167169
{
168170
"id": "interrupt-headless",
@@ -254,7 +256,8 @@
254256
"category": "operational-generative-ui",
255257
"description": "Visible reasoning/thinking chain alongside the final answer",
256258
"og_docs_url": "https://docs.copilotkit.ai/custom-look-and-feel/reasoning-messages",
257-
"shell_docs_path": "/custom-look-and-feel/reasoning-messages"
259+
"shell_docs_path": "/custom-look-and-feel/reasoning-messages",
260+
"deprecated": true
258261
},
259262
{
260263
"id": "reasoning-default-render",
@@ -263,7 +266,8 @@
263266
"description": "Built-in CopilotChatReasoningMessage renders without a custom slot",
264267
"kind": "testing",
265268
"shell_docs_path": "/custom-look-and-feel/reasoning-messages",
266-
"og_docs_url": "https://docs.copilotkit.ai/custom-look-and-feel/reasoning-messages"
269+
"og_docs_url": "https://docs.copilotkit.ai/custom-look-and-feel/reasoning-messages",
270+
"deprecated": true
267271
},
268272
{
269273
"id": "reasoning-default",

0 commit comments

Comments
 (0)