Skip to content

Commit ef1ca98

Browse files
tylerslatonclaude
andcommitted
feat(showcase/google-adk): QA-blitz parity ports — agent prompts, tool surface, e2e specs
Result of 10 parallel QA agents auditing all 30 active demos against langgraph-python (north-star). Each agent ported drift back to LP-verbatim across three axes: 1. Agent layer - tool_rendering_common.py: rebuilt to LP's surface — get_weather, search_flights(origin, destination), get_stock_price, roll_d20, roll_dice. Removed the ADK-only query_data. - tool_rendering_*_agent.py (4 variants): ported LP's travel/concierge prompt; reasoning-chain variant got LP's chain-two-tools prompt. - beautiful_chat_agent.py: ported LP's per-tool system prompt; added manage_sales_todos / get_sales_todos / generate_a2ui; dropped the redundant schedule_meeting (frontend HITL handles it). - open_gen_ui_agents.py: ported LP's full SYSTEM_PROMPT for both variants, including the Websandbox.connection.remote.* contract for the advanced sandbox demo (was `window.sandbox.*`, which the LP frontend's Websandbox bridge silently no-ops). - byoc_agents.py: fused LP's hashbrown + json-render prompts so the single ADK byoc_agent emits both wire shapes. Aliases exported for a future per-route split. - declarative_gen_ui_agent.py: ported LP's a2ui_dynamic SYSTEM_PROMPT. - a2ui_fixed_agent.py: picked up LP's CopilotKit#4734 regression guard ("exactly ONCE", "do NOT call again"). - agent_config_agent.py: rewrote to read useAgentContext (was state["config"]); reconciled schema to LP's 3-field camelCase {tone, expertise, responseLength} with LP's value enums. - subagents_agent.py: dropped the "running" placeholder; returns plain str so the LP-verbatim frontend's `result?.trim()` works. - hitl_in_app_agent.py / hitl_in_chat_book_call_agent.py: prompts + tool-result shape ({approved, reason}) aligned to LP. - AGUIToolset() added wherever it was missing on the bespoke agents (multimodal, mcp_apps, a2ui_fixed) so frontend-registered tools reach the model. 2. Dedicated runtime routes - copilotkit-multimodal/route.ts (new) — mirrors LP shape with ADK's HttpAgent + AGENT_URL pattern. - copilotkit-agent-config/route.ts (new) — same pattern. - copilotkit-mcp-apps/route.ts — refreshed. 3. Frontend ports (ADK frontend brought to LP-verbatim where it had drifted from the parity blitz state) - tool-rendering family (4 demos): full re-port — WeatherCard, FlightListCard, StockCard, D20Card, ReasoningBlock, CatchallRenderer, suggestions, and the page wiring with all useRenderTool / useDefaultRenderTool / reasoningMessage registrations. - a2ui-fixed-schema, mcp-apps, multimodal: full frontend re-ports with their _components/ Tailwind primitives. - frontend-tools, frontend-tools-async, agent-config: ported LP's component structure (separate Background, NotesCard with query_notes, config-context-relay). - shared-state-read, shared-state-read-write, readonly-state-agent-context: ported LP's demo-layout + _components + suggestions. recipe-card.tsx pulled directly from LP (one QA agent had adapted to Unicode glyphs thinking ADK lacked lucide-react — it doesn't, after the parity blitz). - shared-state-streaming, subagents, hitl-in-app: ported LP's DocumentView / supervisor-activity / TicketsPanel structure. hitl-in-app/page.tsx pulled directly from LP to keep the hyphenated agent slug aligned with the renamed registry key. - auth, hitl-in-chat: ported LP's SignInCard-first auth UX and the time-picker Tailwind port. - prebuilt-popup: pulled LP's main-content + suggestions split. 4. Test fixtures - 30 tests/e2e/<slug>.spec.ts ported from LP, several overwriting stale stubs (shared-state-streaming, subagents, auth, hitl-in-chat, shared-state-read, agent-config). - 30 qa/<slug>.md ported from LP with ADK env-var and registry references substituted (GOOGLE_API_KEY, AGENT_URL, registry.py). - QA3's byoc-hashbrown / byoc-json-render specs renamed to declarative-hashbrown / declarative-json-render with internal URL references substituted (the orchestrator pass had already renamed the demo dirs + manifest entries). Frontend changes from QA agents were filtered: kept where they ported LP-verbatim into ADK, replaced with direct LP pulls where the agent had made ADK-specific adaptations (one Unicode-glyph case, one stale-registry-slug case). Not touched per blitz rules: shared_chat.py, registry.py, manifest.yaml, src/app/api/copilotkit/route.ts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent ac36b01 commit ef1ca98

82 files changed

Lines changed: 5928 additions & 903 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# QA: Declarative Generative UI (A2UI — Fixed Schema) — Google ADK
2+
3+
## Prerequisites
4+
5+
- Demo is deployed and accessible at `/demos/a2ui-fixed-schema` on the dashboard host
6+
- Agent backend is healthy; `GOOGLE_API_KEY` is set on Railway; `AGENT_URL` points at the ADK agent server exposing the `a2ui_fixed_schema` endpoint (registered as agent name `a2ui-fixed-schema` — see `src/app/api/copilotkit-a2ui-fixed-schema/route.ts`)
7+
- Note: the demo source contains no `data-testid` attributes. Checks below rely on verbatim visible text, DOM structure, and the JSON schema at `src/agents/a2ui_schemas/flight_schema.json`
8+
9+
## Test Steps
10+
11+
### 1. Basic Functionality
12+
13+
- [ ] Navigate to `/demos/a2ui-fixed-schema`; verify the page renders within 3s and a single `CopilotChat` pane is centered (max-width ~896px, rounded-2xl, full-height)
14+
- [ ] Verify the chat is wired to `runtimeUrl="/api/copilotkit-a2ui-fixed-schema"` and `agent="a2ui-fixed-schema"` (DevTools → Network: sending a message hits that endpoint)
15+
- [ ] Verify the single suggestion pill is visible with verbatim title "Find SFO → JFK" (message body: "Find me a flight from SFO to JFK on United for $289.")
16+
- [ ] Send "Hello" and verify an assistant text response appears within 10s (no flight card for plain text)
17+
18+
### 2. Feature-Specific Checks
19+
20+
#### Schema Wiring (fixed catalog + `includeBasicCatalog`)
21+
22+
- [ ] DevTools → Network: after the first successful `display_flight` call, verify the response stream contains an `a2ui_operations` container with `catalogId: "copilotkit://flight-fixed-catalog"` (matches `CATALOG_ID` in `src/agents/a2ui_fixed_agent.py` and `src/app/demos/a2ui-fixed-schema/a2ui/catalog.ts`)
23+
- [ ] Verify the same container carries the full `FLIGHT_SCHEMA` component tree (12 nodes from `src/agents/a2ui_schemas/flight_schema.json`: `root`, `content`, `title`, `route`, `from`, `arrow`, `to`, `meta`, `airline`, `price`, `bookButton`, `bookButtonLabel`)
24+
25+
#### Search-Flights Prompt (`display_flight` tool → `flight_schema.json`)
26+
27+
- [ ] Click the "Find SFO → JFK" suggestion; within 20s verify a single flight card renders in-transcript assembled per `flight_schema.json`:
28+
- outer `Card` with a `Column` of children in this order: title row, route row, meta row, book button
29+
- `Title` node renders the literal text "Flight Details" (1.15rem / 600 weight, color `#010507`)
30+
- `route` row shows `Airport` "SFO" → `Arrow` (``, color `#AFAFB7`) → `Airport` "JFK" (both monospaced, 1.5rem, 600 weight, 0.05em letter-spacing)
31+
- `meta` row shows `AirlineBadge` "UNITED" (uppercase pill, lilac `#BEC2FF` border, 0.08em tracking) on the left and `PriceTag` "$289" (monospaced, color `#189370`, 1.1rem / 600 weight) on the right
32+
- `Button` renders full-width with label "Book flight" (black `#010507` background, white text, 12px radius)
33+
- [ ] Verify all four data-model fields resolved correctly (origin=`SFO`, destination=`JFK`, airline=`United`, price=`$289`) — each is a `{ path: "/..." }` binding in the schema and must reach the DOM as a plain string via the `GenericBinder` (no literal `{path}` leak and no React error #31)
34+
35+
#### Book-Flight Button (inert — pure presentation)
36+
37+
- [ ] Verify the "Book flight" button is rendered with the schema-declared label and is clickable, but the click is a no-op: the agent is not invoked, no schema swap occurs, and the button does not transition to a "Booked" state. Schema-swap-on-action will be wired up once the A2UI Python SDK exposes `action_handlers=` on `a2ui.render` (see comment in `src/agents/a2ui_fixed_agent.py`).
38+
39+
#### Follow-up Prompt (data-model refresh)
40+
41+
- [ ] Send "Find me a flight from LAX to ORD on Delta for $412."; within 20s verify the card updates in place with origin=`LAX`, destination=`ORD`, airline=`DELTA`, price=`$412` (same schema, new data model — proves the fixed-schema pattern: schema once, data streams)
42+
43+
### 3. Error Handling
44+
45+
- [ ] Send an empty message; verify it is a no-op (no user bubble, no assistant response)
46+
- [ ] Send "What is the capital of France?"; verify the agent replies in plain text without invoking `display_flight` (no flight card rendered, no `a2ui_operations` in the response)
47+
- [ ] DevTools → Console: walk through all flows above; verify no uncaught errors and specifically no React error #31 ("objects are not valid as a React child, found: object with keys {path}") — the `DynString` union in `a2ui/definitions.ts` is what prevents this, so a single occurrence is a regression
48+
49+
## Expected Results
50+
51+
- Chat loads within 3s; plain-text response within 10s; flight card renders within 20s of the search prompt
52+
- `display_flight` is called exactly once per search prompt; result contains an `a2ui_operations` container with `catalogId: "copilotkit://flight-fixed-catalog"` and the full 12-node flight schema
53+
- All custom renderers in `a2ui/renderers.tsx` (`Card`, `Title`, `Airport`, `Arrow`, `AirlineBadge`, `PriceTag`, `Button`) render at least once per search-flights run
54+
- Clicking "Book flight" is a no-op (inert presentation button — see comment in `src/agents/a2ui_fixed_agent.py`)
55+
- No UI layout breaks, no `{path}` leak into the DOM, no uncaught console errors
Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# QA: Agent Config Object — Google ADK
2+
3+
## Prerequisites
4+
5+
- Demo deployed and accessible at `/demos/agent-config`
6+
- Railway service `showcase-google-adk` healthy
7+
- `GOOGLE_API_KEY` set on Railway
8+
9+
## Test Steps
10+
11+
### 1. Initial state
12+
13+
- [ ] Navigate to `/demos/agent-config`
14+
- [ ] Header "Agent Config Object" visible
15+
- [ ] `agent-config-card` is visible with the heading "Agent Config"
16+
- [ ] Tone dropdown (`data-testid="agent-config-tone-select"`) shows "professional"
17+
- [ ] Expertise dropdown (`data-testid="agent-config-expertise-select"`) shows "intermediate"
18+
- [ ] Response length dropdown (`data-testid="agent-config-length-select"`) shows "concise"
19+
- [ ] `<CopilotChat />` composer visible below the card
20+
21+
### 2. Default send
22+
23+
- [ ] Type "Tell me about black holes" and send
24+
- [ ] Agent responds within 15 seconds
25+
- [ ] Response is brief (1-3 sentences), professional tone, no emoji (consistent with the default config)
26+
27+
### 3. Enthusiastic + detailed
28+
29+
- [ ] Change Tone to "enthusiastic"
30+
- [ ] Change Response length to "detailed"
31+
- [ ] Verify both select values updated in the DOM
32+
- [ ] Send "Tell me about black holes" again
33+
- [ ] Response is noticeably longer (multiple paragraphs) and uses upbeat / energetic language
34+
- [ ] Compare to Step 2's response — the style difference is visible
35+
36+
### 4. Beginner expertise
37+
38+
- [ ] Change Expertise to "beginner"
39+
- [ ] Send "What is quantum entanglement?"
40+
- [ ] Response defines jargon and uses analogies
41+
42+
### 5. Expert expertise
43+
44+
- [ ] Change Expertise to "expert"
45+
- [ ] Send the same question
46+
- [ ] Response uses precise terminology, skips basics
47+
48+
### 6. Reactivity mid-thread
49+
50+
- [ ] Without reloading the page, with previous replies visible, change Tone to "casual"
51+
- [ ] Send a follow-up
52+
- [ ] Reply reflects the casual tone; previous replies in the transcript remain unchanged
53+
54+
### 7. Error handling
55+
56+
- [ ] Send an empty message; verify no-op or graceful empty-message handling
57+
- [ ] Verify no console errors during any of the above steps
58+
59+
## Expected Results
60+
61+
- Dropdown value changes appear in the DOM within 100ms of selection
62+
- Agent responses arrive within 15s per send
63+
- Visible style differences across tone / expertise / length changes (qualitative, but clear side-by-side)
64+
- Transcript preserves history when config changes mid-thread
Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# QA: Authentication — Google ADK
2+
3+
## Prerequisites
4+
5+
- Demo deployed and accessible at /demos/auth
6+
- ADK agent backend healthy (check /api/health)
7+
- GOOGLE_API_KEY set in the deployment
8+
9+
## Test Steps
10+
11+
### 1. Initial unauthenticated state (SignInCard)
12+
13+
- [ ] Navigate to /demos/auth (in a fresh browser session — clear
14+
`localStorage["copilotkit:auth-demo:token"]` if it exists)
15+
- [ ] Verify the SignInCard is visible (`data-testid="auth-sign-in-card"`)
16+
- [ ] Verify the demo token is displayed in plain text
17+
(`data-testid="auth-demo-token"`)
18+
- [ ] Verify the "Sign in with demo token" button is enabled
19+
(`data-testid="auth-sign-in-button"`)
20+
- [ ] Verify the AuthBanner is NOT yet rendered
21+
(`data-testid="auth-banner"` absent)
22+
- [ ] Verify the chat composer is NOT yet rendered (no "Type a message" placeholder)
23+
- [ ] Verify no console errors on page load
24+
25+
### 2. Signing in mounts the chat surface
26+
27+
- [ ] Click "Sign in with demo token"
28+
- [ ] Verify the AuthBanner appears with the green/success variant
29+
(`data-authenticated="true"`)
30+
- [ ] Verify the auth-status text reads "✓ Signed in as demo user"
31+
- [ ] Verify the "Sign out" button is visible and enabled
32+
(`data-testid="auth-sign-out-button"`)
33+
- [ ] Verify the SignInCard is gone (`data-testid="auth-sign-in-card"` absent)
34+
- [ ] Verify the chat composer is mounted ("Type a message" placeholder visible)
35+
- [ ] Verify no `auth-demo-error` surface
36+
37+
### 3. Authenticated send → assistant response
38+
39+
- [ ] Type "Hello" and press Enter
40+
- [ ] Within 30 seconds, an assistant message is rendered
41+
(`[data-role="assistant"]`)
42+
- [ ] No auth-demo-error surface appears
43+
44+
### 4. Sign out flips banner amber, keeps chat mounted
45+
46+
- [ ] Click "Sign out"
47+
- [ ] Within 1 second, the banner flips to the amber variant
48+
(`data-authenticated="false"`)
49+
- [ ] Verify the auth-status text reads
50+
"⚠ Signed out — the agent will reject your messages until you sign in."
51+
- [ ] Verify the "Sign in" button is visible
52+
(`data-testid="auth-authenticate-button"`)
53+
- [ ] Verify the chat composer is STILL visible — page must not return
54+
to the SignInCard
55+
- [ ] Verify no white-screen
56+
57+
### 5. Unauthenticated send surfaces a 401 without crashing
58+
59+
- [ ] After signing out, type "Hello again" and press Enter
60+
- [ ] Within 15 seconds, the page-level error surface appears:
61+
`data-testid="auth-demo-error"` with the runtime's 401 message
62+
- [ ] Verify the banner is STILL visible
63+
- [ ] Verify no assistant response appears
64+
65+
### 6. Re-signing in clears the error and resumes chat
66+
67+
- [ ] Click "Sign in" (amber-state authenticate button)
68+
- [ ] Within 1 second, the banner flips back to green
69+
- [ ] Verify the `auth-demo-error` surface is cleared
70+
- [ ] Type "Hello" and press Enter; an assistant response appears within 30s
71+
72+
### 7. Hard-reload behavior
73+
74+
- [ ] Hard-reload the page (clear cache + reload)
75+
- [ ] With a valid `localStorage` token, the demo lands directly in the
76+
authenticated chat state (no SignInCard)
77+
- [ ] With `localStorage` cleared, the demo lands on the SignInCard
78+
79+
### 8. Error Handling
80+
81+
- [ ] With DevTools Network panel blocking `/api/copilotkit-auth`, send a
82+
message while authenticated
83+
- [ ] Verify a network-level error surfaces cleanly (no uncaught promise
84+
rejection in console)
85+
- [ ] Restore network; verify sends work again without a page reload
86+
87+
## Expected Results
88+
89+
- Initial paint shows SignInCard (no 401 crash on `/info`)
90+
- Banner state flips within 1s of Sign out / Sign in clicks
91+
- Post-sign-out sends produce a visible 401 error within 15s via
92+
`auth-demo-error`
93+
- Page never white-screens after sign out — banner and composer stay mounted
94+
- Authenticated sends produce an assistant response within 30s
95+
- Token persists across reloads via localStorage; clearing localStorage
96+
returns the demo to its first-paint SignInCard state
Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# QA: Chat Customization (Slots) — Google ADK
2+
3+
## Prerequisites
4+
5+
- Demo is deployed and accessible at `/demos/chat-slots` on the dashboard host
6+
- Agent backend is healthy (`/api/health` or `/api/copilotkit` GET); `GOOGLE_API_KEY` is set on Railway; the ADK agent server has `chat_slots` registered against the neutral `_simple_chat` agent
7+
- Note: this demo DOES include `data-testid` attributes on every custom slot. Use them as the primary selectors. The underlying agent is the neutral "helpful, concise assistant" (no frontend tools, no agent tools) — this demo exercises frontend slot customization only.
8+
9+
## Test Steps
10+
11+
### 1. Basic Functionality
12+
13+
- [ ] Navigate to `/demos/chat-slots`; verify the page renders a centered chat surface (max-width 4xl, full viewport height) within 3s
14+
- [ ] Verify the custom welcome screen is visible (`data-testid="custom-welcome-screen"`), replacing the default welcome
15+
- [ ] Verify the gradient welcome card displays "Welcome to the Slots demo" with the "Custom Slot" badge and the body text referencing the `welcomeScreen` slot
16+
- [ ] Verify the welcome card wraps the default chat `input` element and a `suggestionView` row beneath it (both passed in as props by CopilotChatView)
17+
18+
### 2. Feature-Specific Checks
19+
20+
#### Welcome Screen Slot (`welcomeScreen`)
21+
22+
- [ ] Confirm the welcome card displays the gradient indigo-to-purple background and the "Custom Slot" badge in the top-left — visually distinct from the default CopilotChat welcome
23+
- [ ] Confirm NO default CopilotChat welcome heading is rendered (the custom card fully replaces it)
24+
25+
#### Suggestions (`useConfigureSuggestions`)
26+
27+
- [ ] Verify two suggestion pills render in the `suggestionView` slot beneath the input with verbatim titles:
28+
- "Write a sonnet"
29+
- "Tell me a joke"
30+
- [ ] Click "Tell me a joke"; verify it sends the message "Tell me a short joke." and an assistant text response appears within 10s
31+
32+
#### Disclaimer Slot (`input.disclaimer`) — visible after first message
33+
34+
- [ ] After sending the first message, verify the custom disclaimer renders below the chat input (`data-testid="custom-disclaimer"`) containing:
35+
- a small badge reading "slot" (indigo background, lowercase bold)
36+
- body text "Custom disclaimer injected via `input.disclaimer`." (the phrase `input.disclaimer` is in monospace)
37+
- [ ] Verify the default CopilotChat disclaimer text (if any) is NOT present — the custom disclaimer replaces it
38+
39+
#### Assistant Message Slot (`messageView.assistantMessage`)
40+
41+
- [ ] After the assistant response arrives, verify the assistant message is wrapped in the custom container (`data-testid="custom-assistant-message"`) with:
42+
- an indigo-tinted card background (`bg-indigo-50/60` in light mode)
43+
- an indigo border (`border-indigo-200`)
44+
- a small absolute-positioned "slot" badge at the top-left corner (indigo-600 background, white uppercase bold text)
45+
- [ ] Verify the user message bubble is NOT wrapped in the custom container (user messages use the default styling)
46+
- [ ] Send a second prompt ("Write a one-line sonnet"); verify the second assistant response is also wrapped in the `custom-assistant-message` container
47+
48+
### 3. Error Handling
49+
50+
- [ ] Attempt to send an empty message; verify it is a no-op (no user bubble, no network request)
51+
- [ ] Send a ~500-character message; verify it wraps within the max-w-4xl container without horizontal scroll or layout break; the custom assistant-message card grows to fit the response
52+
- [ ] Verify DevTools → Console shows no uncaught errors or missing-prop warnings during any flow above
53+
54+
## Expected Results
55+
56+
- Chat surface renders within 3 seconds with the custom welcome card visible
57+
- Assistant text response within 10 seconds; wrapped in the custom assistant-message slot on every turn
58+
- All three custom slots (`welcomeScreen`, `input.disclaimer`, `messageView.assistantMessage`) replace their defaults and are visually distinguishable via their "slot" badges / gradient styling
59+
- No UI layout breaks, no uncaught console errors

0 commit comments

Comments
 (0)