Skip to content

Commit af35569

Browse files
committed
chore(showcase/packages): QA markdown parity + integration tooling
Rename hitl.md → hitl-in-chat.md across all 17 showcase packages, add shared-state-read / shared-state-write / shared-state-streaming / gen-ui-agent / subagents QA docs where missing, update demos/hitl README cross-links. Refresh showcase/shell + shell-dojo registry + demo-content JSON to match new QA shape. Update integration tooling (audit, create-integration, bundle-demo-content, generate-registry, validate-parity, capture-previews, manifest lib) + e2e tests to the new parity contract.
1 parent 96e29c8 commit af35569

166 files changed

Lines changed: 7805 additions & 4679 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 10 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99

1010
### 1. Basic Functionality
1111

12-
- [ ] Navigate to the hitl demo page
12+
- [ ] Navigate to the HITL demo page (`/demos/hitl`) > _Note: the URL path `/demos/hitl` is intentionally shorter than the feature id `hitl-in-chat`._
1313
- [ ] Verify the chat interface loads in a centered max-w-4xl container
1414
- [ ] Verify the chat input placeholder "Type a message" is visible
1515
- [ ] Send a basic message
@@ -24,7 +24,12 @@
2424
- [ ] Click the "Simple plan" suggestion
2525
- [ ] Verify it triggers a message about planning a trip to Mars in 5 steps
2626

27-
#### LangGraph Interrupt (useLangGraphInterrupt)
27+
#### Step Selection and Approval
28+
29+
The HITL demo renders a single StepSelector card regardless of whether
30+
the underlying flow uses an interrupt hook or a frontend HITL tool. The
31+
framework-specific hook name is an implementation detail covered in each
32+
package's demo source; QA below validates the user-visible card behavior.
2833

2934
- [ ] Send "Plan a trip to Mars in 5 steps"
3035
- [ ] Verify the StepSelector card appears (`data-testid="select-steps"`)
@@ -33,18 +38,10 @@
3338
- [ ] Verify the selected count display shows "N/N selected"
3439
- [ ] Toggle a step checkbox off and verify the count decreases
3540
- [ ] Toggle it back on and verify the count increases
36-
- [ ] Click "Perform Steps (N)" button
41+
- [ ] Click "Perform Steps (N)" / "Confirm (N)" button
3742
- [ ] Verify the agent continues processing after confirmation
38-
39-
#### Human-in-the-Loop Feedback (useHumanInTheLoop)
40-
41-
- [ ] Trigger a task that generates steps via the generate_task_steps tool
42-
- [ ] Verify the "Review Steps" card appears (`data-testid="select-steps"`)
43-
- [ ] Verify Accept and Reject buttons are visible
44-
- [ ] Click "Confirm (N)" and verify the card shows "Accepted" status
45-
- [ ] In a new conversation, trigger the same flow and click "Reject"
46-
- [ ] Verify the card shows "Rejected" status
47-
- [ ] Verify buttons are disabled after a decision is made
43+
- [ ] In a new conversation, trigger the same flow and click "Reject" (where present)
44+
- [ ] Verify the card reflects the decision (Accepted / Rejected) and buttons disable
4845

4946
### 3. Error Handling
5047

showcase/packages/ag2/qa/shared-state-streaming.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,8 @@
2323

2424
#### Note: Stub Demo
2525

26-
- [ ] This demo is currently a stub (TODO: implement)
26+
> **Status: Stub** — This demo is currently a stub (TODO: implement)
27+
2728
- [ ] Verify the basic CopilotChat loads and accepts messages
2829
- [ ] Verify the agent responds to messages
2930
- [ ] No custom UI components are expected beyond the chat interface

showcase/packages/ag2/qa/shared-state-write.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,8 @@
2323

2424
#### Note: Stub Demo
2525

26-
- [ ] This demo is currently a stub (TODO: implement)
26+
> **Status: Stub** — This demo is currently a stub (TODO: implement)
27+
2728
- [ ] Verify the basic CopilotChat loads and accepts messages
2829
- [ ] Verify the agent responds to messages
2930
- [ ] No custom UI components are expected beyond the chat interface

showcase/packages/ag2/qa/subagents.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,8 @@
2323

2424
#### Note: Stub Demo
2525

26-
- [ ] This demo is currently a stub (TODO: implement)
26+
> **Status: Stub** — This demo is currently a stub (TODO: implement)
27+
2728
- [ ] Verify the basic CopilotChat loads and accepts messages
2829
- [ ] Verify the agent responds to messages
2930
- [ ] No custom UI components are expected beyond the chat interface

showcase/packages/ag2/src/app/demos/hitl/README.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,27 +2,27 @@
22

33
## What This Demo Shows
44

5-
User approves agent actions before execution
5+
Interactive components from the agent; user approves actions from within the chat
66

77
## How to Interact
88

99
Try asking your Copilot to:
1010

11-
- "Change the background color to a warm sunset gradient"
12-
- "Set the theme to dark mode"
13-
- "Make the background a calming blue-green gradient"
11+
- "Plan a trip to Mars in 5 steps"
12+
- "Please plan a pasta dish in 10 steps"
13+
- "Draft a product launch checklist in 7 steps"
1414

15-
When the agent proposes an action, you'll see an approval prompt. Click **Approve** to let it proceed or **Reject** to cancel.
15+
The agent proposes a plan as a **StepSelector** card with a checkbox per step. Toggle individual steps on or off (the selected count updates as "N/N selected"), then click **Perform Steps (N)** / **Confirm (N)** to approve, or **Reject** to cancel. The card reflects the final decision ("Accepted" / "Rejected") and disables its buttons afterward.
1616

1717
## Technical Details
1818

1919
What's happening technically:
2020

21-
- **Human-in-the-Loop (HITL)** lets the agent propose actions that require user approval before execution
22-
- The agent calls a tool (like `change_background`), and CopilotKit intercepts it to show a confirmation dialog
23-
- `useHumanInTheLoop` registers a frontend tool with `requireConfirmation: true`, adding the approval step
24-
- The user sees what the agent wants to do (with the proposed arguments) and can approve or reject
25-
- This pattern is essential for high-stakes actionsdatabase writes, API calls, or any irreversible operation
21+
- **Human-in-the-Loop (HITL)** lets the agent pause and hand control back to the user to review a proposed plan before continuing
22+
- The agent calls a tool (e.g. `generate_task_steps`) whose payload is a list of steps; the frontend renders a **StepSelector** card rather than executing immediately
23+
- `useHumanInTheLoop` (frontend-tool flow) or `useLangGraphInterrupt` (interrupt flow) registers a `render` that receives `{ args, respond, status }` — the StepSelector keeps local state for which steps are enabled and calls `respond({ accepted, steps })` on Confirm / Reject (CopilotKit v2 API; existing showcase packages use v1 `useLangGraphInterrupt` for the interrupt-flow demo alongside v2 hooks like `useHumanInTheLoop` — new integrations generated by `create-integration` default to the v2 `useInterrupt`.)
24+
- The card exposes per-step checkboxes, a live "N/N selected" count, and Confirm / Reject buttons; after a decision the buttons disable and the card shows "Accepted" or "Rejected" so the outcome is auditable in the transcript
25+
- This pattern is essential for plan-style flowsmulti-step actions where the user needs to curate, edit, or veto what the agent is about to do before any of it runs
2626

2727
## Building With This
2828

showcase/packages/claude-sdk-python/tests/e2e/hitl.spec.ts renamed to showcase/packages/ag2/tests/e2e/hitl-in-chat.spec.ts

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,9 @@ test.describe("Human in the Loop", () => {
1414
await input.fill("Hello");
1515
await input.press("Enter");
1616

17-
await expect(page.locator('[data-role="assistant"]').first()).toBeVisible({
17+
await expect(
18+
page.locator(".copilotKitAssistantMessage").first(),
19+
).toBeVisible({
1820
timeout: 30000,
1921
});
2022
});
@@ -26,7 +28,8 @@ test.describe("Human in the Loop", () => {
2628
);
2729
await input.press("Enter");
2830

29-
// Either the LangGraph interrupt StepSelector or the HITL StepsFeedback should appear
31+
// The HITL demo surfaces a single StepSelector card regardless of whether the
32+
// underlying flow is an interrupt hook or a frontend HITL tool — assert on that.
3033
const stepSelector = page.locator('[data-testid="select-steps"]');
3134
await expect(stepSelector.first()).toBeVisible({ timeout: 60000 });
3235

@@ -77,8 +80,10 @@ test.describe("Human in the Loop", () => {
7780
await expect(actionBtn.first()).toBeVisible({ timeout: 5000 });
7881
await actionBtn.first().click();
7982

80-
// After approval, agent should continue — look for accepted/confirmed state
81-
const confirmed = page.locator("text=/Accepted|Confirmed|selected/i");
82-
await expect(confirmed.first()).toBeVisible({ timeout: 10000 });
83+
// After approval, the step selector's action button should disappear
84+
// (StepSelector unmounts after onConfirm; StepsFeedback replaces buttons with
85+
// the "Accepted" or "Rejected" banner). Assert on a post-click transition
86+
// that does NOT match the pre-click "N/N selected" counter text.
87+
await expect(actionBtn.first()).not.toBeVisible({ timeout: 10000 });
8388
});
8489
});

showcase/packages/ag2/tsconfig.tsbuildinfo

Lines changed: 0 additions & 1 deletion
This file was deleted.
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# QA: Agentic Generative UI — Agno
2+
3+
## Prerequisites
4+
5+
- Demo is deployed and accessible
6+
- Agent backend is healthy (check /api/health)
7+
8+
## Test Steps
9+
10+
### 1. Basic Functionality
11+
12+
- [ ] Navigate to the gen-ui-agent demo page
13+
- [ ] Verify the chat interface loads in a centered full-height layout
14+
- [ ] Verify the chat input placeholder "Type a message" is visible
15+
- [ ] Send a basic message
16+
- [ ] Verify the agent responds
17+
18+
### 2. Feature-Specific Checks
19+
20+
#### Suggestions
21+
22+
- [ ] Verify "Simple plan" suggestion button is visible (plan to go to Mars in 5 steps)
23+
- [ ] Verify "Complex plan" suggestion button is visible (plan to make pizza in 10 steps)
24+
25+
#### Task Progress Tracker (useAgent with state streaming)
26+
27+
- [ ] Click "Simple plan" suggestion or type "Build a plan to go to Mars in 5 steps"
28+
- [ ] Verify the TaskProgress component renders (`data-testid="task-progress"`)
29+
- [ ] Verify the progress bar appears with a gradient fill
30+
- [ ] Verify step items appear with descriptions (`data-testid="task-step-text"`)
31+
- [ ] Verify the "N/N Complete" counter updates as steps complete
32+
- [ ] Verify completed steps show:
33+
- Green background gradient
34+
- Check icon
35+
- Green text color
36+
- [ ] Verify the current pending step shows:
37+
- Blue/purple background gradient
38+
- Spinner icon with "Processing..." text
39+
- Pulsing animation
40+
- [ ] Verify future pending steps show:
41+
- Gray background
42+
- Clock icon
43+
- Muted text color
44+
45+
#### Complex Plan
46+
47+
- [ ] Type "Plan to make pizza in 10 steps"
48+
- [ ] Verify 10 steps appear in the progress tracker
49+
- [ ] Verify progress bar width increases as steps complete
50+
51+
### 3. Error Handling
52+
53+
- [ ] Send an empty message (should be handled gracefully)
54+
- [ ] Verify no console errors during normal usage
55+
56+
## Expected Results
57+
58+
- Chat loads within 3 seconds
59+
- Agent responds within 10 seconds
60+
- Task progress tracker shows live step completion
61+
- Progress bar animates smoothly
62+
- No UI errors or broken layouts
Lines changed: 10 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99

1010
### 1. Basic Functionality
1111

12-
- [ ] Navigate to the hitl demo page
12+
- [ ] Navigate to the HITL demo page (`/demos/hitl`) > _Note: the URL path `/demos/hitl` is intentionally shorter than the feature id `hitl-in-chat`._
1313
- [ ] Verify the chat interface loads in a centered max-w-4xl container
1414
- [ ] Verify the chat input placeholder "Type a message" is visible
1515
- [ ] Send a basic message
@@ -24,7 +24,12 @@
2424
- [ ] Click the "Simple plan" suggestion
2525
- [ ] Verify it triggers a message about planning a trip to Mars in 5 steps
2626

27-
#### LangGraph Interrupt (useLangGraphInterrupt)
27+
#### Step Selection and Approval
28+
29+
The HITL demo renders a single StepSelector card regardless of whether
30+
the underlying flow uses an interrupt hook or a frontend HITL tool. The
31+
framework-specific hook name is an implementation detail covered in each
32+
package's demo source; QA below validates the user-visible card behavior.
2833

2934
- [ ] Send "Plan a trip to Mars in 5 steps"
3035
- [ ] Verify the StepSelector card appears (`data-testid="select-steps"`)
@@ -33,18 +38,10 @@
3338
- [ ] Verify the selected count display shows "N/N selected"
3439
- [ ] Toggle a step checkbox off and verify the count decreases
3540
- [ ] Toggle it back on and verify the count increases
36-
- [ ] Click "Perform Steps (N)" button
41+
- [ ] Click "Perform Steps (N)" / "Confirm (N)" button
3742
- [ ] Verify the agent continues processing after confirmation
38-
39-
#### Human-in-the-Loop Feedback (useHumanInTheLoop)
40-
41-
- [ ] Trigger a task that generates steps via the generate_task_steps tool
42-
- [ ] Verify the "Review Steps" card appears (`data-testid="select-steps"`)
43-
- [ ] Verify Accept and Reject buttons are visible
44-
- [ ] Click "Confirm (N)" and verify the card shows "Accepted" status
45-
- [ ] In a new conversation, trigger the same flow and click "Reject"
46-
- [ ] Verify the card shows "Rejected" status
47-
- [ ] Verify buttons are disabled after a decision is made
43+
- [ ] In a new conversation, trigger the same flow and click "Reject" (where present)
44+
- [ ] Verify the card reflects the decision (Accepted / Rejected) and buttons disable
4845

4946
### 3. Error Handling
5047

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# QA: Shared State (Reading) — Agno
2+
3+
## Prerequisites
4+
5+
- Demo is deployed and accessible
6+
- Agent backend is healthy (check /api/health)
7+
8+
## Test Steps
9+
10+
### 1. Basic Functionality
11+
12+
- [ ] Navigate to the shared-state-read demo page
13+
- [ ] Verify the recipe card form loads (`data-testid="recipe-card"`)
14+
- [ ] Verify the CopilotSidebar opens by default with title "AI Recipe Assistant"
15+
- [ ] Send a message via the sidebar
16+
- [ ] Verify the agent responds
17+
18+
### 2. Feature-Specific Checks
19+
20+
#### Initial Recipe State
21+
22+
- [ ] Verify the recipe title input shows "Make Your Recipe"
23+
- [ ] Verify the cooking time dropdown defaults to "45 min"
24+
- [ ] Verify the skill level dropdown defaults to "Intermediate"
25+
- [ ] Verify the default ingredients are displayed:
26+
- [ ] Carrots (3 large, grated) with carrot emoji
27+
- [ ] All-Purpose Flour (2 cups) with wheat emoji
28+
- [ ] Verify the default instruction is displayed: "Preheat oven to 350 F"
29+
30+
#### Suggestions
31+
32+
- [ ] Verify "Create Italian recipe" suggestion is visible
33+
- [ ] Verify "Make it healthier" suggestion is visible
34+
- [ ] Verify "Suggest variations" suggestion is visible
35+
36+
#### Recipe Editing (Local State)
37+
38+
- [ ] Edit the recipe title and verify it updates
39+
- [ ] Change the skill level dropdown and verify it updates
40+
- [ ] Change the cooking time dropdown and verify it updates
41+
- [ ] Toggle a dietary preference checkbox (e.g. "Vegetarian") and verify it's checked
42+
- [ ] Click "+ Add Ingredient" (`data-testid="add-ingredient-button"`) and verify a new empty row appears
43+
- [ ] Edit an ingredient name and amount
44+
- [ ] Remove an ingredient by clicking the "x" button
45+
- [ ] Click "+ Add Step" and verify a new instruction row appears
46+
- [ ] Edit an instruction and verify it saves
47+
- [ ] Remove an instruction by clicking the "x" button
48+
49+
#### AI-Powered Recipe Updates (useAgent with shared state)
50+
51+
- [ ] Click "Create Italian recipe" suggestion
52+
- [ ] Verify the agent updates the recipe title, ingredients, and instructions
53+
- [ ] Verify the ping indicator appears on changed sections
54+
- [ ] Verify the "Improve with AI" button (`data-testid="improve-button"`) changes to "Please Wait..." while loading
55+
- [ ] Click "Improve with AI" and verify the recipe is enhanced
56+
57+
#### Agent Reads Frontend State
58+
59+
- [ ] Edit the recipe (change title, add ingredients)
60+
- [ ] Ask the agent "What recipe am I making?"
61+
- [ ] Verify the agent's response references the current recipe state
62+
63+
### 3. Error Handling
64+
65+
- [ ] Send an empty message (should be handled gracefully)
66+
- [ ] Verify no console errors during normal usage
67+
- [ ] Verify the "Improve with AI" button is disabled while loading
68+
69+
## Expected Results
70+
71+
- Recipe card and sidebar load within 3 seconds
72+
- Agent responds within 10 seconds
73+
- Recipe state syncs bidirectionally between UI and agent
74+
- Ping indicators highlight changed sections
75+
- No UI errors or broken layouts

0 commit comments

Comments
 (0)