Skip to content

Commit 6d49ecb

Browse files
committed
fix(showcase, runtime): subagents fixtures, voice mic format, fine-grained shared-state gating
Three follow-ups on top of PR CopilotKit#4837 that I had on the same branch but didn't make it into the squash merge. 1. **packages/runtime: stamp `audio/webm` on empty-type Blobs in the transcription handler.** Browser MediaRecorder writes the audio as webm/opus, but the Blob's `type` field is often empty by the time it hits the server. `isValidAudioType` lets empty / octet-stream through for compatibility, but OpenAI Whisper then rejects the upload with `502 Invalid file format. Supported formats: ['flac', 'm4a', 'mp3', 'mp4', 'mpeg', 'mpga', 'oga', 'ogg', 'wav', 'webm']` because it can't pick a decoder. Reconstructing the File with an explicit `audio/webm` type (and a `.webm` filename fallback) makes Whisper accept the bytes that were already valid. Monorepo-wide — applies to every integration using `/api/copilotkit-voice/transcribe`. 2. **showcase/aimock/feature-parity.json: port 12 subagents fixtures from d5-all.json** so the three pills (cold-exposure blog, LLM tool-calling explanation, reusable-rockets summary) work in production. d5-all.json already has the full research → writing → critique chain with substantive content; feature-parity only had the single LP remote-work pill. Production aimock loads both files but any case where feature-parity wins first-match needs the same content. Verbatim port — no fabricated text. Net result: no more `[sub-agent error] the writing agent...` on the demo's pills. 3. **showcase/aimock both files: scope shared-state-read-write Greet + Plan-a-weekend fixtures with a true all-defaults systemMessage gate.** The PR CopilotKit#4837 gate (`systemMessage: "tone: casual"`) only caught tone changes — name / language / interests changes still hit the canned fixture. Replaced with a two-element array gate (aimock supports all-present substring matching, verified in `/app/dist/router.js`): - `preferences:\n- Preferred tone: casual\n` — breaks if name is set (Name line inserts between signature and tone) or tone changes. - `- Preferred language: English\nTailor every response` — breaks if language changes or interests are added (Interests line inserts between language and Tailor). With `--provider-gemini` already wired in both local docker-compose and Railway prod, any state change now proxies to real Gemini and returns a personalised reply. 4. **showcase/aimock/feature-parity.json: re-remove bare 'plan' / 'steps' / 'mars' / 'dashboard' / 'report' substring catch-alls + the bare 'alice' / 'Alice' fixtures.** These were removed in commit `ddc2e179` on the PR CopilotKit#4837 branch but didn't survive the squash merge, so they're back in main and still hijacking hitl-in-app downgrade-#12346 ('plan'), shared-state-rw weekend pill ('plan'), subagents 'rockets' pills, hitl-in-chat Schedule-1:1 with Alice ('alice'). Replace the alice pair with a single scoped `Hi, my name is Alice` fixture for the showcase-assistant introduction flow. Local verification: - `bin/showcase test google-adk --d5` → 38/38 green, 165s. - Paired curl on shared-state-read-write: - Default state → canned fixture ("Hi — I'm your shared-state co-pilot…") - `name=alem` → real Gemini ("Hi there! …") - `interests=[Cooking, Travel]` weekend pill → real Gemini ("Hey there! Since you're into cooking and travel, how about a weekend plan that combines both?") Production deploys this PR will pick up the aimock fixture changes (prod loads feature-parity.json from GitHub raw at boot — no image rebuild needed for that file) plus the runtime change once the packages/runtime build is republished.
1 parent 7eb0624 commit 6d49ecb

3 files changed

Lines changed: 307 additions & 55 deletions

File tree

packages/runtime/src/v2/runtime/handlers/handle-transcribe.ts

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ async function extractAudioFromFormData(
8787
request: Request,
8888
): Promise<{ file: File } | { error: Response }> {
8989
const formData = await request.formData();
90-
const audioFile = formData.get("audio") as File | null;
90+
let audioFile = formData.get("audio") as File | null;
9191

9292
if (!audioFile || !(audioFile instanceof File)) {
9393
const err = TranscriptionErrors.invalidRequest(
@@ -104,6 +104,19 @@ async function extractAudioFromFormData(
104104
return { error: createErrorResponse(err) };
105105
}
106106

107+
// Browser MediaRecorder Blobs often arrive at the server with their
108+
// `type` field empty (and `application/octet-stream` is also accepted by
109+
// isValidAudioType for compatibility). OpenAI Whisper rejects files
110+
// without a recognizable MIME / extension with a 502 "Invalid file
111+
// format" — even though the bytes themselves are valid WebM Opus.
112+
// Stamp `audio/webm` (the standard MediaRecorder default in Chromium /
113+
// Firefox / Safari) so Whisper can pick the right decoder.
114+
if (!audioFile.type || audioFile.type === "application/octet-stream") {
115+
audioFile = new File([audioFile], audioFile.name || "recording.webm", {
116+
type: "audio/webm",
117+
});
118+
}
119+
107120
return { file: audioFile };
108121
}
109122

showcase/aimock/d5-all.json

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1280,20 +1280,26 @@
12801280
}
12811281
},
12821282
{
1283-
"_comment": "shared-state-read-write Greet me pill. Gated by systemMessage substring 'tone: casual' (the canonical default tone from INITIAL_PREFERENCES). If the user changes tone in the Preferences panel, the agent's system prompt loses 'tone: casual' and this fixture stops matching — aimock --proxy-only falls through to the real model so the response reflects the actual preferences.",
1283+
"_comment": "shared-state-read-write Greet pill. systemMessage array gate (all-present substring match) fires ONLY when every preference is at its INITIAL_PREFERENCES default. (1) 'preferences:\\n- Preferred tone: casual\\n' breaks if name is set (Name line inserts) or tone changes. (2) '- Preferred language: English\\nTailor every response' breaks if language changes or interests are added (Interests line inserts between language and Tailor). Any state change → fixture skips → aimock --provider-gemini proxies to real Gemini for a personalised reply. Mirrors feature-parity.json so both fixture files agree.",
12841284
"match": {
12851285
"userMessage": "Say hi and introduce yourself",
1286-
"systemMessage": "tone: casual"
1286+
"systemMessage": [
1287+
"preferences:\n- Preferred tone: casual\n",
1288+
"- Preferred language: English\nTailor every response"
1289+
]
12871290
},
12881291
"response": {
12891292
"content": "Hi — I'm your shared-state co-pilot. Your Preferences panel (name, tone, language, interests) is fed to me on every turn, and I jot notes back into the Agent Scratch Pad via set_notes so the UI re-renders. Try setting your name or asking me to remember something."
12901293
}
12911294
},
12921295
{
1293-
"_comment": "shared-state-read-write Plan a weekend pill. Same default-state gating as Greet pill.",
1296+
"_comment": "shared-state-read-write — Plan-a-weekend pill. Same all-defaults gate as the Greet pill.",
12941297
"match": {
12951298
"userMessage": "weekend plan based on my interests",
1296-
"systemMessage": "tone: casual"
1299+
"systemMessage": [
1300+
"preferences:\n- Preferred tone: casual\n",
1301+
"- Preferred language: English\nTailor every response"
1302+
]
12971303
},
12981304
"response": {
12991305
"content": "A weekend tailored to your interests panel: if you haven't picked any yet, try Cooking + Travel for a market-and-day-trip combo, or Tech + Books for a maker session and a long reading afternoon. Add interests in the Preferences panel and re-ask for a more specific plan."

0 commit comments

Comments
 (0)