Skip to content

Commit 0fbf0e9

Browse files
committed
fix(showcase): path-based content-level starter-smoke probe
Rewrite the starter-smoke probe to the v2 path-based multi-route runtime protocol the deployed starters actually speak (proven via a local build+curl gate against examples/integrations/crewai-crews at 1.59.5): - agent rung: GET /api/copilotkit/info -> 200 + version (was: any non-404) - chat rung: POST /api/copilotkit/agent/<id>/run (Accept: text/event-stream) asserting >=1 TEXT_MESSAGE_CONTENT delta + terminal RUN_FINISHED + no RUN_ERROR (was: non-empty body) - health rung: repointed to GET /api/copilotkit/info (the starters serve no /api/health route) - interaction rung: GET / (unchanged) - drop trailing slash on all runtime POSTs (the old /api/copilotkit/ 308s and drops the POST -> 404, the core cause of the red rung) Plus a unified abort/body-read error classification (one abortOutcome() helper keyed on the local externallyAborted flag): a self-timeout or a non-abort error is transport-error (soft), only a genuine external abort is 'aborted'; a cut-short body read softens to transport-error instead of hard smoke-failed; the level loop short-circuits on external abort. NOTE: the content-level chat rung will correctly read RED against the deployed starters until the Python-side ag-ui-crewai RUN_ERROR (a real-LLM, in-process agent failure, NOT a probe bug) is fixed. Deferred follow-ups: verify the alert/staleness consumer branches on errorClass (not state) so the soft/hard split is actually consumed; res.text() resolve-partial precision edge; redirect:follow POST edge; health/interaction body-drain; health/agent shared /info; CLASS_RANK aborted-lowest; unmapped-starter columnSlug/prototype-key; test-coverage gaps (resolveTimeoutMs, deriveStarterSlug, real-timer abort).
1 parent a266ed3 commit 0fbf0e9

3 files changed

Lines changed: 1094 additions & 151 deletions

File tree

showcase/harness/config/probes/starter_smoke.yml

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,21 @@
1313
#
1414
# Each driver invocation issues FOUR HTTP checks against the deployed base
1515
# URL and side-emits one row per smoke level plus an aggregate primary
16-
# (see drivers/starter-smoke.ts):
17-
# - health GET `${base}/api/health` → 200 + JSON body
18-
# - agent POST `${base}/api/copilotkit/` → any non-404 response
19-
# - chat POST `${base}/api/copilotkit/` → non-empty body (aimock)
20-
# - interaction GET `${base}/` → 200 (app shell serves)
16+
# (see drivers/starter-smoke.ts). The deployed starters mount the v2 runtime
17+
# in its default PATH-BASED `mode:"multi-route"` at `/api/copilotkit`, so the
18+
# checks dispatch by URL path (the dead single-route envelope POST to bare
19+
# `/api/copilotkit` is never issued):
20+
# - health GET `${base}/api/copilotkit/info` → 200 (lightweight
21+
# runtime-mount liveness;
22+
# the starter has no
23+
# `/api/health` route)
24+
# - agent GET `${base}/api/copilotkit/info` → 200 JSON carrying a
25+
# `version` field
26+
# - chat POST `${base}/api/copilotkit/agent/default/run`
27+
# (Accept: text/event-stream) → AG-UI SSE stream asserting ≥1
28+
# TEXT_MESSAGE_CONTENT delta + terminal RUN_FINISHED + NO
29+
# RUN_ERROR
30+
# - interaction GET `${base}/` → 200 (app shell serves)
2131
#
2232
# Rows emitted per invocation:
2333
# - Primary `starter:<column-slug>` aggregate { total, passed,

0 commit comments

Comments
 (0)