Skip to content

Commit 6bddaa6

Browse files
authored
feat(showcase): add Status tab + probe introspection/trigger API (CopilotKit#4293)
## Summary Implements the [Showcase-Ops Introspection & Control — Status Tab spec (2026-04-25)](https://www.notion.so/copilotkit/Showcase-Ops-Introspection-Control-Status-Tab-Spec-2026-04-25-34d3aa381852811a8715e781cf4c5930). Three operator capabilities land together: 1. **Introspect scheduled probes** — `GET /api/probes` exposes every registered probe with cron, next-run, last-run summary, and inflight progress. `GET /api/probes/:id` adds the last 10 run records. 2. **See what's running** — a new `ProbeRunTracker` is registered on each scheduler entry while a probe is inflight, recording per-service queued/running/completed/failed transitions. Surfaced via the API and rendered live in the dashboard. 3. **Trigger on demand** — `POST /api/probes/:id/trigger` fires a probe outside its cron schedule. Bearer-auth gated (`OPS_TRIGGER_TOKEN`), rate-limited to 1 trigger / probe / 5 min, returns 409 on inflight conflict. A new **Status** tab in `shell-dashboard` consumes the API: live probe schedule table, currently-running panel with progress bar, trigger dropdown (all / specific slugs), and a per-probe detail panel with last-10-runs list + duration sparkline. Polls every 10s. A `probe_runs` PocketBase collection persists run history (probe_id, started_at, finished_at, duration_ms, triggered, summary, state) for trend analysis and the detail view. ## Architecture ``` ┌──────────────────────┐ │ shell-dashboard │ │ Status tab │ │ (10s poll) │ └──────────┬───────────┘ │ /api/probes/* ▼ ┌────────────┐ setEntryTracker ┌──────────────────────┐ │ Scheduler │◄─────────────────────│ probe-invoker │ │ (croner) │ │ (per-service) │ │ EntrySlot │ trigger/getEntry │ ├ tracker.start() │ │ ├ tracker│◄─────────────────────│ ├ tracker.complete │ │ ├ lastRun│ │ └ tracker.fail │ │ └ next() │ │ runWriter.start/ │ └────────────┘ │ runWriter.finish │ └──────────┬───────────┘ │ ▼ ┌──────────────────────┐ │ PocketBase │ │ probe_runs │ └──────────────────────┘ ``` ## Slot manifest (blitz dispatch — 9 slots + 1 integration follow-up) | Slot | Branch | Concern | |------|--------|---------| | B1 | `feat/ops-scheduler-introspection` | Scheduler EntrySlot extensions, getEntry/trigger/nextRunAt | | B2 | `feat/ops-probe-run-tracker` | ProbeRunTracker pure data class | | B3 | `feat/ops-probes-api` | HTTP /api/probes routes + bearer auth + rate limit | | B4a | `feat/dashboard-status-tab-ui` | Status tab UI components + page registration | | B4b | `feat/dashboard-ops-api-client` | ops-api types + fetch + 10s poll hooks | | B5 | `feat/ops-bearer-auth` | bearerAuth Hono middleware | | B6 | `feat/ops-probe-runs-collection` | PocketBase migration + run-history writer | | B7 | `feat/ops-invoker-integration` | Wire tracker + probe_runs writes into invoker | | B8 | `feat/dashboard-run-history-panel` | Detail panel + runs list + duration sparkline | | F1 | (folded into PR branch) | Orchestrator threads scheduler+runWriter into invoker, mounts router with getProbeConfig | ## Tests - showcase-ops: 896 pass / 2 pre-existing failures unrelated to this change (`aimock-fixture-coverage` missing langgraph-python prompts; `probe-loader` chokidar watcher flake) - shell-dashboard: 204 pass / 1 skipped (pre-existing) - New tests added: 19 (B1 scheduler) + 14 (B2 tracker) + 19 (B3 routes) + 19 (B4a UI) + 33 (B4b client/hooks) + 18 (B5 auth) + 9 (B6 run-history) + 7 (B7 invoker) + 21 (B8 detail panel) + 1 (F1 orchestrator wiring) = 160 new - TypeScript clean (`tsc --noEmit`) on both packages ## Known follow-ups (out of scope) 1. `lastRun.state` schedule-level field is hard-coded `"completed"` in `GET /api/probes` — actual per-run state lives on the `probe_runs` rows surfaced by `GET /api/probes/:id`. Threading state up to the scheduler entry-status surface is a small follow-up. 2. Production deploys MUST set `OPS_TRIGGER_TOKEN` or the probes router skips mounting (logged at info). Could be flipped to fail-loud. 3. `trigger(opts.filter)` is accepted by the scheduler + router but not yet applied by the invoker — invoker currently runs the full discovery set. Filter wiring is a separate small change in probe-invoker. ## Test plan - [ ] CI green (showcase-ops + shell-dashboard test suites + lint + typecheck) - [ ] Local: `pnpm --filter @copilotkit/showcase-ops test` green - [ ] Local: cd `showcase/shell-dashboard && npm test` green - [ ] Local: `pnpm --filter @copilotkit/showcase-ops typecheck` green - [ ] Manual: hit `GET /api/probes` on a running ops instance, see the schedule - [ ] Manual: hit `POST /api/probes/smoke/trigger` with bearer, observe inflight progress - [ ] Manual: open the dashboard Status tab, click a probe row, see the detail panel
2 parents ac28335 + b3ec2a1 commit 6bddaa6

40 files changed

Lines changed: 10844 additions & 139 deletions

showcase/aimock/feature-parity.json

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,32 @@
8686
]
8787
}
8888
},
89+
{
90+
"match": {
91+
"userMessage": "revenue by category as a pie chart"
92+
},
93+
"response": {
94+
"toolCalls": [
95+
{
96+
"name": "render_pie_chart",
97+
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
98+
}
99+
]
100+
}
101+
},
102+
{
103+
"match": {
104+
"userMessage": "monthly expenses as a bar chart"
105+
},
106+
"response": {
107+
"toolCalls": [
108+
{
109+
"name": "render_bar_chart",
110+
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
111+
}
112+
]
113+
}
114+
},
89115
{
90116
"match": {
91117
"userMessage": "30-minute meeting to learn about CopilotKit"

showcase/ops/src/http/auth.test.ts

Lines changed: 207 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
import { Hono } from "hono";
2+
import { describe, it, expect, afterEach } from "vitest";
3+
import {
4+
bearerAuth,
5+
MissingAuthTokenError,
6+
constantTimeEqual,
7+
} from "./auth.js";
8+
9+
const ENV_VAR = "OPS_TRIGGER_TOKEN_TEST";
10+
11+
function makeApp(token: string): Hono {
12+
const app = new Hono();
13+
app.use("/protected/*", bearerAuth({ expectedToken: token }));
14+
app.post("/protected/trigger", (c) => c.json({ ok: true }));
15+
return app;
16+
}
17+
18+
describe("constantTimeEqual", () => {
19+
it("returns true for identical strings", () => {
20+
expect(constantTimeEqual("abc123", "abc123")).toBe(true);
21+
});
22+
23+
it("returns false for different strings of equal length", () => {
24+
expect(constantTimeEqual("abcdef", "abcdeg")).toBe(false);
25+
});
26+
27+
it("returns false for strings of differing length without throwing", () => {
28+
// Length mismatch must NOT throw — that would leak length via timing.
29+
// Returning false is the correct constant-time-safe behavior.
30+
expect(constantTimeEqual("a", "abc")).toBe(false);
31+
expect(constantTimeEqual("abc", "a")).toBe(false);
32+
});
33+
34+
it("returns false when either input is empty", () => {
35+
expect(constantTimeEqual("", "x")).toBe(false);
36+
expect(constantTimeEqual("x", "")).toBe(false);
37+
// Two empty strings: defensively false — empty is never a valid token.
38+
expect(constantTimeEqual("", "")).toBe(false);
39+
});
40+
});
41+
42+
describe("bearerAuth construction", () => {
43+
afterEach(() => {
44+
delete process.env[ENV_VAR];
45+
delete process.env.OPS_TRIGGER_TOKEN;
46+
});
47+
48+
it("throws MissingAuthTokenError when env var is unset and no expectedToken given", () => {
49+
delete process.env[ENV_VAR];
50+
expect(() => bearerAuth({ envVar: ENV_VAR })).toThrow(
51+
MissingAuthTokenError,
52+
);
53+
});
54+
55+
it("throws MissingAuthTokenError when env var is empty string and no expectedToken given", () => {
56+
process.env[ENV_VAR] = "";
57+
expect(() => bearerAuth({ envVar: ENV_VAR })).toThrow(
58+
MissingAuthTokenError,
59+
);
60+
});
61+
62+
it("does not throw when explicit expectedToken provided (env var unset)", () => {
63+
delete process.env[ENV_VAR];
64+
expect(() =>
65+
bearerAuth({ expectedToken: "explicit-tok", envVar: ENV_VAR }),
66+
).not.toThrow();
67+
});
68+
69+
it("explicit expectedToken takes priority over env lookup", () => {
70+
process.env[ENV_VAR] = "env-token";
71+
const mw = bearerAuth({ expectedToken: "explicit-tok", envVar: ENV_VAR });
72+
expect(mw).toBeTypeOf("function");
73+
});
74+
75+
it("defaults to OPS_TRIGGER_TOKEN env var when envVar option not provided", () => {
76+
delete process.env.OPS_TRIGGER_TOKEN;
77+
expect(() => bearerAuth()).toThrow(MissingAuthTokenError);
78+
process.env.OPS_TRIGGER_TOKEN = "from-default-env";
79+
expect(() => bearerAuth()).not.toThrow();
80+
});
81+
});
82+
83+
describe("bearerAuth middleware", () => {
84+
const TOKEN = "s3cr3t-trigger-token";
85+
86+
it("returns 401 with JSON {error: 'unauthorized'} when Authorization header is missing", async () => {
87+
const app = makeApp(TOKEN);
88+
const res = await app.request("/protected/trigger", { method: "POST" });
89+
expect(res.status).toBe(401);
90+
const body = await res.json();
91+
expect(body).toEqual({ error: "unauthorized" });
92+
});
93+
94+
it("returns 401 when Authorization header lacks the Bearer prefix", async () => {
95+
const app = makeApp(TOKEN);
96+
const res = await app.request("/protected/trigger", {
97+
method: "POST",
98+
headers: { Authorization: TOKEN }, // no "Bearer "
99+
});
100+
expect(res.status).toBe(401);
101+
const body = await res.json();
102+
expect(body).toEqual({ error: "unauthorized" });
103+
});
104+
105+
it("returns 401 when Authorization header uses a non-Bearer scheme (Basic, Token, etc.)", async () => {
106+
const app = makeApp(TOKEN);
107+
const res = await app.request("/protected/trigger", {
108+
method: "POST",
109+
headers: { Authorization: `Token ${TOKEN}` },
110+
});
111+
expect(res.status).toBe(401);
112+
});
113+
114+
it("returns 401 when Bearer prefix is present but token is wrong", async () => {
115+
const app = makeApp(TOKEN);
116+
const res = await app.request("/protected/trigger", {
117+
method: "POST",
118+
headers: { Authorization: "Bearer wrong-token" },
119+
});
120+
expect(res.status).toBe(401);
121+
const body = await res.json();
122+
expect(body).toEqual({ error: "unauthorized" });
123+
});
124+
125+
it("returns 401 when Bearer is present but token portion is empty", async () => {
126+
const app = makeApp(TOKEN);
127+
const res = await app.request("/protected/trigger", {
128+
method: "POST",
129+
headers: { Authorization: "Bearer " },
130+
});
131+
expect(res.status).toBe(401);
132+
});
133+
134+
it("calls next() when token matches — handler runs and returns 200", async () => {
135+
const app = makeApp(TOKEN);
136+
const res = await app.request("/protected/trigger", {
137+
method: "POST",
138+
headers: { Authorization: `Bearer ${TOKEN}` },
139+
});
140+
expect(res.status).toBe(200);
141+
const body = await res.json();
142+
expect(body).toEqual({ ok: true });
143+
});
144+
145+
it("is case-sensitive on the token (constant-time-safe compare)", async () => {
146+
const app = makeApp(TOKEN);
147+
const res = await app.request("/protected/trigger", {
148+
method: "POST",
149+
headers: { Authorization: `Bearer ${TOKEN.toUpperCase()}` },
150+
});
151+
expect(res.status).toBe(401);
152+
});
153+
154+
it("rejects a token of different length without throwing", async () => {
155+
// Constant-time compare must handle length mismatch gracefully —
156+
// a throw here would leak length via timing AND surface as a 500.
157+
const app = makeApp(TOKEN);
158+
const res = await app.request("/protected/trigger", {
159+
method: "POST",
160+
headers: { Authorization: "Bearer x" },
161+
});
162+
expect(res.status).toBe(401);
163+
});
164+
165+
it("accepts the Bearer scheme case-insensitively (RFC 6750)", async () => {
166+
// RFC 6750 §2.1: "Bearer" auth-scheme is case-insensitive. Be lenient
167+
// on the scheme match; strict only on the token.
168+
const app = makeApp(TOKEN);
169+
const res = await app.request("/protected/trigger", {
170+
method: "POST",
171+
headers: { Authorization: `bearer ${TOKEN}` },
172+
});
173+
expect(res.status).toBe(200);
174+
});
175+
176+
// R3-A.4: presented-token whitespace was trimmed but expected token was
177+
// compared verbatim. An OPS_TRIGGER_TOKEN env var that was copy-pasted
178+
// with surrounding whitespace would silently 401 every well-formed
179+
// request. Trim BOTH sides at construction so the comparison is symmetric.
180+
it("R3-A.4: accepts a Bearer token when expectedToken has leading/trailing whitespace", async () => {
181+
// expectedToken padded with spaces — common env-var copy-paste mistake.
182+
const padded = " abc ";
183+
const app = new Hono();
184+
app.use("/protected/*", bearerAuth({ expectedToken: padded }));
185+
app.post("/protected/trigger", (c) => c.json({ ok: true }));
186+
const res = await app.request("/protected/trigger", {
187+
method: "POST",
188+
headers: { Authorization: "Bearer abc" },
189+
});
190+
expect(res.status).toBe(200);
191+
});
192+
193+
it("R3-A.4: accepts a Bearer token presented with extra whitespace when expected token is clean", async () => {
194+
// Clean expectedToken; presented header has extra whitespace around the
195+
// token portion. The regex captures anything after `Bearer\s+` up to the
196+
// header end so trailing spaces show up in the captured group; .trim()
197+
// normalises them.
198+
const app = new Hono();
199+
app.use("/protected/*", bearerAuth({ expectedToken: "abc" }));
200+
app.post("/protected/trigger", (c) => c.json({ ok: true }));
201+
const res = await app.request("/protected/trigger", {
202+
method: "POST",
203+
headers: { Authorization: "Bearer abc " },
204+
});
205+
expect(res.status).toBe(200);
206+
});
207+
});

showcase/ops/src/http/auth.ts

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
import crypto from "node:crypto";
2+
import type { MiddlewareHandler } from "hono";
3+
4+
/**
5+
* Thrown at construction time when no expected token is available.
6+
*
7+
* This is intentionally fail-loud: a misconfigured deployment that silently
8+
* defaulted to "always allow" is the failure mode this gate exists to
9+
* prevent. Operators must see the error during boot, not after the trigger
10+
* endpoint starts accepting unauthenticated requests in production.
11+
*/
12+
export class MissingAuthTokenError extends Error {
13+
constructor(envVar: string) {
14+
super(
15+
`bearerAuth: no expected token configured. Set ${envVar} or pass an explicit expectedToken option.`,
16+
);
17+
this.name = "MissingAuthTokenError";
18+
}
19+
}
20+
21+
/**
22+
* Constant-time string comparison.
23+
*
24+
* Wraps `crypto.timingSafeEqual` with two operator-friendly behaviors that
25+
* the raw primitive lacks:
26+
*
27+
* 1. Length-mismatch returns `false` instead of throwing. The naked
28+
* `timingSafeEqual` requires equal-length buffers; throwing on
29+
* mismatch leaks length via timing AND surfaces as a 500 to the
30+
* caller. We swallow the mismatch and report a plain auth failure.
31+
* 2. Empty strings always return `false`. An empty token is never a
32+
* valid credential — we refuse to even attempt the compare so a
33+
* construction bug that produces "" can't masquerade as a valid
34+
* match.
35+
*
36+
* Exported for direct unit-test coverage; the regex on
37+
* `crypto.timingSafeEqual` is non-trivial to assert on indirectly.
38+
*/
39+
export function constantTimeEqual(a: string, b: string): boolean {
40+
if (a.length === 0 || b.length === 0) return false;
41+
const bufA = Buffer.from(a, "utf8");
42+
const bufB = Buffer.from(b, "utf8");
43+
if (bufA.length !== bufB.length) return false;
44+
return crypto.timingSafeEqual(bufA, bufB);
45+
}
46+
47+
export interface BearerAuthOptions {
48+
/**
49+
* Explicit expected token. When provided, takes priority over the env
50+
* lookup. Useful for tests and for callers that load the token from a
51+
* non-env source (e.g. a secrets manager).
52+
*/
53+
expectedToken?: string;
54+
/**
55+
* Name of the env var to read at construction time when `expectedToken`
56+
* is not provided. Defaults to `OPS_TRIGGER_TOKEN`.
57+
*/
58+
envVar?: string;
59+
}
60+
61+
const DEFAULT_ENV_VAR = "OPS_TRIGGER_TOKEN";
62+
63+
/**
64+
* Construct a Hono bearer-auth middleware.
65+
*
66+
* Rejects with 401 + `{error: "unauthorized"}` JSON when:
67+
* - the Authorization header is absent
68+
* - the header is present but does not start with `Bearer ` (case-insensitive scheme per RFC 6750 §2.1)
69+
* - the bearer token does not match the expected token (constant-time compare)
70+
*
71+
* Calls `next()` on a successful match.
72+
*
73+
* Construction is fail-loud: if no `expectedToken` is supplied AND the
74+
* env var is unset/empty, this function throws `MissingAuthTokenError`
75+
* during boot. Defaulting to "always reject" or "always allow" would both
76+
* be footguns — a misconfigured deployment must surface during init, not
77+
* silently degrade once a trigger endpoint is hit.
78+
*/
79+
export function bearerAuth(opts: BearerAuthOptions = {}): MiddlewareHandler {
80+
const envVar = opts.envVar ?? DEFAULT_ENV_VAR;
81+
const rawExpected =
82+
opts.expectedToken !== undefined ? opts.expectedToken : process.env[envVar];
83+
84+
// R3-A.4: trim BOTH sides at construction so the comparison is symmetric
85+
// with the trimmed presented token below. Pre-fix, an OPS_TRIGGER_TOKEN
86+
// env var copy-pasted with surrounding whitespace would silently 401
87+
// every well-formed Bearer request — the presented side was trimmed but
88+
// the expected side held the leading/trailing spaces verbatim. Trim
89+
// here normalises both sides; an all-whitespace expected token is then
90+
// caught by the empty-string fail-loud guard below.
91+
const expectedToken = rawExpected?.trim() ?? "";
92+
93+
if (!expectedToken) {
94+
// Fail-loud: empty string, undefined, AND whitespace-only are all
95+
// treated as misconfig (whitespace-only collapses to "" after trim).
96+
throw new MissingAuthTokenError(envVar);
97+
}
98+
99+
return async (c, next) => {
100+
const header = c.req.header("Authorization") ?? "";
101+
// RFC 6750 §2.1: the auth-scheme is case-insensitive. Be lenient on
102+
// "Bearer "/"bearer " etc., strict on the token portion.
103+
const match = /^Bearer\s+(.+)$/i.exec(header);
104+
if (!match) {
105+
return c.json({ error: "unauthorized" }, 401);
106+
}
107+
const presented = match[1].trim();
108+
if (!constantTimeEqual(presented, expectedToken)) {
109+
return c.json({ error: "unauthorized" }, 401);
110+
}
111+
await next();
112+
};
113+
}

0 commit comments

Comments
 (0)