Tags: CopilotKit/pathfinder
Tags
chore: release v1.16.0 (#129) ## Release v1.16.0 (minor) Releases the user-visible changes already merged to `main`: **#124** (config validation relaxation) and **#127** (Atlas distiller "Premature close" runtime fix). Current `1.15.4` → `1.16.0`. #128 was a CI-only vitest scoping change and is intentionally not in the changelog (does not ship). ### CHANGELOG ## 1.16.0 ### Minor Changes - Config validation accepts minimal RAG configs — `indexing` and per-source `chunk` now optional with defaults (#124) - Fixed Atlas distiller LLM client "Premature close" errors, by disabling keep-alive on the test/proxy client (#127) ### Version bumps - `package.json` `version`: 1.15.4 → 1.16.0 - `src/cli.ts` `.version(...)`: 1.15.4 → 1.16.0 - version-sync verified in agreement. ### Publishing Merging this PR triggers `publish-release.yml` to publish `@copilotkit/pathfinder@1.16.0` to npm and create the tag/GitHub Release. Do not tag or release by hand.
Release v1.15.4 — re-ship v1.15.3 analytics changes with fixed SQL + … …defensive Promise.allSettled (#118) ## Summary Re-ships the three reverted v1.15.3 changes (blocked filter on empty-results, new Blocked Queries panel, Pacific timestamps) with the two defects fixed that took v1.15.3 down within minutes of deploy (reverted via PR #117): 1. **`getBlockedQueries` SQL — rewrote as a CTE-backed correlated subquery.** v1.15.3 grouped the outer query on `COALESCE(block_reason, '<unknown>')` but the correlated subquery referenced raw `query_log.block_reason`, which Postgres rejects as an ungrouped column reference (`subquery uses ungrouped column "query_log.block_reason" from outer query`). The CTE (`blocked_rows.block_reason_key`) pre-projects the COALESCEd expression so the outer `GROUP BY` and the inner correlated reference operate on the same column. 2. **`Promise.all` → `Promise.allSettled` in `docs/analytics.html`.** A single panel's fetch failure used to throw out of `Promise.all` and blank the entire `/analytics` page (which is what made v1.15.3's SQL bug catastrophic instead of merely cosmetic). Each panel now renders independently. Auth (401) still bubbles to the outer catch since a bad token fails every endpoint identically; a failed summary still surfaces via the top-level banner. Per-panel failures degrade to an empty state with `console.error`. ## Local verification (the discipline that was missing on v1.15.3) Booted Pathfinder locally against the real prod `DATABASE_URL`: - `/api/analytics/blocked-queries?days=7` returned **HTTP 200** + valid JSON. Verified with three test rows inserted into prod: response correctly grouped one bucket by `block_reason='hotfix-test'` (2 hits, 2 sample_queries) and one `<unknown>` bucket for the NULL row (1 hit, 1 sample_query). Test rows then deleted; prod has zero blocked rows in the 7-day window (current state). - `/api/analytics/empty-queries?days=7` returned HTTP 200, no abuse strings (toy story / box office / kalshi / scotus) leaked into results. - `/analytics` page returned HTTP 200 with no rendered error banner. Pre-fix SQL was directly verified to fail against prod psql with the exact error above. Post-fix SQL was directly verified to return real grouped rows when run against prod psql with a wider window (`days=30` returns ~13k legacy `<unknown>` rows). Both pre-fix and post-fix were observed end-to-end against the actual production schema before push. ## Test plan - [ ] CI green - [ ] Post-deploy: `/analytics` renders all sections, no error banner - [ ] `/api/analytics/blocked-queries?days=7` returns 200 + grouped JSON
Release v1.15.3 — analytics view: blocked filter + Blocked Queries pa… …nel + Pacific timestamps (#116) Three operator-facing /analytics fixes shipped together: 1. **Hide `blocked=true` rows from "Empty Result Queries (Last 7 days)"** — v1.15.2 already short-circuits these via the blocklist; they were visible in the empty-result table only because the SQL filtered on `result_count = 0` alone. Added `AND blocked = false` to `getEmptyQueries`. Predicate is safe for historical rows that predate the column (schema declares `BOOLEAN NOT NULL DEFAULT FALSE`). 2. **New "Blocked Queries (Last 7 days)" panel** on /analytics — count + last_seen + up to 5 sample queries per `block_reason`, surfaced via a new `/api/analytics/blocked-queries` route. Operator can now see the blocklist actively catching abuse. Window honors `days` but takes no other filter params (the rows it surfaces never reached the layer those filters describe). 3. **All timestamps now render in Pacific time** (America/Los_Angeles, DST-aware via `Intl.DateTimeFormat`) — emits e.g. `2026-06-17 09:40:32 PDT` (or `PST` in winter) instead of UTC. Storage (Postgres TIMESTAMPTZ in UTC), server-side window math, and the URL `from`/`to` contract are all unchanged — only the display side of absolute-instant cells flips. No search API or telemetry changes — operator-facing only. ## Test coverage - `getEmptyQueries` SQL now contains `blocked = false` (regression-pinned). - `getBlockedQueries` returns one row per `block_reason`, groups null-reason rows under `<unknown>`, orders by hits DESC, and uses a parameterized window interval. - `/api/analytics/blocked-queries` route: 200 with valid token, 401 without, 500 envelope on DB-layer rejection. - Dashboard UI: empty-queries `last_seen` cell renders PDT (June) and PST (January) markers; missing/non-parseable values render an em-dash; new Blocked Queries panel fetches the endpoint, renders one row per reason with PDT marker, joined sample queries, and shows a 'No blocked queries' fallback when empty. ## Notes for reviewer - The pre-existing UTC scaffolding (`parseISODate`, `todayISO`, `tomorrowISO`, the `shortDateFmt` pill formatter) is intentionally NOT changed — those govern the URL `from`/`to` contract and date-picker math, which must stay UTC-aligned with the server's window math. Only operator-facing absolute timestamps flip to Pacific. - Pre-existing failures in `cli.test.ts` (dist-not-built env) are out of scope; verified they fail identically on `main`.
Release v1.15.2 — abuse query-pattern blocklist + query_log IP/UA att… …ribution (#113) ## Summary Two related changes, shipped together as v1.15.2: 1. **Abuse query-pattern blocklist** (`src/mcp/abuse-blocklist.ts`) — short-circuits searches for off-topic terms identified from production analytics (Toy Story 5 / Disclosure Day / Obsession 2026 / Scary Movie / box-office / SCOTUS Kalshi / CFTC certiorari / sports event contracts). Matched queries return a structured `{ results: [], blocked: true, domain: ..., hint: ... }` response — teaches the calling LLM what's actually in scope. Zero false-positives on legitimate documentation queries (negative tests in `src/__tests__/abuse-blocklist.test.ts` cover near-miss phrasings like `useCopilotAction box for the office layout`). Wired into both `registerSearchTool` and `registerKnowledgeTool` (search-mode only on knowledge; browse mode has no user query). 2. **`query_log` attribution columns** — added `client_ip`, `user_agent`, `blocked`, `block_reason`. Per-request IP/UA captured at MCP session init via the same trust-proxy boundary as `oauthClientIp`, then closed over for the lifetime of the session so every tool call records identical attribution — no more session-id-prefix joins against PostHog for IP attribution. Schema additions are nullable / defaulted (`blocked DEFAULT FALSE`); backward compatible. ## Context A recent analytics investigation traced a high-volume off-topic empty-query cluster to Anthropic's `Claude-User` egress pool (`160.79.106.32/29`) — a shared Anthropic infrastructure pool that ALSO serves legitimate Claude-User fetches to this server (~46k legit sessions/7d). IP-level block would punish ~99.9% legit users. Pattern-based block is the surgical alternative. Long-term defense (scope classifier with adaptive per-session/per-IP reputation) is still on the roadmap. ## Observability A new `oauthLog.searchBlocked({ ip, reason, tool })` helper emits `[oauth] search_blocked reason=pattern:<name> ip=<ip> tool=<tool>` on every block, parallel to the existing `[oauth] bearer_failure` surface. Greppable for abuse volume independent of the per-row `query_log.blocked` flag. ## Test plan - [ ] CI green (tsc, prettier, tests, build) - [ ] Post-merge: smoke-test a known abuse string (e.g. `Toy Story 5 box office`) on `mcp.copilotkit.ai` → expect `blocked: true` response + `[oauth] search_blocked` log line - [ ] Smoke-test a legit query (e.g. `useCopilotAction onClick handler`) → normal results - [ ] DB inspect: `SELECT client_ip, user_agent, blocked, block_reason FROM query_log ORDER BY created_at DESC LIMIT 10;` — confirm new columns populated
Release @copilotkit/pathfinder 1.14.0 (atlas CLI) (#95) ## Summary Version-only release cut. Bumps `@copilotkit/pathfinder` **1.13.3 → 1.14.0** so the version-gated `publish-release.yml` workflow fires on merge and publishes `@copilotkit/pathfinder@1.14.0` to npm — carrying the new `bin.atlas` CLI. The server/source code is **unchanged from #94** (commit `d43b88e`), which is already deployed to prod; this PR only changes the version (`package.json`, `package-lock.json`, `src/cli.ts`) and adds a CHANGELOG entry. ### Why - The npm publish is version-gated: pushing to `main` publishes only when `package.json` version is unpublished. `1.14.0` is not yet on npm (verified `npm view @copilotkit/pathfinder@1.14.0` → 404), so merging this fires the publish. - Unblocks internal-skills #121, which needs to pin `@copilotkit/pathfinder@1.14.0` for the `atlas` CLI. ### What 1.14.0 ships (all from #94, now released) - **`atlas` CLI** — first-party client as `bin.atlas`: `atlas search` + `atlas feedback`, with hardened `tools/call` response handling. - **`prepublishOnly` build guard** — the published tarball always ships a fresh `dist/` (incl. `dist/atlas-cli.js`). - **Atlas foundation** — off-by-default codebase-knowledge layer: additive schema, ratification endpoints, gardener, webhook PR ingestion. Disabled unless explicitly enabled; existing deployments unaffected. ### Bump rationale Minor bump (new additive `atlas` CLI feature, backward compatible). Repo uses a plain `package.json` version bump (no changesets/release-please). `src/cli.ts` `.version()` bumped in lockstep to satisfy the `version-sync` CI gate and the publish workflow's "Verify CLI version matches package version" check. ## Local gate results (node 25 / `/tmp/pf-release`) - prettier `--check` (package.json, src/cli.ts, CHANGELOG.md): **clean** - `scripts/check-version-sync.sh`: **✓ in sync (1.14.0)** - `npx tsc --noEmit`: **0 errors** - `npm run build`: **succeeds**, emits `dist/atlas-cli.js`, `dist/cli.js`, `dist/index.js` - `npm test`: **3712 passed (254 files)** after build (CI test job builds before testing, matching this order) ## Merge note Do **not** auto-merge. The orchestrator will merge after the prod deploy is confirmed healthy; merge fires the npm publish.
Post-reindex health audit with Slack alerting (#60) ## Summary - **Post-reindex health audit** — three read-only checks after each reindex cycle: 1. **Stale files**: DB entries with no corresponding file on disk 2. **Scope leaks**: DB paths outside the configured `config.path` prefix (git sources) 3. **Count divergence**: mismatch between DB and disk file counts - **Slack alerting** via `SLACK_WEBHOOK_URL` env var — fires when any check detects issues - **`walkSourceFiles` utility** — enumerates matching files for a source config without reading content - 25 new tests (18 audit + 7 walkSourceFiles) ## Why After fixing the stale chunk cleanup bug (PR #59), we need monitoring to detect if the index drifts again — stale files accumulating, files indexed outside their configured scope, or count mismatches between what's on disk and what's in the DB. ## Test plan - [x] 18 reindex audit tests pass (stale files, scope leaks, count divergence, Slack alerting, error handling) - [x] 7 walkSourceFiles tests pass - [x] Full test suite passes (1708 tests, only pre-existing cli.test.ts failures) - [x] TypeScript clean - [ ] Set `SLACK_WEBHOOK_URL` on Railway and verify alert fires on next reindex
Enable JSON responses on Streamable HTTP for Codex CLI compatibility (#… …57) ## Summary A community user reported that Pathfinder's MCP server doesn't work with OpenAI Codex CLI — initialization fails with connection drops. Confirmed working with Claude Code. **Root cause:** Codex CLI uses the `rmcp` Rust MCP SDK, which has open bugs ([#5619](openai/codex#5619), [#11284](openai/codex#11284)) around parsing SSE-formatted responses from Streamable HTTP endpoints. Pathfinder was returning `Content-Type: text/event-stream` for all responses (including simple request/response like `initialize` and `tools/list`), which rmcp fails to parse — it closes the connection before reading the response data. **Fix:** The MCP SDK's `StreamableHTTPServerTransport` has an `enableJsonResponse` option (default: `false`). When enabled, the server responds with `Content-Type: application/json` instead of SSE-wrapping for single-message exchanges. This is one line: ```typescript const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: () => preSid, enableJsonResponse: true, // ← new ... }); ``` **Spec compliance:** Fully compliant. The MCP spec (2025-06-18) explicitly states: "If the input is a JSON-RPC request, the server MUST either return `Content-Type: text/event-stream`, to initiate an SSE stream, or `Content-Type: application/json`, to return one JSON object. The client MUST support both these cases." The server gets to choose. **Tradeoff:** Loses the ability to send server-initiated notifications mid-request via SSE. Pathfinder's tools are all request/response — none stream intermediate results. The GET SSE endpoint still works for server-initiated messages. **No impact on Claude Code** or any other spec-compliant client, since they're required to handle both formats. ## Changes - `src/server.ts`: Add `enableJsonResponse: true` to `StreamableHTTPServerTransport` constructor - `docs/clients/index.html`: Add Known Concerns section documenting SSE parsing bugs in some MCP clients, our mitigation, and the `mcp-remote` stdio bridge workaround for any remaining edge cases - Version bump to 1.13.1 (`package.json`, `src/cli.ts`, `package-lock.json`, `CHANGELOG.md`) ## Test plan - [x] Started Pathfinder locally with breeze-docs fixture - [x] Verified `/mcp` POST returns `Content-Type: application/json` (previously returned `text/event-stream`) - [x] Verified response body is plain JSON-RPC (`{"result":{"protocolVersion":"2025-06-18",...},"jsonrpc":"2.0","id":1}`) — not SSE-wrapped (`event: message\ndata: {...}\n\n`) - [x] Ran `codex exec` against local Pathfinder — successfully initialized and discovered MCP tools (`search_breeze_docs`, `submit_breeze_feedback`) - [x] Claude Code connectivity unaffected (spec-compliant clients handle both formats)
PreviousNext