atlas: disable keep-alive on the test/proxy LLM client to fix Premature close#127
Merged
Merged
Conversation
…re close The Atlas LLM seam (OpenAIDistiller) is exercised in tests against an in-process aimock server via the OpenAI SDK, whose node-fetch transport pools keep-alive sockets. Under the full suite — every aimock-backed test file runs its own server in parallel, doubled by the built dist copy — a server can close an idle pooled socket exactly as a request reuses it, which node-fetch surfaces as 'FetchError: ... Premature close'. This made all aimock-backed suites (atlas-llm, atlas-exclude, atlas-adapter-episodic, atlas-artifact-sync, ...) fail under load: flaky locally, deterministic in CI (64 tests, incl. the 9 atlas-artifact-sync failures). When a baseURL is set (in this repo only ever a local aimock/proxy), pin a non-keep-alive http/https Agent so each request uses a fresh socket, removing the reuse race. Production (no baseURL -> real OpenAI over HTTPS) keeps the SDK default keep-alive agent.
jpr5
added a commit
that referenced
this pull request
Jun 25, 2026
## What Add `vitest.config.ts` scoping test discovery to `src/**`, excluding `dist/**` (and `.claude/**` worktree checkouts). ## Why CI currently has no vitest config, so default discovery collects **both** `src/__tests__/*.test.ts` **and** their compiled `dist/__tests__/*.test.js` twins — every test runs twice. Beyond the wasted wall-clock, the duplicate aimock-backed runs double the exposure to the LLMock socket-reuse race (the "Premature close" failures addressed by #127). This change makes each test run once. ## Proof (deterministic) `vitest list` collection: - dist test cases collected: **23743 → 0** - src test cases collected: **3186** (unchanged) Build green, prettier clean. ## Sequencing Independent of #127, but its CI is most reliably green once #127's keep-alive root-cause fix is in `main` — recommend merging after #127. Rebase + re-run available on request.
Merged
jpr5
added a commit
that referenced
this pull request
Jun 25, 2026
## Release v1.16.0 (minor) Releases the user-visible changes already merged to `main`: **#124** (config validation relaxation) and **#127** (Atlas distiller "Premature close" runtime fix). Current `1.15.4` → `1.16.0`. #128 was a CI-only vitest scoping change and is intentionally not in the changelog (does not ship). ### CHANGELOG ## 1.16.0 ### Minor Changes - Config validation accepts minimal RAG configs — `indexing` and per-source `chunk` now optional with defaults (#124) - Fixed Atlas distiller LLM client "Premature close" errors, by disabling keep-alive on the test/proxy client (#127) ### Version bumps - `package.json` `version`: 1.15.4 → 1.16.0 - `src/cli.ts` `.version(...)`: 1.15.4 → 1.16.0 - version-sync verified in agreement. ### Publishing Merging this PR triggers `publish-release.yml` to publish `@copilotkit/pathfinder@1.16.0` to npm and create the tag/GitHub Release. Do not tag or release by hand.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Disable HTTP keep-alive on
OpenAIDistiller's client only when abaseURLis configured (in this repo, always a local aimock/proxy server in tests). Production (nobaseURL→ real OpenAI over HTTPS) keeps the SDK's default keep-alive agent. One-file change insrc/atlas/llm.ts.Root cause
The
Static Qualitytestjob went green on 2026-06-23 and red on 2026-06-25 with no material code change — it is a load-dependent flake that turned deterministic under CI's contention, not a date/time bomb.The Atlas LLM seam (
OpenAIDistiller) is exercised against an in-process aimock server through the OpenAI SDK, whose node-fetch transport pools keep-alive sockets. Under the full suite, every aimock-backed test file runs its own server in parallel — doubled by the committed-then-rebuiltdist/copy. A server can close an idle pooled socket exactly as a request reuses it, which node-fetch surfaces asFetchError: ... Premature close. This affected all aimock-backed suites (atlas-llm,atlas-exclude,atlas-adapter-episodic,atlas-artifact-sync), including the 9atlas-artifact-syncfailures in this report.Pinning a non-keep-alive
http/httpsAgent for the mock/proxy case forces a fresh socket per request, removing the reuse race.Verbatim RED (from CI run 28190002185, 2026-06-25)
Local RED-GREEN proof (real surface: OpenAIDistiller + real aimock server + OpenAI SDK transport)
The race is environment-sensitive (deterministic on CI's contended Ubuntu runner, invisible on a fast 18-core dev box under the natural suite). To prove the fix on the real failure surface, a harness drove a real
OpenAIDistilleragainst a realLLMockthrough a TCP proxy that reaps idle keep-alive sockets — modeling a server closing a pooled socket as the client reuses it. The ONLY variable is the keep-alive setting the fix toggles:The SDK masks the torn-socket-on-reuse as "Connection error" locally; CI's node-fetch transport surfaces the same race as "Premature close" — same root cause, same fix.
GREEN: affected suites (src + built dist), post-fix
The 9
atlas-artifact-syncfailures (and the paralleldistcopies) all pass.Other Static Quality gates (run locally, all green)
Scope
One file (
src/atlas/llm.ts). No changes to notification workflows, RAG config, or anything unrelated.dist/is gitignored and regenerated by CI'snpm run build, so no committed compiled copy to sync.