[pull] main from CopilotKit:main#307
Merged
Merged
Conversation
## Summary Removes the unused QA-to-Notion sync feature from the showcase: - Deleted `.github/workflows/showcase_qa-sync.yml` (the "Showcase: Sync QA to Notion" workflow). - Deleted `showcase/scripts/sync-qa-to-notion.ts` (the sync script). - Removed the `sync-qa` and `sync-qa:dry` npm script entries from `showcase/scripts/package.json`. A reference sweep across the repo (excluding `node_modules`) for `sync-qa`, `sync_qa`, `syncQa`, `sync-qa-to-notion`, `showcase_qa-sync`, and `qa-sync` found no other references after these removals. ## Rationale Unused per repo owner. The `qa/*.md` integration content under `showcase/integrations/*/qa/**` is intentionally retained (validate-parity counts it); only the sync machinery is removed. ## Orphaned secret `SHOWCASE_NOTION_API_KEY` was referenced only by the removed `showcase_qa-sync.yml` workflow. It is now orphaned and can be deleted from the repo's GitHub Actions secrets if it has no other use. ## Test plan - [ ] Confirm CI is green on the branch. - [ ] Confirm no other workflow references the removed job. - [ ] (Optional) Delete the orphaned `SHOWCASE_NOTION_API_KEY` GitHub Actions secret.
…nected browsers to avoid PID-exhaustion crashes
…5133) ## Summary The staging \`d6-all-pills-e2e\` probe was aborting at T+2s with every per-feature \`browser.newContext()\` failing with \"Target page, context or browser has been closed.\" ### Root cause The probe-invoker's worker fan-out launches \`max_concurrency\` workers in the SAME JS tick (a 4ms thundering herd). On d6 (\`max_concurrency: 8\`, ~8 services) each worker's first call drives \`pool.acquire()\` → a Chromium launch (~50 threads/procs each) on top of the already-warm 10-browser pool. The container hits its PID/thread ceiling, the kernel returns EAGAIN on fork/pthread_create, fresh Chromium processes die, and per-feature \`browser.newContext()\` then fails on every single feature — wiping out every service's full ~40-feature matrix. Not OOM, not a code race in the pool's locking. ### Fix 1. **Stagger service startup** in the invoker's worker fan-out (\`probe-invoker.ts\`): worker \`i\` sleeps \`i * SERVICE_STARTUP_STAGGER_MS\` before its FIRST pull, so per-Chromium thread-spawn bursts (~150ms each in practice) overlap rather than collide. Subsequent iterations of each worker run at full speed — wall-clock throughput is preserved. Configurable via \`SERVICE_STARTUP_STAGGER_MS\` env (default 300ms, set 0 to disable). 2. **Defensive re-acquire** in \`createPooledE2eFullLauncher\` (\`d6-all-pills.ts\`): after \`pool.acquire()\`, check \`browser.isConnected()\`; if false, \`pool.release(browser)\` + re-acquire once. Covers the narrow window where a browser dies AFTER acquire returns but BEFORE the caller hands it to \`newContext()\` — so a dead browser doesn't doom an entire service's ~40 features. \`BROWSER_POOL_SIZE\`, \`max_concurrency\`, and \`FEATURE_CONCURRENCY_D6\` defaults are unchanged — concurrency is preserved; the stagger only spreads the initial-acquire burst. ## Test plan - [x] \`pnpm typecheck\` passes for \`showcase/harness\` - [x] Targeted vitest run (\`d6-all-pills.test.ts\`, \`browser-pool.test.ts\`, \`probe-invoker.test.ts\`) passes — added focused coverage for stagger ordering, stagger=0 opt-out, and re-acquire-on-disconnected - [x] Full harness vitest (1668 tests) passes - [x] \`oxfmt --write\` + \`oxlint\` on touched files; no new warnings introduced by this change - [ ] Watch CI to green - [ ] Verify in staging after merge: d6 runs complete past T+2s and emit non-empty per-service aggregates
…out + lower FEATURE_CONCURRENCY_D6 to isolate fixture-miss cascade
## Root cause In `d6-all-pills-e2e`, each service acquires ONE pooled browser and runs `FEATURE_CONCURRENCY_D6` (=4) feature workers concurrently via `browser.newContext()`. When a feature's prompt has no matching aimock fixture, the agent hangs/loops until the 300s `featureTimeoutMs`. Four such concurrent hung 5-minute contexts (each holding a loaded page + accumulating SSE/DOM state) create enough memory pressure to OOM-kill the shared Chromium subprocess. After Chromium dies, every subsequent `browser.newContext()` / `newPage()` on that handle fails with `"Target page, context or browser has been closed"` — wiping the rest of that service's ~40 features and destroying per-feature D6 signal. The per-feature timeout path (`d6-all-pills.ts`) already aborts the hung feature but does NOT recycle the shared browser, so the cascade propagates. ## Fix 1. **Recycle the pooled browser between features when disconnected / after timeout.** The per-service feature loop now checks `browser.isConnected()` before each feature's `newContext()` call and single-flight recycles (close + relaunch) when dead. It also recycles proactively after a `feature-timeout` result so the next feature on this worker does not inherit the pressure. Capped at `MAX_BROWSER_RECYCLES_PER_SERVICE = 5` to prevent infinite-loop on a poisoned pool. 2. **Lower `FEATURE_CONCURRENCY_D6` from 4 to 2** to roughly halve the simultaneous in-flight contexts per service. Worst-case wall-clock per service rises modestly (~10-14 min vs ~7-10 min) but per-feature signal quality — the actual ask of D6 — matters more here. Now env-overridable via `FEATURE_CONCURRENCY_D6`. ## Out of scope Does NOT change `max_concurrency`, `BROWSER_POOL_SIZE`, or `DEFAULT_FEATURE_TIMEOUT_MS`. The pool's own dead-slot recovery (PR #5133) handles the per-launcher zombie-acquire case; this PR adds the orthogonal between-features-in-one-service recovery. ## Test plan - [x] `pnpm typecheck` clean - [x] `pnpm vitest run src/probes/drivers/d6-all-pills src/probes/helpers/browser-pool` — 43 tests pass, including two new focused tests for the between-features recycle path (one verifies the swap to a fresh browser, one verifies the recycle cap) - [x] Full harness `pnpm vitest run` — 1670 tests pass - [x] `oxfmt --write` + `oxlint` on touched files (no new warnings) - [ ] Production verification: deploy to harness service, observe a D6 run where a fixture-miss happens and confirm subsequent features in the same service still run (`probe.e2e-full.between-features-recycle` log + fresh per-feature results)
…5134) (#5135) Reverts #5134. Post-deploy verification on the 23:40Z staging d6 run showed #5134 did NOT isolate the cascade AND introduced a regression: 12/18 services fail with `BrowserPool acquire timeout` before running any feature (pool starvation from the recycle-between-features path + FEATURE_CONCURRENCY_D6 4→2). Reverting to restore the #5133 state where services at least execute features. The BrowserPool cascade needs deeper investigation; tracking separately.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )