feat(video): native Google Veo + xAI Grok Imagine async video providers (#278)#282
Merged
Conversation
commit: |
…ideo providers (#278) Introduce the shared async-video proxy module and wire server dispatch, types, and metrics for Veo/Grok providers.
…rd-mode (#278) Implement the native Veo predictLongRunning poll lifecycle with record-mode fixtures, tests, and provider docs.
…fe dispatch (#278) Add the native xAI Grok Imagine video provider with record-mode fixtures, Sora-safe dispatch, tests, and provider docs.
Document the Veo and Grok Imagine video providers in the sidebar and README, and record the changelog entry.
jpr5
force-pushed
the
feat/video-veo-grok
branch
from
1e61168 to
d00981c
Compare
This was referenced Jun 28, 2026
Closed
jpr5
added a commit
that referenced
this pull request
Jun 28, 2026
…-class + docs (#285) ## What Completes the (still-unreleased) #274 `blocks` fixture-ordering feature so it works **uniformly across every provider**, and makes blocks-only fixtures **first-class**. Builds on #283 (blocks v1) and #282 (async video), both already on `main` under `[Unreleased]`. No version bump — stays `1.34.0`. The dedicated `chore: release v1.35.0` PR owns the bump. ## Why #283 shipped `blocks` for a subset of providers and left the record-side capture + several provider builders out of scope, with blocks-only fixtures treated as by-design-unsupported. To ship the feature as *complete* for 1.35.0, it needs to: capture/replay block order on the remaining providers, accept a fixture that is *just* `{ blocks: [...] }`, and document the feature where authors actually look. ## Changes **Feature completion** - **First-class blocks-only fixtures** — a response may be `{ blocks: [...] }` with no `content`/`toolCalls`; the shared recognizer was relaxed *additively* (legacy recognition byte-identical). - **Record side** — block-order capture added to the Cohere, Bedrock (event-stream), and Gemini-Interactions collapsers; `normalizeToolArguments` zero-arg ("{}") fix. Gemini-Interactions is args-only by design (step-index can't reconcile arrival-order blocks). - **Replay builders** — Cohere, Bedrock (invoke), Bedrock-Converse, and Gemini-Interactions now honor `blocks` ordering (streaming + non-streaming where the wire allows). - **toolCall `arguments`** may be a JSON object or a string (objects auto-stringify), consistently across top-level toolCalls and block toolCalls. - **Validation hardening** — `validateBlocks` rejects empty-text blocks and warns on blocks/content divergence. **Completion gaps found + fixed during code review** (each with a local red→green on the real surface) - Ollama **non-streaming** dropped the entire payload for a blocks-only fixture (F0 made it newly reachable) — now backfills content/tool_calls from blocks. - Realtime + Gemini-Live WS surfaces silently dropped blocks-only payloads — now honor blocks. - Programmatic `addFixture` with object block args returned HTTP 500 (bypassed normalize) — `resolveFixtureBlocks` now tolerates object args. - A valid `{ content: "", blocks: [...] }` fixture spuriously hard-errored at load — suppressed when non-empty blocks drive output. - Gemini-Live empty-text block leaked past `truncateAfterChunks` and shifted recorded-timing indices — guard now skips emission cleanly. **Docs** - `docs/fixtures/index.html`: per-provider observability matrix, blocks-only authoring, reasoning cells. - Authoring on-ramps: `write-fixtures` skill/command, `docs/examples` (worked **tool-first** example), `docs/record-replay`, root + pytest READMEs. - A runnable `fixtures/examples/llm/blocks-tool-first.json` with a permanent loadability test. ## Wire limitations (documented, not gaps) OpenAI-chat, Ollama, and Cohere **non-streaming** expose separate content/tool_calls fields, so order is not observable on the wire — payload is delivered, order is a no-op. Captured in the observability matrix. ## Test plan - Full suite: **144 files / 4244 tests** pass - `tsc --noEmit`, `eslint`, `prettier --check`, `build`, `test:exports` (node10/node16-CJS/ESM/bundler) all green - Every behavioral fix verified red→green on the real surface (live endpoint / real WS / real loader), independently re-confirmed in review ## Known non-blocking follow-ups Pre-existing items (predate this branch) and minor polish are tracked but intentionally out of scope: a shared stringify helper to de-dup the three normalize sites, JSON-serializability check for object args at load time, and assorted pre-existing collapser/request-converter edge cases.
Merged
jpr5
added a commit
that referenced
this pull request
Jun 28, 2026
## Release v1.35.0 Version bump cut off latest `main` (includes #285 blocks-completion and #286 video-testid). ### Version surfaces bumped (1.34.0 → 1.35.0) These five literal build surfaces track the aimock npm release line and were bumped together: 1. `package.json` — `"version"` → `1.35.0` 2. `.claude-plugin/plugin.json` — `"version"` → `1.35.0` 3. `.claude-plugin/marketplace.json` — `plugins[0].source.version` → `^1.35.0` (caret preserved) 4. `charts/aimock/Chart.yaml` — `appVersion` → `"1.35.0"` 5. `packages/aimock-pytest/src/aimock_pytest/_version.py` — `AIMOCK_VERSION` → `"1.35.0"` ### Deliberately NOT changed (independently versioned) Verified against release history — these do **not** track the aimock release line and follow their own cadence, so they are left as-is: - `charts/aimock/Chart.yaml` `version:` stays **`0.1.0`** (Helm chart version — independent of the app it packages) - `packages/aimock-pytest/pyproject.toml` `version` stays **`0.4.0`** (the Python package's own release cadence, distinct from the npm release it pins via `AIMOCK_VERSION`) ### Hygiene fix - `packages/aimock-pytest/README.md` — corrected the stale `--aimock-version` default from `1.33.0` → `1.35.0` to match `AIMOCK_VERSION`. ### CHANGELOG - Renamed the accumulated `## [Unreleased]` section to `## [1.35.0] - 2026-06-27` and inserted a fresh empty `## [Unreleased]` above it. Bullet contents unchanged. ### What's in 1.35.0 - **`blocks` feature completed across all providers** — ordered text/tool-call block streaming now honored on replay across Anthropic, OpenAI (Responses + chat-completions), Gemini, Ollama, Cohere, Bedrock (invoke + Converse), and Gemini Interactions, with record-side block capture where the wire protocol allows (#274). - **Blocks-only fixtures are first-class** — a non-empty `blocks` array is a complete response shape on its own (no `content`/`toolCalls` required); `validateBlocks` rejects malformed arrays at load time (#274). - **Veo / Grok multi-tenant testId isolation** — native Google Veo and xAI Grok Imagine async video lifecycle mocks with record-mode live proxying and per-tenant testId isolation (#278, #282). ### Publishing Publishing is **not** performed here. The release is cut and verified; the actual publish is the manual `npm run release` step, pending maintainer action. ### Verification - `npm run build` — clean (exit 0) - `npx vitest run` — **4250/4250 passed**, 144/144 files (exit 0) - `npm run test:exports` — all 🟢 (node10 / node16 CJS+ESM / bundler, across all entry points) - Type-emitting build clean (`.d.ts` emit succeeded; no separate `tsc` script in this package)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #278.
Adds first-class aimock support for two async video-generation providers, mirroring the existing Sora / OpenRouter / fal job-lifecycle model — replay and live record-mode proxy for both.
Google Veo (Gemini)
POST /v1beta/models/{model}:predictLongRunning→{ name }, thenGET /v1beta/operations/{name}poll →{ done:false }…{ done:true, response.generateVideoResponse.generatedSamples[].video.uri }, with poll-count progression.uriis served as-is (aimock does not proxy the video bytes — Veo's envelope carries no cost field).veoupstream and eager-captures the fixture ondone:true.xAI Grok Imagine
POST /v1/videos/generations→{ request_id }(multipart rejected with 400), thenGET /v1/videos/{request_id}→{ status: pending|done|failed|expired, progress, video, usage.cost_in_usd_ticks }. Progress synthesized from poll count;cost_in_usd_ticks= USD × 1e10./v1/videos/{id}with Sora: job-map-first lookup, falling through to the unchanged Sora handler on a miss (pinned byte-for-byte by a test).grokupstream and eager-captures on terminal status.Shared proxy helpers were extracted into
src/video-proxy-shared.ts(behavior-preserving; openrouter-video re-pointed at it, its 233 tests green before & after).Verification
tsc --noEmitall clean.Known follow-ups (not blocking)
@google/genaiforwardsx-test-idonoperations.getVideosOperationpolls; if it does not, Veo would need testId conveyance via the operation name (like OpenRouter'stestIdSuffix) for non-default-testId clients. Consistent with Sora's contract today.Number.isFiniteguards + warns on malformed upstream duration/cost.🤖 Generated with Claude Code