v6: Phase 1 — Arrow IPC consumption path on ReadResult (server-gated)

[Pandys]

Supersedes the original 2026-07-03 framing of this issue. This was written when "Arrow IPC over the streaming wire" was rejected (client-side-only Arrow conversion). The 2026-07-06 decision (planning/arrow-read-path-decision.md) re-opens that: the server team is prototyping server-side Arrow IPC over native gRPC, gated on a ≥2× benchmark. This issue is now Phase 1 of that hand-off.

⛔ Gated. Do not start until the server benchmark passes and Arrow is confirmed shipped on at least the staging cluster. Phase 0 (#58, #59) is built regardless.

The server writes Arrow IPC stream bytes into AggregateResponse.data when the request sets format = ARROW_IPC. One gRPC response chunk = one Arrow record batch. Consume it without ever materialising Python rows between the wire and the DataFrame.

Scope — the three (four) methods on ReadResult

• to_arrow() -> pa.Table — native Arrow, the fast base. pa.ipc.open_stream(chunk.data) per chunk, Table.from_batches(...); zero per-row Python objects.
• to_polars() -> pl.DataFrame — pl.from_arrow(self.to_arrow()), zero-copy.
• to_pandas() — when Arrow was negotiated, route through to_arrow() (fast path), not the JSON codec.
• __arrow_c_stream__ — Arrow PyCapsule interface so Polars/DuckDB/etc. pull results directly, no clappform glue.

Decisions to confirm with the server team before coding

• Chunk framing: is each chunk's .data a complete IPC stream (schema repeated per chunk → open_stream per chunk), or is the schema sent once and later chunks are schema-less batches (→ one long-lived RecordBatchStreamReader)? Both are valid; the benchmark settles it. Pick the reader after you get the answer.
• pandas dtypes: pd.ArrowDtype is near-zero-copy but changes dtypes notebooks may depend on. Recommendation: default read()/to_pandas() to numpy dtypes (no surprise), document to_pandas(arrow_dtypes=True) as opt-in.

Constraints (from planning/pypi-arrow-handoff.md §5, "what NOT to do")

• Native gRPC only — never route Arrow through the REST/JSON gateway (base64 negates the win).
• No list[dict] anywhere on the Arrow path — if you build one, you've lost the win.
• No eager concat of chunks into one giant bytes before decoding — keep batch-wise so streaming/memory-bounded reads survive.
• Not Arrow Flight — single-connection backend; Flight's parallel-endpoint benefit doesn't apply.

Depends on: #58, #59. Related: format negotiation/fallback and LocalMock/docs (separate Phase 1 issues).

Source: planning/pypi-arrow-handoff.md §4.1–4.2, §5; planning/arrow-read-path-decision.md §5 (Tier 1).