Skip to content

Support OpenAI Realtime SIP/SDP calls endpoint (POST /v1/realtime/calls) #55

Description

@claw-io

Support OpenAI Realtime SIP/SDP calls endpoint (POST /v1/realtime/calls)

Split from #53, separated out from the JSON-only Realtime session control endpoints because the transport and content shape are different.

Background

POST /v1/realtime/calls carries a raw SDP body with Content-Type: application/sdp, not JSON. This is the SIP/telephony entry point for OpenAI Realtime — used by clients that want to bridge a phone call into a Realtime model session. Searched src/ for realtime, client_secrets, realtime/calls, and application/sdp — zero hits.

The JSON-only Realtime session control endpoints (client_secrets, sessions, transcription_sessions, translations) are tracked separately; they share a handler shape with each other but not with this one.

Requested endpoint

Method Path Content-Type Body
POST /v1/realtime/calls application/sdp Raw SDP offer

Expected response: SDP answer from the provider with Content-Type: application/sdp and a Location header for call control, per OpenAI's Realtime SIP/SDP contract.

Why this needs its own issue

/v1/realtime/calls shares almost no code with the JSON session control endpoints:

  • Body type. All other proxy routes use await request.json() (main.py:963, main.py:1124) or a Pydantic body model (main.py:1426, main.py:1246). This one needs await request.body() with the bytes forwarded verbatim — and the response is also raw SDP bytes, not a JSON object.
  • Content-Type preservation. The handler must read Content-Type from the inbound request and forward it (along with the body bytes) to the provider unchanged. The provider response Content-Type: application/sdp must also be preserved end-to-end.
  • Logging. RawIOLogger.log_request(...) (main.py:993, main.py:1129) takes a parsed dict. The SDP handler needs a body-bytes variant of the request log, or a separate log entry, because SDP isn't valid JSON.
  • Response shape. litellm.EmbeddingResponse(**...) and JSONResponse(content=...) won't work here — the handler must return a Response(content=..., media_type="application/sdp") and copy upstream headers (notably Location).
  • Streaming. If the provider returns the SDP answer incrementally, the proxy needs to stream the body bytes too. Likely a one-shot SDP exchange, but worth confirming.
  • Error mapping. The existing per-exception mapping (main.py:1490-1513) assumes a JSON error response. For SDP, the provider's error body may be non-JSON or carry a SIP status code in a custom header.

Proposed initial implementation

Scope: thin passthrough with SDP body preservation. No SDP parsing, no transformation.

  1. Route. Register @app.post("/v1/realtime/calls") in src/proxy_app/main.py. Apply verify_api_key (main.py:1429 style).

  2. Body pass-through. Read the raw body with body_bytes = await request.body(). Forward body_bytes and the inbound Content-Type header verbatim to the resolved provider via RotatingClient. Do not parse the SDP. Do not apply apply_model_alias to SDP — model selection here is typically driven by a query parameter or the provider's account config, not by a request body field. Confirm with the OpenAI spec before the design step.

  3. Provider resolution. For the first pass, route to a fixed provider (e.g. OpenAI) selected by an env var or per-request header, since RotatingClient doesn't currently have a non-JSON request path. Routing for arbitrary OpenAI-compatible Realtime providers comes in a follow-up.

  4. Response. Return Response(content=upstream_body, status_code=upstream_status, headers=upstream_headers, media_type=upstream_content_type). Strip hop-by-hop headers (Connection, Transfer-Encoding, Content-Length) before forwarding.

  5. Error mapping. Two-tier:

    • If the upstream returns a JSON error body, parse and remap to OpenAI's error envelope (status code from the parsed error.code, message from error.message).
    • If the upstream returns a non-JSON error (or empty body), return the upstream status with an empty body and log the raw bytes at WARN for triage.
  6. Logging. Pipe through log_request_to_console(...) for the request line + headers only (skip body bytes for SDP to avoid log bloat — SDP offers are typically 1-3 KB). If ENABLE_RAW_LOGGING is set, log the body bytes separately under RawIOLogger with a SDP-specific marker so it doesn't pollute JSON request logs.

  7. Tests.

    • Happy path: POST a synthetic SDP offer (v=0\r\no=- ... m=audio ...) with Content-Type: application/sdp; assert response Content-Type is application/sdp and the response body parses as SDP.
    • 415: POST with Content-Type: application/json returns 415 Unsupported Media Type.
    • 401: missing/invalid PROXY_API_KEY.
    • Header preservation: upstream returns Location: <url> and the proxy preserves it in the response.
    • Body integrity: assert request.body() bytes equal what the provider received (use a mock upstream that echoes its input).

Open questions (decide before coding)

  • Provider selection. Is the model in the SDP offer (m=audio ... a=rtpmap ...), in a query param (?model=gpt-4o-realtime-preview), or in the provider's account config? Likely a query param based on OpenAI's docs, but verify before designing the URL shape.
  • Streaming. Can the SDP answer arrive incrementally, or is it always a single response? Affects whether the proxy needs StreamingResponse or a plain Response.
  • SIP headers. Some telephony clients send Via, From, To, Call-ID headers alongside the SDP body. Forward all of them? Or whitelist a known set?
  • Auth. Realtime calls may use a different credential than chat completions (e.g. Realtime-specific OpenAI key with billing model attached). Decide whether to add a REALTIME_API_KEY separate from per-credential PROXY_API_KEY.

Out of scope here

  • JSON-only Realtime session control (client_secrets, sessions, transcription_sessions, translations) — separate issue.
  • GET /v1/realtime WebSocket audio transport — separate issue.
  • SDP parsing/validation — the proxy should treat SDP as opaque bytes.

References (HEAD on dev: d77a246)

  • src/proxy_app/main.py:1423-1488 — existing /v1/embeddings handler (closest analog for passthrough structure, but JSON-only)
  • src/proxy_app/main.py:178-183EmbeddingRequest Pydantic model (does NOT apply here)
  • src/proxy_app/main.py:963await request.json() pattern (do NOT use here)
  • src/proxy_app/main.py:1490-1513 — per-exception error mapping (JSON-only; needs SDP variant)
  • src/proxy_app/main.py:776-794streaming_response_wrapper (may apply if SDP answers stream)
  • src/rotator_library/providers/example_provider.py:326,344-345 — Realtime model alias map

Checklist for review

  • Confirm provider selection mechanism (query param vs SDP body vs account config) before designing the URL shape
  • Confirm whether SDP answers can be incremental or are always single-shot
  • Confirm telephony-relevant SIP headers to forward
  • Decide on REALTIME_API_KEY separation from PROXY_API_KEY
  • Sign off on scope: passthrough only, no SDP parsing

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions