Skip to content

Commit b94478e

Browse files
ericc-chericc-ch
committed
feat: Add streaming support and examples for Anthropic messages API
1 parent 44f733f commit b94478e

3 files changed

Lines changed: 472 additions & 4 deletions

File tree

‎docs/anthropic.md‎

Lines changed: 285 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -77,9 +77,41 @@ A successful **non-streaming** request returns a `Message` object.
7777

7878
#### Streaming Response (200 OK)
7979

80-
If `stream: true` is set, the API streams back a sequence of server-sent events. The response is a series of JSON events that incrementally build the complete message object.
80+
When `stream: true` is set, the API streams the response using server-sent events (SSE). Each event is named (e.g., `event: message_start`) and contains associated JSON data.
8181

82-
According to the documentation, the `stop_reason` provides insight into the stream's state: in the initial `message_start` event, the `stop_reason` field will be `null`. In all other events, it will be non-null once the stopping condition is known.
82+
The event flow for a stream is as follows:
83+
84+
1. `message_start`: Contains a `Message` object with empty `content`.
85+
2. A series of content blocks. Each block has a `content_block_start` event, one or more `content_block_delta` events, and a `content_block_stop` event. The `index` in these events corresponds to the content block's position in the final `content` array.
86+
3. One or more `message_delta` events, which indicate top-level changes to the final `Message` object. The `usage` field in this event contains cumulative token counts.
87+
4. A final `message_stop` event.
88+
89+
The stream may also include `ping` events to keep the connection alive and `error` events if issues occur.
90+
91+
##### Content Block Delta Types
92+
93+
Each `content_block_delta` event contains a `delta` object that updates a content block.
94+
95+
- **Text Delta**: Updates a `text` content block.
96+
97+
```json
98+
event: content_block_delta
99+
data: {"type": "content_block_delta","index": 0,"delta": {"type": "text_delta", "text": "ello frien"}}
100+
```
101+
102+
- **Input JSON Delta**: Used for `tool_use` blocks, these deltas contain partial JSON strings for the tool's `input` field. The partial strings must be accumulated and parsed into a final JSON object upon receiving the `content_block_stop` event.
103+
104+
```json
105+
event: content_block_delta
106+
data: {"type": "content_block_delta","index": 1,"delta": {"type": "input_json_delta","partial_json": "{\"location\": \"San Fra"}}}
107+
```
108+
109+
- **Thinking Delta**: When extended thinking is enabled, these deltas update the `thinking` field of a thinking content block. A special `signature_delta` event is sent just before the `content_block_stop` to verify the block's integrity.
110+
111+
```json
112+
event: content_block_delta
113+
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "thinking_delta", "thinking": "Let me solve this step by step:\n\n1. First break down 27 * 453"}}
114+
```
83115

84116
#### The Usage Object
85117

@@ -93,6 +125,257 @@ The `usage` object details billing and rate-limit token counts.
93125
| `cache_read_input_tokens` | integer | The number of input tokens read from the cache. |
94126
| `service_tier` | string | The service tier used for the request (`standard`, `priority`, or `batch`). |
95127

128+
### Streaming Examples
129+
130+
#### Basic Streaming Request
131+
132+
```bash
133+
curl https://api.anthropic.com/v1/messages \
134+
--header "anthropic-version: 2023-06-01" \
135+
--header "content-type: application/json" \
136+
--header "x-api-key: $ANTHROPIC_API_KEY" \
137+
--data \
138+
'{
139+
"model": "claude-opus-4-20250514",
140+
"messages": [{"role": "user", "content": "Hello"}],
141+
"max_tokens": 256,
142+
"stream": true
143+
}'
144+
```
145+
146+
**Response:**
147+
148+
```json
149+
event: message_start
150+
data: {"type": "message_start", "message": {"id": "msg_1nZdL29xx5MUA1yADyHTEsnR8uuvGzszyY", "type": "message", "role": "assistant", "content": [], "model": "claude-opus-4-20250514", "stop_reason": null, "stop_sequence": null, "usage": {"input_tokens": 25, "output_tokens": 1}}}
151+
152+
event: content_block_start
153+
data: {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}}
154+
155+
event: ping
156+
data: {"type": "ping"}
157+
158+
event: content_block_delta
159+
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": "Hello"}}
160+
161+
event: content_block_delta
162+
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": "!"}}
163+
164+
event: content_block_stop
165+
data: {"type": "content_block_stop", "index": 0}
166+
167+
event: message_delta
168+
data: {"type": "message_delta", "delta": {"stop_reason": "end_turn", "stop_sequence":null}, "usage": {"output_tokens": 15}}
169+
170+
event: message_stop
171+
data: {"type": "message_stop"}
172+
```
173+
174+
#### Streaming Request with Tool Use
175+
176+
```bash
177+
curl https://api.anthropic.com/v1/messages \
178+
-H "content-type: application/json" \
179+
-H "x-api-key: $ANTHROPIC_API_KEY" \
180+
-H "anthropic-version: 2023-06-01" \
181+
-d '{
182+
"model": "claude-opus-4-20250514",
183+
"max_tokens": 1024,
184+
"tools": [
185+
{
186+
"name": "get_weather",
187+
"description": "Get the current weather in a given location",
188+
"input_schema": {
189+
"type": "object",
190+
"properties": {
191+
"location": {
192+
"type": "string",
193+
"description": "The city and state, e.g. San Francisco, CA"
194+
}
195+
},
196+
"required": ["location"]
197+
}
198+
}
199+
],
200+
"tool_choice": {"type": "any"},
201+
"messages": [
202+
{
203+
"role": "user",
204+
"content": "What is the weather like in San Francisco?"
205+
}
206+
],
207+
"stream": true
208+
}'
209+
```
210+
211+
**Response:**
212+
213+
```json
214+
event: message_start
215+
data: {"type":"message_start","message":{"id":"msg_014p7gG3wDgGV9EUtLvnow3U","type":"message","role":"assistant","model":"claude-opus-4-20250514","stop_sequence":null,"usage":{"input_tokens":472,"output_tokens":2},"content":[],"stop_reason":null}}
216+
217+
event: content_block_start
218+
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
219+
220+
event: ping
221+
data: {"type": "ping"}
222+
223+
event: content_block_delta
224+
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Okay"}}
225+
226+
event: content_block_delta
227+
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":","}}
228+
229+
event: content_block_delta
230+
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" let"}}
231+
232+
event: content_block_delta
233+
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"'s"}}
234+
235+
event: content_block_delta
236+
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" check"}}
237+
238+
event: content_block_delta
239+
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" the"}}
240+
241+
event: content_block_delta
242+
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" weather"}}
243+
244+
event: content_block_delta
245+
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" for"}}
246+
247+
event: content_block_delta
248+
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" San"}}
249+
250+
event: content_block_delta
251+
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" Francisco"}}
252+
253+
event: content_block_delta
254+
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":","}}
255+
256+
event: content_block_delta
257+
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" CA"}}
258+
259+
event: content_block_delta
260+
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":":"}}
261+
262+
event: content_block_stop
263+
data: {"type":"content_block_stop","index":0}
264+
265+
event: content_block_start
266+
data: {"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"toolu_01T1x1fJ34qAmk2tNTrN7Up6","name":"get_weather","input":{}}}
267+
268+
event: content_block_delta
269+
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":""}}
270+
271+
event: content_block_delta
272+
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"{\"location\":"}}
273+
274+
event: content_block_delta
275+
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":" \"San"}}
276+
277+
event: content_block_delta
278+
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":" Francisc"}}
279+
280+
event: content_block_delta
281+
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"o,"}}
282+
283+
event: content_block_delta
284+
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":" CA\""}}
285+
286+
event: content_block_delta
287+
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":", "}}
288+
289+
event: content_block_delta
290+
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"\"unit\": \"fah"}}
291+
292+
event: content_block_delta
293+
data: {"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"renheit\"}"}}
294+
295+
event: content_block_stop
296+
data: {"type":"content_block_stop","index":1}
297+
298+
event: message_delta
299+
data: {"type":"message_delta","delta":{"stop_reason":"tool_use","stop_sequence":null},"usage":{"output_tokens":89}}
300+
301+
event: message_stop
302+
data: {"type":"message_stop"}
303+
```
304+
305+
#### Streaming Request with Extended Thinking
306+
307+
```bash
308+
curl https://api.anthropic.com/v1/messages \
309+
--header "x-api-key: $ANTHROPIC_API_KEY" \
310+
--header "anthropic-version: 2023-06-01" \
311+
--header "content-type: application/json" \
312+
--data \
313+
'{
314+
"model": "claude-opus-4-20250514",
315+
"max_tokens": 20000,
316+
"stream": true,
317+
"thinking": {
318+
"type": "enabled",
319+
"budget_tokens": 16000
320+
},
321+
"messages": [
322+
{
323+
"role": "user",
324+
"content": "What is 27 * 453?"
325+
}
326+
]
327+
}'
328+
```
329+
330+
**Response:**
331+
332+
```json
333+
event: message_start
334+
data: {"type": "message_start", "message": {"id": "msg_01...", "type": "message", "role": "assistant", "content": [], "model": "claude-opus-4-20250514", "stop_reason": null, "stop_sequence": null}}
335+
336+
event: content_block_start
337+
data: {"type": "content_block_start", "index": 0, "content_block": {"type": "thinking", "thinking": ""}}
338+
339+
event: content_block_delta
340+
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "thinking_delta", "thinking": "Let me solve this step by step:\n\n1. First break down 27 * 453"}}
341+
342+
event: content_block_delta
343+
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "thinking_delta", "thinking": "\n2. 453 = 400 + 50 + 3"}}
344+
345+
event: content_block_delta
346+
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "thinking_delta", "thinking": "\n3. 27 * 400 = 10,800"}}
347+
348+
event: content_block_delta
349+
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "thinking_delta", "thinking": "\n4. 27 * 50 = 1,350"}}
350+
351+
event: content_block_delta
352+
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "thinking_delta", "thinking": "\n5. 27 * 3 = 81"}}
353+
354+
event: content_block_delta
355+
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "thinking_delta", "thinking": "\n6. 10,800 + 1,350 + 81 = 12,231"}}
356+
357+
event: content_block_delta
358+
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "signature_delta", "signature": "EqQBCgIYAhIM1gbcDa9GJwZA2b3hGgxBdjrkzLoky3dl1pkiMOYds..."}}
359+
360+
event: content_block_stop
361+
data: {"type": "content_block_stop", "index": 0}
362+
363+
event: content_block_start
364+
data: {"type": "content_block_start", "index": 1, "content_block": {"type": "text", "text": ""}}
365+
366+
event: content_block_delta
367+
data: {"type": "content_block_delta", "index": 1, "delta": {"type": "text_delta", "text": "27 * 453 = 12,231"}}
368+
369+
event: content_block_stop
370+
data: {"type": "content_block_stop", "index": 1}
371+
372+
event: message_delta
373+
data: {"type": "message_delta", "delta": {"stop_reason": "end_turn", "stop_sequence": null}}
374+
375+
event: message_stop
376+
data: {"type": "message_stop"}
377+
```
378+
96379
### Count Message Tokens
97380

98381
Calculates the number of tokens for a given set of messages without creating it.

‎src/routes/messages/anthropic-types.ts‎

Lines changed: 32 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,11 @@ export interface AnthropicMessagesPayload {
1818
type: "auto" | "any" | "tool" | "none"
1919
name?: string
2020
}
21+
thinking?: {
22+
type: "enabled"
23+
budget_tokens?: number
24+
}
25+
service_tier?: "auto" | "standard_only"
2126
}
2227

2328
export interface AnthropicTextBlock {
@@ -48,6 +53,11 @@ export interface AnthropicToolUseBlock {
4853
input: Record<string, unknown>
4954
}
5055

56+
export interface AnthropicThinkingBlock {
57+
type: "thinking"
58+
thinking: string
59+
}
60+
5161
export type AnthropicUserContentBlock =
5262
| AnthropicTextBlock
5363
| AnthropicImageBlock
@@ -56,6 +66,7 @@ export type AnthropicUserContentBlock =
5666
export type AnthropicAssistantContentBlock =
5767
| AnthropicTextBlock
5868
| AnthropicToolUseBlock
69+
| AnthropicThinkingBlock
5970

6071
export interface AnthropicUserMessage {
6172
role: "user"
@@ -92,6 +103,9 @@ export interface AnthropicResponse {
92103
usage: {
93104
input_tokens: number
94105
output_tokens: number
106+
cache_creation_input_tokens?: number
107+
cache_read_input_tokens?: number
108+
service_tier?: "standard" | "priority" | "batch"
95109
}
96110
}
97111

@@ -118,6 +132,7 @@ export interface AnthropicContentBlockStartEvent {
118132
| (Omit<AnthropicToolUseBlock, "input"> & {
119133
input: Record<string, unknown>
120134
})
135+
| { type: "thinking"; thinking: string }
121136
}
122137

123138
export interface AnthropicContentBlockDeltaEvent {
@@ -126,6 +141,8 @@ export interface AnthropicContentBlockDeltaEvent {
126141
delta:
127142
| { type: "text_delta"; text: string }
128143
| { type: "input_json_delta"; partial_json: string }
144+
| { type: "thinking_delta"; thinking: string }
145+
| { type: "signature_delta"; signature: string }
129146
}
130147

131148
export interface AnthropicContentBlockStopEvent {
@@ -139,21 +156,34 @@ export interface AnthropicMessageDeltaEvent {
139156
stop_reason?: AnthropicResponse["stop_reason"]
140157
stop_sequence?: string | null
141158
}
142-
// OpenAI does not provide token usage per chunk, so this is omitted.
143-
// usage: { output_tokens: number }
159+
usage?: { output_tokens: number }
144160
}
145161

146162
export interface AnthropicMessageStopEvent {
147163
type: "message_stop"
148164
}
149165

166+
export interface AnthropicPingEvent {
167+
type: "ping"
168+
}
169+
170+
export interface AnthropicErrorEvent {
171+
type: "error"
172+
error: {
173+
type: string
174+
message: string
175+
}
176+
}
177+
150178
export type AnthropicStreamEventData =
151179
| AnthropicMessageStartEvent
152180
| AnthropicContentBlockStartEvent
153181
| AnthropicContentBlockDeltaEvent
154182
| AnthropicContentBlockStopEvent
155183
| AnthropicMessageDeltaEvent
156184
| AnthropicMessageStopEvent
185+
| AnthropicPingEvent
186+
| AnthropicErrorEvent
157187

158188
// State for streaming translation
159189
export interface AnthropicStreamState {

0 commit comments

Comments
 (0)