--- title: CopilotRuntime description: "Server-side runtime for hosting CopilotKit agents" --- The `CopilotRuntime` package gives you everything you need to host CopilotKit agents on your own infrastructure. It owns the agent registry, wires an `AgentRunner`, exposes HTTP endpoints (Express or Hono), and optionally coordinates middleware and transcription services. ## Basic Setup ```ts import { CopilotRuntime, InMemoryAgentRunner } from "@copilotkit/runtime"; import { BasicAgent } from "@copilotkit/runtime/v2"; const runtime = new CopilotRuntime({ agents: { default: new BasicAgent({ model: "openai/gpt-4o", prompt: "You are a helpful assistant.", }), }, runner: new InMemoryAgentRunner(), }); ``` Once you create a runtime instance you can mount one of the provided HTTP endpoints (Express or Hono) described below. ## `CopilotRuntime` Constructor ```ts new CopilotRuntime(options: CopilotRuntimeOptions) ``` ### `CopilotRuntimeOptions` | Option | Type | Description | | ------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `agents` | `Promise> \| Record` | Required. A map from agent identifier to an `AbstractAgent`. You can return the map synchronously or asynchronously. | | `runner` | `AgentRunner` | Optional. Defaults to `new InMemoryAgentRunner()`. Controls how agent runs are scheduled and streamed. | | `transcriptionService` | `TranscriptionService` | Optional. Enables `/transcribe` and reports `audioFileTranscriptionEnabled: true` from `/info`. | | `beforeRequestMiddleware` | `BeforeRequestMiddleware` | Optional. Runs before every request. Can mutate the incoming `Request` or return `void`. | | `afterRequestMiddleware` | `AfterRequestMiddleware` | Optional. Runs after every handler resolves. Receives the response along with parsed messages, thread ID, and run ID extracted from the SSE stream. Use this for logging, metrics, telemetry, etc. | ### Passing Agents - Provide a plain object mapping agent ids to instances. The helper `BasicAgent` covers common OpenAI/Anthropic/Gemini setups, but you can supply any `AbstractAgent`. - Because `agents` may be an async function, you can lazily load credentials or fetch configuration before returning the record. - Each request clones the selected agent instance so per-request state (messages, headers, tool registration) does not leak between threads. - `Authorization` and `x-*` headers on the incoming request are forwarded to the cloned agent automatically. Override or strip them inside `beforeRequestMiddleware` if needed. ```ts const runtime = new CopilotRuntime({ agents: async () => ({ default: buildDefaultAgent(), support: buildSupportAgent(), }), }); ``` ### Choosing an `AgentRunner` `InMemoryAgentRunner` is the default and streams events directly from your Node.js process. Swap it with a custom `AgentRunner` if you need to enqueue work or forward to another service: ```ts class QueueBackedRunner implements AgentRunner { /* ... */ } const runtime = new CopilotRuntime({ agents, runner: new QueueBackedRunner(), }); ``` ### Middleware Hooks ```ts const runtime = new CopilotRuntime({ agents, beforeRequestMiddleware: async ({ request, path }) => { const auth = request.headers.get("authorization"); if (!auth) { throw new Response("Unauthorized", { status: 401 }); } }, afterRequestMiddleware: async ({ response, path, messages, threadId, runId, }) => { console.info("Handled", path, response.status); console.info("Thread:", threadId, "Run:", runId); console.info("Messages:", messages); }, }); ``` Throwing a `Response` from `beforeRequestMiddleware` short-circuits the request. `afterRequestMiddleware` runs in the background and should swallow its own errors. #### `AfterRequestMiddlewareParameters` | Parameter | Type | Description | | ---------- | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `runtime` | `CopilotRuntime` | The runtime instance. | | `response` | `Response` | A clone of the response sent to the client. The body is still readable. | | `path` | `string` | The matched route path (e.g. `/agent/default/run`). | | `messages` | `Message[]` | Reconstructed messages from the SSE stream. Empty for non-SSE responses. Each message has `id`, `role`, and optionally `content`, `toolCalls`, or `toolCallId`. | | `threadId` | `string \| undefined` | Thread ID from the `RUN_STARTED` event. | | `runId` | `string \| undefined` | Run ID from the `RUN_STARTED` event. | This makes `afterRequestMiddleware` suitable for telemetry, audit logging, and post-run analytics without needing to manually parse the streamed response. ### Audio Transcription Enable audio transcription by providing a `transcriptionService`. The `@copilotkit/voice` package includes providers: ```ts import { CopilotRuntime } from "@copilotkit/runtime"; import { TranscriptionServiceOpenAI } from "@copilotkit/voice"; import OpenAI from "openai"; const runtime = new CopilotRuntime({ agents: { default: yourAgent }, transcriptionService: new TranscriptionServiceOpenAI({ openai: new OpenAI({ apiKey: process.env.OPENAI_API_KEY }), }), }); ``` This enables the `/transcribe` endpoint and shows a microphone button in the chat UI. The `/info` endpoint will report `audioFileTranscriptionEnabled: true`. For custom providers, extend `TranscriptionService` from runtime: ```ts import { TranscriptionService, TranscribeFileOptions, } from "@copilotkit/runtime"; class MyTranscriptionService extends TranscriptionService { async transcribeFile(options: TranscribeFileOptions): Promise { // options.audioFile, options.mimeType, options.size return "transcribed text"; } } ``` ## HTTP Endpoints Import the helper that matches your server framework and the transport style you want to expose. ### Hono (REST-style) ```ts import { createCopilotEndpoint } from "@copilotkit/runtime"; const copilot = createCopilotEndpoint({ runtime, basePath: "/api/copilotkit" }); const app = new Hono(); app.route("/", copilot); ``` This mounts five routes under the base path: | Method | Path | Purpose | | ------ | -------------------------------- | -------------------------------------------------------------- | | `POST` | `/agent/:agentId/run` | Streams agent events (SSE). | | `POST` | `/agent/:agentId/connect` | Creates a live WebSocket/stream connection. | | `POST` | `/agent/:agentId/stop/:threadId` | Cancels an in-flight thread. | | `GET` | `/info` | Returns runtime metadata and agent list. | | `POST` | `/transcribe` | Proxies audio transcription (requires `transcriptionService`). | ### Hono (Single-route) ```ts import { createCopilotEndpointSingleRoute } from "@copilotkit/runtime"; const copilot = createCopilotEndpointSingleRoute({ runtime, basePath: "/api/copilotkit", }); const app = new Hono(); app.route("/", copilot); ``` All interactions happen through a single `POST /api/copilotkit` endpoint. The client sends a JSON envelope: ```json { "method": "agent/run", "params": { "agentId": "default" }, "body": { "messages": [...], "threadId": "thread-123" } } ``` Allowed `method` values are: - `agent/run` - `agent/connect` - `agent/stop` - `info` - `transcribe` Pair this server endpoint with the React provider flag ``. ### Next.js (App Router) with Hono When you're inside a Next.js `app/` route, export the Hono handlers directly: ```ts // app/api/copilotkit/[[...slug]]/route.ts import { CopilotRuntime, createCopilotEndpoint } from "@copilotkit/runtime"; import { handle } from "hono/vercel"; const runtime = new CopilotRuntime({ agents, runner }); const app = createCopilotEndpoint({ runtime, basePath: "/api/copilotkit" }); export const GET = handle(app); export const POST = handle(app); ``` Swap `createCopilotEndpoint` for `createCopilotEndpointSingleRoute` if you configure the client with `useSingleEndpoint`. ### Express (REST-style) ```ts import express from "express"; import { createCopilotEndpointExpress } from "@copilotkit/runtime/express"; const app = express(); app.use( "/api/copilotkit", createCopilotEndpointExpress({ runtime, basePath: "/" }), ); ``` The router mirrors the Hono REST routes and automatically enables permissive CORS (allowing all origins, headers, and methods). Adjust headers upstream if you need tighter controls. ### Express (Single-route) ```ts import express from "express"; import { createCopilotEndpointSingleRouteExpress } from "@copilotkit/runtime/express"; const app = express(); app.use( "/api/copilotkit", createCopilotEndpointSingleRouteExpress({ runtime, basePath: "/" }), ); ``` Single-route Express works identically to the Hono version: send the JSON envelope described earlier, and set `useSingleEndpoint` on the client. ### NestJS (Express backend) NestJS uses Express by default. Mount the Express router in `main.ts`: ```ts // main.ts import { NestFactory } from "@nestjs/core"; import { AppModule } from "./app.module"; import { NestExpressApplication } from "@nestjs/platform-express"; import { CopilotRuntime } from "@copilotkit/runtime"; import { createCopilotEndpointExpress } from "@copilotkit/runtime/express"; import { BasicAgent } from "@copilotkit/runtime/v2"; async function bootstrap() { const app = await NestFactory.create(AppModule); const runtime = new CopilotRuntime({ agents: { default: new BasicAgent({ model: "openai/gpt-4o", prompt: "You are helpful.", }), }, }); // Mount under /api/copilotkit (REST-style routes) app.use( "/api/copilotkit", createCopilotEndpointExpress({ runtime, basePath: "/" }), ); await app.listen(3000); } bootstrap(); ``` Available routes (no global prefix): - `POST /api/copilotkit/agent/:agentId/run` - `POST /api/copilotkit/agent/:agentId/connect` - `POST /api/copilotkit/agent/:agentId/stop/:threadId` - `GET /api/copilotkit/info` - `POST /api/copilotkit/transcribe` If you prefer the single-route transport, mount the single-route helper instead and set `useSingleEndpoint` on the client: ```ts import { createCopilotEndpointSingleRouteExpress } from "@copilotkit/runtime/express"; app.use( "/api/copilotkit", createCopilotEndpointSingleRouteExpress({ runtime, basePath: "/" }), ); ``` Notes: - If you call `app.setGlobalPrefix('api')`, the effective paths become `/api/copilotkit/...` (or `/api/copilotkit` for single-route). - Endpoints stream Server‑Sent Events; ensure your proxy honors `text/event-stream` and keep‑alive. - The router enables permissive CORS; tighten via Nest’s `enableCors` if needed. ## Runtime Metadata Calling `GET /info` (or the `info` single-route method) returns: ```json { "version": "0.0.20", "agents": { "default": { "name": "default", "className": "BasicAgent", "description": "You are a helpful assistant." } }, "audioFileTranscriptionEnabled": false } ``` Use this to verify deployments and surface diagnostics in your UI. ## Next Steps - Configure the React client with `runtimeUrl` (and `useSingleEndpoint` if you chose the single endpoint helper). - Register frontend tools with `CopilotKitProvider` so agents can trigger UI actions. - Extend the runtime runner or middleware hooks to integrate logging, rate limiting, or background queues.