CopilotKit/showcase/shell-docs/src/content/docs/backend/custom-agent.mdx at main · samuelson-chen/CopilotKit

title	Use any model router
icon	lucide/Blocks
description	Use BuiltInAgent's factory mode to bring your own AI SDK, TanStack AI, or custom LLM backend.

BuiltInAgent's factory mode gives you full control over the LLM call. You provide a factory function that talks to any backend — CopilotKit handles converting the stream to AG-UI events, managing lifecycle, and wiring it into the runtime.

When to use Simple Mode vs Factory Mode

	Simple Mode	Factory Mode
Setup	Minimal — pass a model string	You own the LLM call and stream
Model resolution	Built-in (`"openai/gpt-4o"`)	You set up the model yourself
Tools, MCP, state tools	Automatically wired	You wire them in your factory
Backend support	Vercel AI SDK only	Any backend: AI SDK, TanStack AI, or custom
Best for	Quick setup, standard use cases	Full control, non-standard backends

If simple mode covers your needs, stick with it — it's simpler. Use factory mode when you need control that simple mode doesn't offer.

Quick Start

You have an existing LLM backend and you want a CopilotKit copilot using it. Pick your backend:

import {
  CopilotRuntime,
  createCopilotEndpoint,
  InMemoryAgentRunner,
  BuiltInAgent,
  convertMessagesToVercelAISDKMessages,
} from "@copilotkit/runtime/v2";
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";

const agent = new BuiltInAgent({
  type: "aisdk",
  factory: ({ input, abortSignal }) =>
    streamText({
      model: openai("gpt-4o"),
      messages: convertMessagesToVercelAISDKMessages(input.messages),
      abortSignal,
    }),
});

const runtime = new CopilotRuntime({
  agents: { default: agent },
  runner: new InMemoryAgentRunner(),
});

const copilotEndpoint = createCopilotEndpoint({
  runtime,
  basePath: "/api/copilotkit",
});
export default copilotEndpoint;

```typescript title="src/copilotkit.ts" import { CopilotRuntime, createCopilotEndpoint, InMemoryAgentRunner, BuiltInAgent, convertInputToTanStackAI, } from "@copilotkit/runtime/v2"; import { chat } from "@tanstack/ai"; import { openaiText } from "@tanstack/ai-openai";

const runtime = new CopilotRuntime({ agents: { default: agent }, runner: new InMemoryAgentRunner(), });

const copilotEndpoint = createCopilotEndpoint({ runtime, basePath: "/api/copilotkit", }); export default copilotEndpoint;

</Tab>
<Tab value="Custom">
```typescript title="src/copilotkit.ts"
import {
  CopilotRuntime,
  createCopilotEndpoint,
  InMemoryAgentRunner,
  BuiltInAgent,
} from "@copilotkit/runtime/v2";
import { EventType, type BaseEvent } from "@ag-ui/client";

const agent = new BuiltInAgent({
  type: "custom",
  factory: async function* ({ input, abortSignal }) {
    const response = await fetch("https://your-llm-api.com/chat", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ messages: input.messages }),
      signal: abortSignal,
    });

    const reader = response.body!.getReader();
    const decoder = new TextDecoder();
    const messageId = crypto.randomUUID();

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      yield {
        type: EventType.TEXT_MESSAGE_CHUNK,
        role: "assistant",
        messageId,
        delta: decoder.decode(value),
      } as BaseEvent;
    }
  },
});

const runtime = new CopilotRuntime({
  agents: { default: agent },
  runner: new InMemoryAgentRunner(),
});

const copilotEndpoint = createCopilotEndpoint({
  runtime,
  basePath: "/api/copilotkit",
});
export default copilotEndpoint;

The frontend setup is the same as BuiltInAgent — wrap your app with <CopilotKit> and add a chat component.

How It Works

Factory mode accepts a config with two fields:

type — which backend you're using: "aisdk", "tanstack", or "custom"
factory — a function that receives the raw request and returns a backend-native stream

The factory receives an AgentFactoryContext (from @copilotkit/runtime/v2):

interface AgentFactoryContext {
  input: RunAgentInput;        // messages, tools, state, context, threadId, runId, forwardedProps
  abortController: AbortController;  // for TanStack AI (requires AbortController)
  abortSignal: AbortSignal;          // preferred for AI SDK, fetch, and custom backends
}

CopilotKit handles everything else: RUN_STARTED and RUN_FINISHED lifecycle events, stream-to-AG-UI conversion, error handling, and abort/cancellation. Your factory never needs to emit lifecycle events.

The factory can be async — return a Promise if you need to do setup before streaming:

factory: async ({ input, abortSignal }) => {
  const apiKey = await getApiKeyFromVault();
  return streamText({ model: openai("gpt-4o", { apiKey }), ... });
}

Examples

With Tools

import {
  BuiltInAgent,
  convertMessagesToVercelAISDKMessages,
  convertToolsToVercelAITools,
} from "@copilotkit/runtime/v2";
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";

const agent = new BuiltInAgent({
  type: "aisdk",
  factory: ({ input, abortSignal }) => {
    const tools = convertToolsToVercelAITools(input.tools);
    return streamText({
      model: openai("gpt-4o"),
      messages: convertMessagesToVercelAISDKMessages(input.messages),
      tools,
      abortSignal,
    });
  },
});

convertToolsToVercelAITools converts the frontend-defined tools (from useFrontendTool) into AI SDK's ToolSet format automatically.

import { BuiltInAgent, convertInputToTanStackAI } from "@copilotkit/runtime/v2";
import { chat, toolDefinition } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";
import { z } from "zod";

const getWeather = toolDefinition({
  name: "getWeather",
  description: "Get the weather for a city",
  inputSchema: z.object({ city: z.string() }),
}).server(async ({ city }) => ({ temp: 72, city }));

const agent = new BuiltInAgent({
  type: "tanstack",
  factory: ({ input, abortController }) => {
    const { messages, systemPrompts } = convertInputToTanStackAI(input);
    return chat({
      adapter: openaiText("gpt-4o"),
      messages,
      systemPrompts,
      tools: [getWeather],
      abortController,
    });
  },
});

```typescript title="src/copilotkit.ts" import { BuiltInAgent } from "@copilotkit/runtime/v2"; import { EventType, type BaseEvent } from "@ag-ui/client";

const agent = new BuiltInAgent({ type: "custom", factory: async function* ({ input }) { const messageId = crypto.randomUUID(); const toolCallId = crypto.randomUUID();

// The LLM decides to call a tool
yield {
  type: EventType.TOOL_CALL_START,
  parentMessageId: messageId,
  toolCallId,
  toolCallName: "getWeather",
} as BaseEvent;

yield {
  type: EventType.TOOL_CALL_ARGS,
  toolCallId,
  delta: JSON.stringify({ city: "San Francisco" }),
} as BaseEvent;

yield {
  type: EventType.TOOL_CALL_END,
  toolCallId,
} as BaseEvent;

// Execute the tool and return the result
yield {
  type: EventType.TOOL_CALL_RESULT,
  role: "tool",
  messageId: crypto.randomUUID(),
  toolCallId,
  content: JSON.stringify({ temp: 72, city: "San Francisco" }),
} as BaseEvent;

// Text response after the tool call
yield {
  type: EventType.TEXT_MESSAGE_CHUNK,
  role: "assistant",
  messageId,
  delta: "The weather in San Francisco is 72°F.",
} as BaseEvent;

}, });


With `type: "custom"`, you yield AG-UI events directly. See the [AG-UI event reference](/backend/ag-ui) for all available event types.
</Tab>
</Tabs>

### With Reasoning (Thinking Models)

<Tabs items={["AI SDK", "TanStack AI"]}>
<Tab value="AI SDK">
```typescript title="src/copilotkit.ts"
import { BuiltInAgent, convertMessagesToVercelAISDKMessages } from "@copilotkit/runtime/v2";
import { streamText } from "ai";
import { anthropic } from "@ai-sdk/anthropic";

const agent = new BuiltInAgent({
  type: "aisdk",
  factory: ({ input, abortSignal }) =>
    streamText({
      model: anthropic("claude-sonnet-4", {
        thinking: { type: "enabled", budgetTokens: 10000 },
      }),
      messages: convertMessagesToVercelAISDKMessages(input.messages),
      abortSignal,
    }),
});

Reasoning events (REASONING_START, REASONING_MESSAGE_CONTENT, REASONING_END) are automatically extracted from the AI SDK stream. The TanStack AI converter does not surface reasoning events (REASONING_START, REASONING_MESSAGE_CONTENT, REASONING_END). Even if the underlying model supports thinking/reasoning, those events will not be forwarded to the frontend. Use the AI SDK backend if you need reasoning events.

import { BuiltInAgent, convertInputToTanStackAI } from "@copilotkit/runtime/v2";
import { chat } from "@tanstack/ai";
import { anthropicText } from "@tanstack/ai-anthropic";

const agent = new BuiltInAgent({
  type: "tanstack",
  factory: ({ input, abortController }) => {
    const { messages, systemPrompts } = convertInputToTanStackAI(input);
    return chat({
      adapter: anthropicText("claude-sonnet-4"),
      messages,
      systemPrompts,
      modelOptions: { thinking: { type: "enabled", budgetTokens: 10000 } },
      abortController,
    });
  },
});

With System Prompt, Context, and State

import {
  BuiltInAgent,
  convertMessagesToVercelAISDKMessages,
} from "@copilotkit/runtime/v2";
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";

const agent = new BuiltInAgent({
  type: "aisdk",
  factory: ({ input, abortSignal }) => {
    const systemParts: string[] = ["You are a helpful assistant."];

    // Add context from the frontend (useAgentContext)
    if (input.context?.length) {
      for (const ctx of input.context) {
        systemParts.push(`${ctx.description}:\n${ctx.value}`);
      }
    }

    // Add shared application state (useCoAgent, etc.)
    if (input.state && Object.keys(input.state).length > 0) {
      systemParts.push(
        `Application State:\n${JSON.stringify(input.state, null, 2)}`,
      );
    }

    const messages = convertMessagesToVercelAISDKMessages(input.messages);
    messages.unshift({ role: "system", content: systemParts.join("\n\n") });

    return streamText({
      model: openai("gpt-4o"),
      messages,
      abortSignal,
    });
  },
});

```typescript title="src/copilotkit.ts" import { BuiltInAgent, convertInputToTanStackAI } from "@copilotkit/runtime/v2"; import { chat } from "@tanstack/ai"; import { openaiText } from "@tanstack/ai-openai";

const agent = new BuiltInAgent({ type: "tanstack", factory: ({ input, abortController }) => { // convertInputToTanStackAI automatically extracts system/developer messages, // context, and state into the systemPrompts array const { messages, systemPrompts } = convertInputToTanStackAI(input);

// Add your own system prompt at the beginning
systemPrompts.unshift("You are a helpful assistant.");

return chat({
  adapter: openaiText("gpt-4o"),
  messages,
  systemPrompts,
  abortController,
});

}, });


`convertInputToTanStackAI` handles system/developer messages, `input.context`, and `input.state` automatically. Prepend your own prompt if needed.
</Tab>
</Tabs>

### With forwardedProps

Let the frontend override model, temperature, or other settings at runtime:

<Tabs items={["AI SDK", "TanStack AI"]}>
<Tab value="AI SDK">
```typescript title="src/copilotkit.ts"
import {
  BuiltInAgent,
  convertMessagesToVercelAISDKMessages,
  resolveModel,
} from "@copilotkit/runtime/v2";
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";

const agent = new BuiltInAgent({
  type: "aisdk",
  factory: ({ input, abortSignal }) => {
    const props = (input.forwardedProps ?? {}) as Record<string, unknown>;

    const model =
      typeof props.model === "string"
        ? resolveModel(props.model)
        : openai("gpt-4o");

    const temperature =
      typeof props.temperature === "number" ? props.temperature : 0.7;

    return streamText({
      model,
      temperature,
      messages: convertMessagesToVercelAISDKMessages(input.messages),
      abortSignal,
    });
  },
});

const agent = new BuiltInAgent({ type: "tanstack", factory: ({ input, abortController }) => { const props = (input.forwardedProps ?? {}) as Record<string, unknown>; const { messages, systemPrompts } = convertInputToTanStackAI(input);

const adapter =
  props.model === "anthropic/claude-sonnet-4"
    ? anthropicText("claude-sonnet-4")
    : openaiText((props.model as string) ?? "gpt-4o");

const modelOptions: Record<string, unknown> = {};
if (typeof props.temperature === "number")
  modelOptions.temperature = props.temperature;

return chat({
  adapter,
  messages,
  systemPrompts,
  modelOptions,
  abortController,
});

}, });

</Tab>
</Tabs>

Forward props from the frontend using the `CopilotKit` provider's `properties` prop:

```tsx title="app/page.tsx"
<CopilotKit properties={{ model: "anthropic/claude-sonnet-4", temperature: 0.3 }}>
  <CopilotChat />
</CopilotKit>

Helper Utilities

These utilities are exported from @copilotkit/runtime/v2 to help convert between CopilotKit's input format and your backend's expected format:

Utility	Description
`convertInputToTanStackAI(input)`	Converts `RunAgentInput` to `{ messages, systemPrompts }` for TanStack AI's `chat()`. Handles system/developer messages, context, and state.
`convertMessagesToVercelAISDKMessages(messages)`	Converts AG-UI messages to Vercel AI SDK's `ModelMessage[]` format.
`convertToolsToVercelAITools(tools)`	Converts frontend-defined tools (JSON Schema) to AI SDK's `ToolSet`.
`convertToolDefinitionsToVercelAITools(tools)`	Converts `defineTool()` definitions (Standard Schema) to AI SDK's `ToolSet`.
`resolveModel(spec)`	Resolves `"openai/gpt-4o"` strings to AI SDK `LanguageModel` instances.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

When to use Simple Mode vs Factory Mode

Quick Start

How It Works

Examples

With Tools

With System Prompt, Context, and State

Helper Utilities

FilesExpand file tree

custom-agent.mdx

Latest commit

History

custom-agent.mdx

File metadata and controls

When to use Simple Mode vs Factory Mode

Quick Start

How It Works

Examples

With Tools

With System Prompt, Context, and State

Helper Utilities