Add $ref support to all four language code generators#1062
Add $ref support to all four language code generators#1062stephentoub wants to merge 1 commit intomainfrom
Conversation
Enable JSON Schema $ref for type deduplication across all SDK code generators (TypeScript, Python, Go, C#). Changes: - utils.ts: Add resolveRef(), refTypeName(), collectDefinitions() helpers; normalize $defs to definitions in postProcessSchema - typescript.ts: Build combined schema with shared definitions and compile once via unreachableDefinitions, instead of per-method compilation - python.ts/go.ts: Include all definitions alongside SessionEvent for quicktype resolution; include shared API defs in RPC combined schema - csharp.ts: Add handling to resolveSessionPropertyType and resolveRpcType; generate classes for referenced types on demand Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the multi-language schema-to-SDK generators (TypeScript, Python, Go, C#) to support JSON Schema $ref so shared definitions can be deduplicated and emitted once per language.
Changes:
- Adds shared
$refutilities (resolveRef,refTypeName,collectDefinitions) and expandsApiSchemato surface schema definitions. - Refactors TypeScript RPC generation to compile a single combined schema (shared definitions + all RPC param/result types) in one pass.
- Updates Python/Go quicktype inputs and C# custom emitters to include/resolve referenced definitions.
Show a summary per file
| File | Description |
|---|---|
| scripts/codegen/utils.ts | Adds $ref helpers and attempts $defs→definitions normalization in schema post-processing. |
| scripts/codegen/typescript.ts | Generates RPC types from one combined schema to dedupe $ref-shared types; re-adds experimental annotations post-compile. |
| scripts/codegen/python.ts | Includes shared definitions for quicktype to resolve $ref during session-event and RPC generation. |
| scripts/codegen/go.ts | Adds $ref handling to Go session-events custom generator; includes shared definitions for RPC quicktype generation. |
| scripts/codegen/csharp.ts | Adds $ref handling for session events + RPC generation, with on-demand referenced type emission. |
Copilot's findings
Comments suppressed due to low confidence (1)
scripts/codegen/python.ts:23
- The import list includes
isRpcMethodtwice, which will cause a TypeScript compile error (“Duplicate identifier 'isRpcMethod'”). Remove the duplicate specifier.
import {
getApiSchemaPath,
getSessionEventsSchemaPath,
isRpcMethod,
postProcessSchema,
writeGeneratedFile,
collectDefinitions,
isRpcMethod,
isNodeFullyExperimental,
type ApiSchema,
type RpcMethod,
} from "./utils.js";
- Files reviewed: 5/5 changed files
- Comments generated: 7
| const defs = (schema.definitions ?? schema.$defs ?? {}) as Record<string, JSONSchema7Definition>; | ||
| return { ...defs } |
There was a problem hiding this comment.
collectDefinitions currently prefers schema.definitions over schema.$defs and ignores the other when both exist. To reliably support mixed-draft schemas, merge both maps (with a defined precedence on collisions) so referenced types aren’t dropped.
| const defs = (schema.definitions ?? schema.$defs ?? {}) as Record<string, JSONSchema7Definition>; | |
| return { ...defs } | |
| const legacyDefinitions = (schema.definitions ?? {}) as Record<string, JSONSchema7Definition>; | |
| const draft2019Definitions = (schema.$defs ?? {}) as Record<string, JSONSchema7Definition>; | |
| return { ...draft2019Definitions, ...legacyDefinitions }; |
| const combinedSchema: JSONSchema7 = { | ||
| $schema: "http://json-schema.org/draft-07/schema#", | ||
| type: "object", | ||
| definitions: { ...sharedDefs }, |
There was a problem hiding this comment.
The combined schema only populates definitions. If an input schema uses $defs and $ref: "#/$defs/...", json-schema-to-typescript won’t be able to resolve those pointers unless the combined schema also includes $defs (or refs are normalized). Consider emitting both definitions and $defs pointing at the shared map, or rewriting $ref paths during preprocessing.
| const combinedSchema: JSONSchema7 = { | |
| $schema: "http://json-schema.org/draft-07/schema#", | |
| type: "object", | |
| definitions: { ...sharedDefs }, | |
| const sharedDefinitions: Record<string, unknown> = { ...sharedDefs }; | |
| const combinedSchema: JSONSchema7 & { $defs: Record<string, unknown> } = { | |
| $schema: "http://json-schema.org/draft-07/schema#", | |
| type: "object", | |
| definitions: sharedDefinitions, | |
| $defs: sharedDefinitions, |
| const schemaWithDefs: JSONSchema7 = { | ||
| ...(typeof def === "object" ? (def as JSONSchema7) : {}), | ||
| definitions: combinedSchema.definitions, |
There was a problem hiding this comment.
During RPC generation, each per-type schema source is created with only definitions. If an API schema uses $defs and $ref: "#/$defs/...", quicktype may fail to resolve those references. Include a $defs field (or normalize refs) alongside definitions when building schemaWithDefs to cover both pointer styles.
| const schemaWithDefs: JSONSchema7 = { | |
| ...(typeof def === "object" ? (def as JSONSchema7) : {}), | |
| definitions: combinedSchema.definitions, | |
| const schemaWithDefs: JSONSchema7 & { $defs?: JSONSchema7["definitions"] } = { | |
| ...(typeof def === "object" ? (def as JSONSchema7) : {}), | |
| definitions: combinedSchema.definitions, | |
| $defs: combinedSchema.definitions, |
| // Build a combined schema for quicktype — prefix types to avoid conflicts. | ||
| // Include shared definitions from the API schema for $ref resolution. | ||
| const sharedDefs = collectDefinitions(schema as Record<string, unknown>); | ||
| const combinedSchema: JSONSchema7 = { | ||
| $schema: "http://json-schema.org/draft-07/schema#", | ||
| definitions: {}, | ||
| definitions: { ...sharedDefs }, | ||
| }; | ||
|
|
There was a problem hiding this comment.
The combined RPC schema only sets definitions. If the API schema’s $ref pointers use #/$defs/..., quicktype may not resolve them unless $defs is also present (or refs are normalized). Consider adding $defs: combinedSchema.definitions (or normalizing $ref) when building the combined schema and per-type sources.
See below for a potential fix:
const combinedDefinitions: Record<string, JSONSchema7Definition> = { ...sharedDefs };
const combinedSchema: JSONSchema7 & { $defs?: Record<string, JSONSchema7Definition> } = {
$schema: "http://json-schema.org/draft-07/schema#",
definitions: combinedDefinitions,
$defs: combinedDefinitions,
| if (!nestedClasses.has(className)) { | ||
| const refSchema = resolveRef(propSchema.$ref, sessionDefinitions); | ||
| if (refSchema) { | ||
| if (refSchema.enum && Array.isArray(refSchema.enum)) { | ||
| return getOrCreateEnum(className, "", refSchema.enum as string[], enumOutput); | ||
| } | ||
| nestedClasses.set(className, generateNestedClass(className, refSchema, knownTypes, nestedClasses, enumOutput)); | ||
| } | ||
| } | ||
| return isRequired ? className : `${className}?`; |
There was a problem hiding this comment.
For $ref to an enum schema, the code returns the enum type name directly, bypassing the isRequired nullability handling. This makes optional enum properties non-nullable. Apply ? when isRequired is false (and keep enum generation/dedup logic intact).
| if (!nestedClasses.has(className)) { | |
| const refSchema = resolveRef(propSchema.$ref, sessionDefinitions); | |
| if (refSchema) { | |
| if (refSchema.enum && Array.isArray(refSchema.enum)) { | |
| return getOrCreateEnum(className, "", refSchema.enum as string[], enumOutput); | |
| } | |
| nestedClasses.set(className, generateNestedClass(className, refSchema, knownTypes, nestedClasses, enumOutput)); | |
| } | |
| } | |
| return isRequired ? className : `${className}?`; | |
| let resolvedTypeName = className; | |
| if (!nestedClasses.has(className)) { | |
| const refSchema = resolveRef(propSchema.$ref, sessionDefinitions); | |
| if (refSchema) { | |
| if (refSchema.enum && Array.isArray(refSchema.enum)) { | |
| resolvedTypeName = getOrCreateEnum(className, "", refSchema.enum as string[], enumOutput); | |
| } else { | |
| nestedClasses.set(className, generateNestedClass(className, refSchema, knownTypes, nestedClasses, enumOutput)); | |
| } | |
| } | |
| } | |
| return isRequired ? resolvedTypeName : `${resolvedTypeName}?`; |
| function generateSessionEventsCode(schema: JSONSchema7): string { | ||
| generatedEnums.clear(); | ||
| sessionDefinitions = schema.definitions as Record<string, JSONSchema7Definition> || {}; | ||
| const variants = extractEventVariants(schema); |
There was a problem hiding this comment.
sessionDefinitions is populated from schema.definitions only. If the session-events schema uses $defs (or if $ref paths point to #/$defs/...), $ref resolution in resolveSessionPropertyType will fail. Populate sessionDefinitions via collectDefinitions(...) (or preprocess the schema to normalize $defs → definitions).
See below for a potential fix:
sessionDefinitions = collectDefinitions(schema);
const variants = extractEventVariants(schema);
const knownTypes = new Map<string, string>();
const nestedClasses = new Map<string, string>();
const enumOutput: string[] = [];
// Extract descriptions for base class properties from the first variant
const sessionEventDefinition = sessionDefinitions.SessionEvent;
const firstVariant = typeof sessionEventDefinition === "object" ? (sessionEventDefinition as JSONSchema7)?.anyOf?.[0] : undefined;
| // Normalize $defs → definitions for draft 2019+ compatibility | ||
| if ("$defs" in processed && !processed.definitions) { | ||
| processed.definitions = (processed as Record<string, unknown>).$defs as Record< | ||
| string, | ||
| JSONSchema7Definition | ||
| >; | ||
| delete (processed as Record<string, unknown>).$defs; | ||
| } |
There was a problem hiding this comment.
postProcessSchema converts $defs to definitions but does not update existing $ref pointers (e.g. #/$defs/Foo). If an input schema uses $defs-style refs, codegen will still try to resolve via $defs and fail after $defs is deleted. Either keep $defs alongside definitions, or rewrite $ref strings during post-processing.
Summary
Enable JSON Schema
$reffor type deduplication across all SDK code generators (TypeScript, Python, Go, C#). This allows the runtime to declare a shared type once using$ref and have each generator produce a single deduplicated type per language.Changes
scripts/codegen/utils.tsresolveRef(),refTypeName(),collectDefinitions()helpers$defs todefinitionsinpostProcessSchemafor cross-draft compatibilitydefinitions/$defs fields toApiSchemainterfacescripts/codegen/typescript.tsunreachableDefinitions, instead of per-method compilation@experimentalJSDoc annotations via post-processingscripts/codegen/python.tsSessionEventfor quicktype$ref resolutionaddSourcecallscripts/codegen/go.ts$ref handling to the custom Go session events generator (resolveGoPropertyType) withdefinitionsonGoCodegenCtx$ref resolutionscripts/codegen/csharp.ts$ref handling toresolveSessionPropertyTypeandresolveRpcTypesessionDefinitions/rpcDefinitionsstate$ref in array itemsMotivation
The copilot-agent-runtime is moving to use
$ref for type deduplication in tool schemas. Without this change, generators would either fail to resolve$ref pointers or inline duplicate type definitions.