Plugin SDKs: express request→argv lowering (into_command) as a transpilable DSL

[WiggidyW]

Child of #232.

Summary

Today every CLI command leaf lowers its typed Request to argv with a hand-written Rust into_command(&self) -> Vec<String> (~355 leaves in objectiveai-sdk-rs/src/cli/command/**). #232 needs this same request→argv lowering in the Go, Python, and TypeScript SDKs. Re-implementing 355 hand-rolled functions per language is error-prone and will drift from the Rust source of truth.

Instead: express the lowering once, in a small transpilable DSL, and generate native into_command for every language (Rust included) from that single spec — folded into the existing JSON-schema-driven codegen.

This issue covers the lowering direction only (request → argv). The inverse from_args parsing is related (and could eventually reuse the same spec), but is out of scope here.

Findings from surveying all ~355 into_command impls

• Mostly mechanical. ~206 (58%) are trivial request_schema/response_schema leaves (subcommand path + base flags). ~100 (28%) are "flat" (fixed required --flag value, optional if let Some, conditional if bool, then base.push_flags). ~13% have one bespoke twist; ~5 are genuinely bespoke.
• Small, closed operation vocabulary. Every body is a sequence of: push subcommand-path tokens → push positional → push --flag value (required / optional) → push bare boolean flag if true → repeat a flag per list element → enum/variant dispatch → JSON-serialize a nested object into a flag value → number/duration/path stringification → delegate to a shared component's push_flags.
• The same Rust type lowers three different ways. Vec<String> appears as (a) repeated --target flags (agents/instances/list), (b) one --args flag carrying a JSON array string (tools/run, plugins/run), and (c) -- then splat as trailing positionals (tasks/schedule). → The lowering cannot be derived from the type alone; it must be declared per field.
• No existing abstraction. No derive/macro; each impl is standalone but pulls from a few shared push_flags(&mut Vec<String>) component helpers (RequestBase, AgentSelector, RequestMessage, FunctionSpec/ProfileSpec) and two key=value,… encoders (remote_path_to_arg_string, per-enum into_arg_string).
• Output is a Vec<String> — no shell quoting/escaping to model. That keeps the DSL purely structural.

The hard cases any DSL must cover

• tasks/schedule — Option<u64> selects between --interval <humantime> and a bare --oneshot; then base.push_flags; then a literal -- separator; then splat a nested Vec<String> as trailing positionals.
• agents/tags/apply — internally-tagged enum where each variant emits a different flag + payload kind, two arms with a nested optional --parent-… flag.
• viewer/send — two positionals: a bare string, then a JSON-serialized body object (JSON as a positional, not behind a flag).
• functions/execute FunctionSpec/ProfileSpec — a nested inline-or-remote enum where Remote → --function <key=value-string> and Inline → --function-inline <json> (same field → two flag names + two encodings).
• functions/execute aggregate Request enum — routes into_command to one of 6 inner variants (needs a "delegate to inner request" node).

DSL decision: invent a small declarative IR, not a general-purpose language

Recommendation: a purpose-built, data-only lowering spec (serializable JSON), consumed at codegen time to emit native into_command per language. Reasons:

• The vocabulary is tiny and closed (~12 node types + a few value transforms). A general language is overkill.
• "Easy to transpile" means generate native code, not interpret. A declarative IR transpiles directly (each node → a few lines of Go/Py/TS); a general language would force us to ship an interpreter in every SDK and run the lowering as a script at runtime.
• It folds into the existing JSON-schema codegen — keep the command types and their lowering in lockstep, one source of truth.
• Because the same type lowers ≥3 ways, we need an explicit per-field directive anyway — which is exactly what a declarative IR is.

Why not the obvious existing candidates:

• Starlark (already used in the expression system) — would require interpreting a script per SDK rather than transpiling, and pulls a Starlark engine into Go/Py/TS just for argv building. Keep it as a fallback only if we decide runtime interpretation + reuse of the existing expression engine is preferable to codegen.
• JMESPath (also used) — a data→data query language; it can't express conditional flags, per-element loops, or variant dispatch. Wrong tool.
• Pure type-derivation / serde-style attributes alone — defeated by the "same type, 3 lowerings" reality; still need an explicit directive.

Proposed IR vocabulary (starting point)

A per-command program = ordered list of emit steps:

• path [tokens] — literal subcommand prefix
• positional <value> — push a value as a positional
• flag <name> <value> — push --name value
• bare_flag <name> when <bool-field> — push --name iff true
• optional <field> { <emit> } — emit only if Some
• option_select <field> { some: <emit>, none: <emit> } — presence chooses (covers --interval/--oneshot)
• repeated <field> { <per-item emit> } — loop over a list
• match <field> { <variant> => <emit>, … } — enum dispatch (with nested optionals allowed in arms)
• delegate <component> — invoke a shared component's lowering (the push_flags reuse model)
• delegate_request — route to an inner Request variant (the aggregate router)
• splat_positionals after "--" <field> — trailing var-args

Value transforms (expressions on a field): string, to_string (numbers), duration_humantime, json (serialize object → string), json_array (string list → JSON array string), key_value_encode {fields} (the docker-style k=v,k=v encoder), path_lossy.

This set expresses all 355 leaves with no per-language escape hatch (possibly excepting an override slot for any future truly-bespoke leaf).

Plan

1. Define the IR schema (the node + transform vocabulary above), versioned alongside the command JSON schemas.
2. Author the lowering spec for the command tree (much of it auto-extractable from the existing Rust into_command patterns; bespoke leaves authored explicitly).
3. Add a codegen backend per target (Rust/Go/Py/TS) that emits native into_command from the IR. Rust regenerates to byte-identical (or behavior-identical) output as a correctness gate.
4. Golden tests: a corpus of (typed request) → expected argv cases run against every language's generated lowering to prove cross-language parity.

Open questions

• Do we keep hand-written Rust into_command and generate the other three, or generate Rust too (the strongest anti-drift guarantee)?
• Should the IR be designed to also drive the from_args parse direction later (note the deliberate emit/parse asymmetries, e.g. selectors that parse --agent <remote> but only ever emit --agent-inline <json>)?
• Where does the spec live — embedded in the JSON schema as an x-argv annotation per field, or a sibling spec file per command?

🤖 Generated with Claude Code

[WiggidyW]

Closing — we're going a different (much simpler) route instead of the transpilable lowering DSL.

The CLI now accepts a JSON request directly via a top-level --request <JSON> flag (mutually exclusive with any command path, enforced by clap's args_conflicts_with_subcommands). It deserializes the JSON straight into the aggregate CliCommandRequest (serde-tagged on path_type) and dispatches to the same execute() the argv path reaches — see objectiveai-sdk-rs/src/cli/command/command.rs (Command parser struct + TryFrom<Command> for Request) and objectiveai-cli/src/run.rs (commit 64109bb on branch 2-2-5).

Because every SDK already has serde serialization of the typed request, there is no per-language argv lowering to implement — the SDK executors just send objectiveai --request <json>. That obsoletes this whole DSL/into_command-port effort (no op-IR, no per-command program registry, no 355 hand-written lowerings), and also removes the argv length/escaping limits that motivated it.

Superseded by the --request work under #232's umbrella.