Add universal_macro_translation + encode_unreserved (pinned to a shared golden fixture)

[bokelley]

Add universal_macro_translation + encode_unreserved to the Python SDK, pinned to a shared golden fixture

The JS SDK is adding a producer-side universal-macro translation helper for seller agents (adcontextprotocol/adcp-client#2263; spec guide adcontextprotocol/adcp#5646). Every seller agent needs this behavior, and the Python adcp package ships the full seller side (serve(), ADCPAgentExecutor, builders, examples/seller_agent.py) — but has no translation helper and no RFC-3986 unreserved encoder. A Python seller reading the new guide gets a TypeScript function it cannot call.

Why this matters (and why parity alone isn't enough)

The seller's pixel translation is an internal step — it is never serialized on any AdCP wire surface. So no conformance storyboard can observe it; the JS expect_universal_macro_substituted assertion grades a different, observable thing (the macro_values → output-manifest contract). That means the only enforcement of helper correctness is unit tests — and the JS conformance check only exercises alphanumeric IDs, which encode identically under every encoder. A Python port that uses urllib.parse.quote() defaults would silently drift from JS on every reserved character (quote('a/b') == 'a/b' vs JS a%2Fb) and nothing would catch it.

What to build

1. adcp.substitution.encode_unreserved(raw) -> str — port the JS encoder byte-for-byte: explicit unreserved whitelist (ALPHA / DIGIT / - . _ ~), UTF-8 encode then uppercase %NN. Do NOT use urllib.parse.quote defaults (safe='/' leaves / unescaped; even safe='' relies on CPython's unreserved set, not the contract's).
2. adcp.substitution.universal_macro_translation(pixel_url, mapping) -> {url, dropped_params, unmapped_macros} — mirror @adcp/sdk's substitution/translate.ts:
   • { "value": v } → substituted now, encode_unreserved'd
   • { "native": t } → inserted raw (ad-server token survives)
   • unmapped-macro param → dropped, reported in dropped_params / unmapped_macros
   • already-minted (no-macro) param → untouched
   • single-pass: a {MACRO} inside a substituted value is data, not re-expanded
   • query string only (path/fragment left raw — documented limitation)

The contract pin (do this regardless)

Land a language-neutral golden fixture that BOTH SDKs validate against — draft attached in .context/universal-macro-translation.vectors.json, proposed home static/test-vectors/universal-macro-translation.json in the adcp repo. The JS PR's substitution-translate.test.js cases seed it. Without this, the two encoders will drift and conformance won't catch it.

Scope

• Helper only — NOT the assertion. There is no Python storyboard runner; the JS @adcp/sdk runner grades Python sellers over the wire. Don't port expect_universal_macro_substituted.
• Natural home: a new adcp.substitution module (neighbors canonical_formats/pixel_tracker.py, which already handles pixel/tracker projection).
• No dependency on #2263 publishing — the helper + fixture are independent and can land in parallel.

Guardrails to fold into the helper/docstring

• Only build-time IDs ({MEDIA_BUY_ID}, {PACKAGE_ID}, {CREATIVE_ID}) and catalog data macros take value; every impression-time macro (privacy/consent {GDPR_CONSENT}/{US_PRIVACY}/{GPP_STRING}, device, geo, {CACHEBUSTER}, {TIMESTAMP}) MUST be native — a value mapping freezes per-impression data into every impression.
• native is inserted raw with no validation → document the trust boundary (must come from trusted ad-server integration config, not buyer input).

Refs: adcontextprotocol/adcp-client#2263 (JS helper + assertion), adcontextprotocol/adcp#5646 (spec guide + storyboard).

[bokelley]

Triage

Classification: Feature request
Bucket(s): none of the gated bucket labels (client/handlers/signing/validation/middleware/examples/docs/cross-repo) exist in this repo's label set — closest conceptual fit is core SDK surface (adcp.substitution, neighboring canonical_formats/pixel_tracker.py) plus a cross-repo fixture dependency. Applying enhancement only.
Status: ready-for-human

What the experts said:

• code-reviewer: shape is a clean, non-breaking addition, but "byte-for-byte" parity is unverifiable without the JS source or fixture — recommends splitting encode_unreserved (implement now, pure RFC 3986, zero ambiguity) from universal_macro_translation (block on fixture/JS reference).
• dx-expert: return type should be a dataclass matching the pixel_tracker.py sibling pattern, not a bare dict; the privacy-macro guardrail needs to be a structured, returned advisory (or raise), not a docstring/warning that's easy to silence.
• security-reviewer: flags this as security-sensitive — native values are inserted raw into URLs rendered across browsers/CTV by design with "no validation," and a value-mapped privacy macro (GDPR_CONSENT/US_PRIVACY/GPP_STRING) freezes one user's consent/device/geo data into a pixel reused across every subsequent impression. Both need fail-closed enforcement, consistent with this repo's existing scheme-allowlist precedent in pixel_tracker.py, not documentation alone.
• ad-tech-protocol-expert: design is spec-consistent with the AdCP universal-macro enum already vendored in this repo, but the golden fixture this issue is pinned to (.context/universal-macro-translation.vectors.json) isn't recoverable from anywhere I have access to — implementing the translator against prose alone risks silent JS drift.

My take: Solid, spec-aligned design, but two independent reasons this isn't a routine hand-off: the security/privacy guardrails (native passthrough, build-time-vs-impression-time macro misuse) need fail-closed code rather than docs per existing repo precedent, and the actual cross-SDK fixture this is pinned to doesn't exist anywhere I can read. Splitting encode_unreserved (no external dependency) from universal_macro_translation (needs the fixture) looks like the cleanest path.

Open questions for @bokelley:

1. Can the .context/universal-macro-translation.vectors.json draft be recovered/re-attached, or pushed to adcontextprotocol/adcp so Python and JS can actually be checked against the same vectors?
2. Should universal_macro_translation fail-closed on native (reject control chars/CRLF and dangerous schemes) and raise — not just warn — when a known privacy/device/geo macro name is paired with value, matching pixel_tracker.py's existing trust-boundary posture?
3. OK to land encode_unreserved standalone now and gate universal_macro_translation on the fixture landing, rather than shipping both together against prose alone?

---

Triaged by Claude Code. Session: https://claude.ai/code/cse_017hA41XWeTJS9X2nyoeZaxZ

---

Generated by Claude Code