Skip to content

Redis cluster support Phase 2a: hash-tag key matrix + trace namespace unification #9

Description

@zhchxiao123

What to build

Roll out Cluster-safe key naming across every key factory method in Constants.java. Two independent things happen under REDIS_KEY_SCHEMA_VERSION=v2:

  1. Every key gains a byai_gateway:v2: prefix (v1 stays unprefixed, unchanged, and remains the default).
  2. Keys belonging to the same logical entity (session, worker, task_group, trace, agent_type, service-discovery name) get a Redis Cluster hash tag around the shared ID, so multi-key operations against the same entity land on the same slot. Single-key methods, global-index keys (SETs shared across entities), and the small set of keys that are read together but never written together atomically do NOT get a tag.

The prefix + tag logic must be centralized in one formatting function that every key factory method calls — no per-method judgment calls about whether to add the prefix.

Same-entity groups requiring a shared tag: session (data_stream / consumer checkpoint / registry), worker (ctrl stream / lock / online-lease / status / executions / active_executions / active_execution_index / active_snapshots / history_snapshots / admin / declared_agent_types), task_group (main + results), agent_type (members + denied — this one is mandatory, not optional: WorkerRegistry.java's denyWorkerForType already writes both keys together and must keep working), service-discovery (active instances + instance details), trace (meta + spans).

Global-index keys (known-workers set, admin-workers set, service-discovery services set) do NOT get a tag.

Separately, trace keys get a bigger change: today Java's trace keys use a by_framework:trace:* namespace (byte-for-byte identical to Python's current format), while TS uses a different byai_gateway:trace:{id}:meta/spans namespace. This is the SDK-wide decision to unify onto one v2 format used by all three languages: byai_gateway:v2:trace:{trace_id} for meta, byai_gateway:v2:trace:spans:{trace_id} for spans (type-then-id ordering, consistent with the rest of the key matrix). trace_meta/trace_spans share the trace_id hash tag. trace_index_session/worker/agent keep their existing path structure, only the prefix changes — they do NOT get a tag (splitting them out of the shared write pipeline is a separate, later issue).

Also add this SDK's contribution to a cross-language "golden key" test: given a fixed set of IDs (e.g. session_id="sess-abc123", worker_id="worker-01", trace_id="trace-xyz", etc.), Java's key factory must produce exact key strings matching a shared fixture (JSON/YAML) that Python has already populated and TS will populate in its own equivalent issue. Under v1, trace keys are expected to still differ from what TS produces (pre-existing inconsistency) — only assert full cross-language equality for trace keys under v2.

Acceptance criteria

  • All key factory methods route through one shared prefix/tag-assembly function; no method independently decides whether to add the v2 prefix
  • Same-entity groups listed above share a hash tag; global-index keys do not
  • agentTypeMembers/agentTypeDenied (or equivalent) share a mandatory tag (not marked optional)
  • traceMeta/traceSpans move to the unified byai_gateway:v2:trace:{trace_id} / byai_gateway:v2:trace:spans:{trace_id} format; trace_index_* keeps its path structure, only the prefix changes
  • Slot-verification unit test: for every same-entity group, assert all member keys hash to the same Cluster slot
  • Golden-key test: Java's key factory output matches the shared fixture under v2; under v1, trace keys are exempted from cross-language equality assertions
  • v1 behavior/output is completely unchanged

Blocked by

#8

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified, ready for an AFK agent

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions