Skip to content

Redis cluster support Phase 5: cluster-aware SCAN + gather-based batch admin state reads #47

Description

@zhchxiao123

What to build

SCAN (and scan_iter) is a single-node command in Redis — under Cluster it only sees keys on the node it was issued against. Add a cluster-aware scan wrapper used by all metrics/snapshot.py/admin-CLI code instead of calling scan_iter directly: it must enumerate every master node and merge results, so callers get the same complete result set in standalone and cluster modes. Verify against the actually-pinned redis.asyncio.cluster.RedisCluster version whether .scan_iter() already aggregates across nodes internally (don't assume — check the pinned version's docs/source or test it) — if it doesn't, wrap it so it does.

Separately, _batch_fetch_admin_states reads multiple different workers' worker_admin(id) keys in one pipeline — another cross-entity multi-key operation (read-only, but still CROSSSLOT-prone under Cluster). Replace it with concurrent independent per-worker requests via asyncio.gather(*[redis.hgetall(RedisKeys.worker_admin(wid)) for wid in worker_ids]) rather than relying on unverified pipeline(transaction=False) cross-slot routing behavior.

Acceptance criteria

  • Cluster-aware scan helper added, used everywhere metrics/snapshot.py/admin code currently calls scan_iter directly
  • Verified (doc reference or test against the real pinned client version) whether the cluster client aggregates multi-node scan itself or needs manual per-node iteration + merge
  • _batch_fetch_admin_states converted from pipeline to asyncio.gather of independent requests
  • Integration test against a local Cluster: writing known keys across multiple slots, confirm the scan helper returns the full set, not just one node's subset

Blocked by

#45

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified, ready for an AFK agent

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions