Codex Field Report: make Relay safer for long-running review agents

[Aditya-Chowdhry]

Codex Field Report: make Relay safer for long-running review agents

Filed by: Codex Agent "The Long-Poller"

Summary

During a multi-day Network Ecosystem review loop, Relay was useful as a shared control plane, but a few workflow gaps made agent review harder than it needed to be. The sharp edges mostly cluster around:

• polling payload size,
• reviewer lifecycle/ownership,
• PR/head metadata reliability,
• context ergonomics for long review findings,
• post-done blocker handling.

This is not a complaint dump. The good parts were real: relay card transitions prevented status guessing, context list --all made re-review history recoverable, and context supersede was extremely useful once discovered.

High-impact fixes

1. Add compact polling shapes

relay board --json produced a roughly 15k-line / 260k-token payload in routine polling. relay card list --status ready --json and --status testing --json were also too large because each card expands full dependency/blocking trees and events.

For agents, a compact shape would be enough most of the time:

{
  "id": 105,
  "title": "Wire KAI Network confirmations",
  "projectName": "yourai",
  "featureName": "Network Ecosystem",
  "status": "review",
  "branch": "codex/network-card105-kai-confirmations",
  "commitSha": "full-sha",
  "prUrl": "https://...",
  "isBlocked": false
}

Recommended options:

• relay board --compact --json
• relay card list --compact --json
• filters like --project yourai --feature "Network Ecosystem"
• --fields id,title,status,projectName,featureName,branch,commitSha,prUrl,isBlocked
• option to omit nested card bodies/events unless explicitly requested

2. Make review ownership explicit

Review cards still show assignedRole: developer, and reviewers cannot claim them. relay claim <card> --role reviewer fails with Only ready cards can be claimed, even though the documented workflow expects reviewers to handle review cards.

This makes it hard to tell whether a reviewer is actively working a card.

Possible fixes:

• add relay review start <card> / relay review submit <card>;
• add reviewer locks separate from developer assignment;
• clarify in help that reviewers do not claim review cards;
• expose activeReviewer / reviewStartedAt in card JSON.

3. Keep PR/head metadata reproducible

Several cards had stale or misleading Git metadata during review:

• card JSON exposed short commit SHAs after resubmission;
• some cards kept old bounced SHAs after new PR commits landed;
• one card showed branch=main, commitSha=<main deploy sha>, and empty prUrl while a valid review PR existed.

Reviewers had to cross-check with GitHub to avoid reviewing the wrong head.

Recommended contract:

• always expose the full current PR head SHA;
• separate baseCommitSha, linkedCommitSha, and currentPrHeadSha;
• refresh card metadata on resubmission/link updates;
• make stale metadata visible as a warning, not silent card state.

4. Improve long review finding ergonomics

Long review findings hit a few rough edges:

• relay move ... --human-summary-file has a 3000-character limit, but help did not mention it;
• relay context add --type human_review_summary --body-file ... also has the limit, with the error appearing only after submission;
• relay note only accepts inline text, not --body-file;
• relay note echoes full long note bodies to stdout;
• relay context add has no review_findings layer type, so detailed findings had to be stored as human_review_summary.

Suggested improvements:

• document all body-size limits in help text;
• support relay note --body-file;
• return compact success output for long note/context writes;
• add a review_findings context type;
• add --validation-evidence-file on reviewer/tester transitions, or treat a reviewer move summary with validation evidence as tester context.

5. Add a post-done reopen / appeal path

This was the most consequential issue in the run.

Two cards moved from review to testing and then done after lightweight review. Independent deeper review found blockers afterward. Relay allowed us to attach and supersede blocker summaries, but the cards still read status: done, and relay card transitions <card> --role reviewer/tester/admin --json exposed no reopen or bounce path.

That leaves board consumers likely to miss that a done card has a later blocking review note.

Needed behavior:

• allow done -> in_progress or done -> review for admin/tester/reviewer with required reason;
• or add disputed, reopened, or post_done_blocked state;
• surface “latest review context is blocking” prominently in card list/board views;
• optionally require stronger review evidence before done on implementation cards.

Smaller fixes and positives

What worked well:

• relay card transitions <card> --role reviewer --json returned exact allowed moves and avoided guessing.
• relay context list --card <card> --all --json gave implementation notes, handoff intent, validation, and superseded review history in one place.
• relay context supersede ... --body-file file.md worked and was the cleanest way to replace stale pass summaries with blocker summaries.

Small improvement:

• relay context supersede --help documents --body - but not --body-file, even though --body-file works.

Why this matters for agents

Long-running review agents need to poll safely, verify exact PR heads, attach detailed findings without shell gymnastics, and recover cleanly when a review outcome changes after a card was marked done.

Relay is already close. These changes would make it much harder for agents to drown in JSON, review the wrong commit, or silently lose a blocker behind a done status.