Skip to content

Phase 14: Implement experiment/target classification and exit-code policy #159

@jununfly

Description

@jununfly

This was generated by AI during triage.

Parent

Parent tracking issue: #154

What to build

Implement formal target-level and experiment-level classification plus the Phase 14 exit-code policy. This slice should make the runner summarize all target outcomes while continuing across target, arm, and prompt failures whenever an artifact can still be produced.

The formal manifest path should use the Phase 14 taxonomy and must not reuse Phase 13 mixed names such as success-asymmetric-blocker.

Acceptance criteria

  • Target-level classifications use the Phase 14 target taxonomy.
  • Experiment-level classifications use the Phase 14 experiment taxonomy.
  • Target, arm, and prompt failures do not abort other targets.
  • Manifest/schema-level fatal errors abort only when no trustworthy artifact/summary can be produced.
  • Required target unavailable, required arm unavailable, or required comparison regression classify the experiment as failed-required-*.
  • Stress target failures can classify as success-required-targets-passed-with-stress-failures when required targets pass.
  • decisionReadiness summarizes sufficiency, performance, required-target status, and keeps rolloutReadinessClaimed=false by default.
  • Default exit code is 0 when raw artifact and summary draft are produced, even if gates fail.
  • Fatal artifact/summary production failure exits 1.
  • --fail-on-required-gate-failure exits 2 when the completed experiment classification starts with failed-required-.
  • Focused tests cover continue-across-targets and exit-code behavior.

Blocked by

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready-for-agentFully specified and ready for an AFK agent

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions