Skip to content

Wire the verdict seam to real hermetic per-PR CI #1

Description

@ronimoe

Context

BatchEngine.tick(verdict, ...) takes a verdict(spec_tree, members, reps) -> BatchVerdict seam; today it's fed by from_green/fakes. Production must run the repo's real CI against the spec tree under repetition and return per-test DET_PASS/DET_FAIL/UNRESOLVED.

Scope

  • Dispatch a GitHub Actions run (or check-run) per spec ref; collect per-test outcomes across reps.
  • Map results into BatchVerdict.per_test; run under the hermetic test runtime (see hermetic-test-runtime issue).

Acceptance

A real PR lands end-to-end on a sandbox repo via Actions; a flaky vs deterministic-red test is classified correctly from real runs (no masking).

Seam

tools/conductor/verdict.py, batch_engine.py (the verdict param).

Metadata

Metadata

Assignees

No one assigned

    Labels

    operational-hardeningWiring real GitHub/CI/registry behind the engine seams

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions