fix(showcase): validate verify-matrix boundaries (SSOT + summary shape), fail loud, test CLI contract

A 7-agent review of the verify-matrix resolver and its surrounding workflow plumbing found three
boundary surfaces that could silently produce a GREEN deploy on a broken release, plus an
untested CLI contract that CI compares against the literal strings 'true' / 'false'.

FIX 1 — Validate the SSOT shape in loadSsotServices(). The prior `JSON.parse(...) as
{services: SsotService[]}` was an unchecked cast: a truncated/drifted SSOT (emitter crashed
mid-write, or schema renamed) parses fine but silently shrinks/empties the probe-eligible set
→ some redeployed services go unverified, or verify is skipped on a real redeploy. Extract a
pure exported parseSsotServices(raw, path) that requires the shape we depend on (non-empty
services array; each entry has a non-empty string name, an optional string|null dispatchName,
and a probe object with a boolean staging). Throw `::error::SSOT <path> malformed: <detail>`
on any violation. Also re-check existsSync(SSOT_JSON) after the regenerate-if-missing
execFileSync — a regen that exits 0 without writing must not proceed to a useless JSON.parse
crash. Drop the defensive `probe?.staging` once shape is guaranteed.

FIX 2 — Validate summary.json shape in the redeploy-gate bash. The bullseye false-green
surface: if redeploy-env.ts's schema ever drifts (e.g. `status` → `state`, `ok` → `success`),
every `jq select(.status==...)` yields empty → redeploy_red=false AND ok_services="" →
resolver skips verify → GREEN CI on a real unverified redeploy. Add a TOTAL vs WITH_STATUS
shape guard right after loading the summary: if TOTAL > 0 && WITH_STATUS == 0, emit
::error::summary.json has $TOTAL entries but none with status ok|error (schema drift?) and
exit 1. The legitimate empty-array path (TOTAL=0) is preserved.

FIX 3 — Fail loud on unknown eventName in resolveVerifyMatrix. The prior code fell through to
the workflow_run intersection branch for ANY unrecognized eventName (typo, unexpected
trigger), silently emitting has_services=false → indistinguishable from a legit "summary
absent" skip. Add an explicit guard so only workflow_run / workflow_dispatch are accepted;
anything else throws ::error::resolve-verify-matrix: unexpected eventName '<value>'. Tighten
the eventName parameter type to the literal union.

FIX 4 — Trim ok tokens + warn on dropped tokens in okCsvToCanonicalNames. Split, then
.map(t => t.trim()).filter(Boolean) so "a, b" (spaces) matches. Collect tokens that match NO
SSOT service (by name or dispatchName) and have the CLI wrapper emit ::warning::ok_services
tokens dropped (no SSOT match): <list> on stderr when non-empty — surfaces SSOT/build drift.
The pure function stays IO-free; logging lives in the wrapper.

FIX 5 — CLI wrapper integration test. New resolve-verify-matrix.cli.test.ts spawns
`npx tsx showcase/scripts/resolve-verify-matrix.ts` with a temp $GITHUB_OUTPUT file across
four scenarios and asserts the temp file contents EXACTLY (the workflow YAML compares
has_services against the literal strings 'true'/'false', so the byte-for-byte format is part
of the contract). Uses the real railway-envs.generated.json so the loader exercise is real.

FIX 6 — Cleanup. Remove the dead `env: DISPATCH_SERVICE: ...` block on the redeploy-gate
step (the next step redeclares it — leftover from the extraction). Soften the §3
decision-table all-errors bullet to match resolve-verify-matrix.ts's careful wording, and
append that when the success-set is empty (or the intersection collapses to empty), verify
is skipped and the gate reds independently. Append to showcase_build.yml's "Upload redeploy
summary" path-(A) comment that `if-no-files-found: error` still reds path (A) even if a
future change adds `if: always()`.

Tests: red→green for FIX 1/3/4/5 verified locally. Resolve-verify-matrix vitest count:
12 → 28. Full requested suite (resolve-verify-matrix + cli + aggregate-build-results +
lint-rule-no-public-env): 72 passed. showcase/bin ruby specs: 87 runs / 0 failures / 0
errors / 0 skips. actionlint baseline preserved (8 findings, identical to integration tip).

Author: jpr5

.github/workflows/showcase_build.yml

@@ -602,6 +602,13 @@ jobs:
         # redeployed", skip the gate, and ship a false-green.
         # The legitimate "services == '' → nothing redeployed → no upload"
         # path is preserved by the services != '' guard.
+        #
+        # If a future change ever switches this step to `if: always()`,
+        # `if-no-files-found: error` STILL reds path (A): the redeploy
+        # step's non-zero exit on a HARD crash is independent of upload
+        # gating, and `if-no-files-found: error` on `always()` then trips
+        # because summary.json was never written. So neither relaxation
+        # alone opens a false-green window.
         if: steps.changed.outputs.services != ''
         uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
         with:

.github/workflows/showcase_deploy.yml

@@ -139,8 +139,6 @@ jobs:
 
       - name: Gate on staging redeploy errors
         id: redeploy-gate
-        env:
-          DISPATCH_SERVICE: ${{ github.event.inputs.service }}
         run: |
           set -euo pipefail
           SUMMARY=".redeploy/summary.json"
@@ -150,6 +148,21 @@ jobs:
             echo "ok_services=" >> "$GITHUB_OUTPUT"
             exit 0
           fi
+          # Shape guard: redeploy-env.ts writes per-entry `status` of
+          # exactly "ok" or "error". If the schema ever drifts (e.g.
+          # `status`→`state`, `ok`→`success`) every `select(.status==...)`
+          # silently yields empty → redeploy_red=false AND ok_services=""
+          # → resolve-verify-matrix skips verify on a real unverified
+          # redeploy = green CI on a broken release. Refuse the ambiguity:
+          # if the file has entries but none carry an ok|error status,
+          # fail loud here so enforce-redeploy-gate reds the workflow
+          # (resolve-matrix.result == 'failure' fans into the gate).
+          TOTAL=$(jq 'length' "$SUMMARY")
+          WITH_STATUS=$(jq '[.[] | select(.status == "ok" or .status == "error")] | length' "$SUMMARY")
+          if [ "$TOTAL" -gt 0 ] && [ "$WITH_STATUS" -eq 0 ]; then
+            echo "::error::summary.json has $TOTAL entries but none with status ok|error (schema drift?)"
+            exit 1
+          fi
           # Per spec §3: the workflow MUST turn red on any staging
           # status:"error", while verify still runs against the success-set.
           ERRORS=$(jq -c '[.[] | select(.status == "error")]' "$SUMMARY")
@@ -180,14 +193,19 @@ jobs:
         #   - workflow_run + summary_present=false → has_services=false
         #                                            (build redeployed nothing).
         #   - workflow_run + summary_present=true + ok empty
-        #       → has_services=false. Every service errored on redeploy;
-        #         the success-set is empty so there is nothing to verify.
-        #         `enforce-redeploy-gate` independently reds the workflow
-        #         on redeploy_red=true, so this case is already loud.
+        #       → has_services=false (success-set empty; verify is skipped).
+        #         In practice redeploy-env.ts only emits status ok|error,
+        #         so this branch implies redeploy_red=true and
+        #         enforce-redeploy-gate reds the workflow independently —
+        #         skipping verify here is correct (no ok services left to
+        #         probe; the gate has already turned the workflow red).
         #   - workflow_run + summary_present=true + ok non-empty
         #       → intersect ok_services (SSOT key OR dispatchName aliases)
         #         with probe.staging-eligible SSOT services. has_services
-        #         reflects CSV emptiness.
+        #         reflects CSV emptiness. When the intersection collapses
+        #         to empty (e.g. every ok service is non-probe-eligible),
+        #         verify is skipped and `enforce-redeploy-gate` reds the
+        #         workflow independently on redeploy_red=true.
         run: npx tsx showcase/scripts/resolve-verify-matrix.ts
 
   verify:

showcase/scripts/__tests__/resolve-verify-matrix.cli.test.ts

@@ -0,0 +1,180 @@
+/**
+ * resolve-verify-matrix.cli.test.ts — pins the byte-for-byte $GITHUB_OUTPUT
+ * contract that showcase_deploy.yml depends on.
+ *
+ * The deploy workflow's `verify:` job has
+ *   if: needs.resolve-matrix.outputs.has_services == 'true'
+ * which compares against the LITERAL strings 'true'/'false' (everything in
+ * $GITHUB_OUTPUT is a string). A regression that writes `has_services=1`
+ * or `has_services=True` would silently skip verify on every redeploy.
+ * Pin that contract here so the byte-for-byte format is part of the test
+ * surface, not just the pure-function unit tests.
+ *
+ * The pure resolver is covered by resolve-verify-matrix.test.ts. THIS
+ * file spawns the CLI as a child process against the real
+ * railway-envs.generated.json so the loader + parseSsotServices +
+ * writeGithubOutput contract gets end-to-end coverage.
+ *
+ * Cases (mirrors decision-table boundaries the workflow YAML depends on):
+ *   1. workflow_run + SUMMARY_PRESENT=true + OK_FROM_REDEPLOY=""
+ *        → services_csv=\nhas_services=false\n      [Issue A pinned end-to-end]
+ *   2. workflow_run + SUMMARY_PRESENT=true + OK_FROM_REDEPLOY="<two real names>"
+ *        → services_csv=<sorted CSV>\nhas_services=true\n
+ *   3. workflow_dispatch + no summary → full probe-eligible set, has_services=true.
+ *   4. workflow_dispatch + service=<unknown> → non-zero exit, stderr ::error::Unknown service.
+ */
+import { execFileSync } from "node:child_process";
+import { mkdtempSync, readFileSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { dirname, join, resolve } from "node:path";
+import { fileURLToPath } from "node:url";
+import { describe, expect, it } from "vitest";
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = dirname(__filename);
+
+// The repo root is the cwd the workflow YAML uses for the script (paths
+// inside the script are written as `showcase/scripts/...`, relative to
+// repo root). Tests therefore must spawn from repo root.
+const REPO_ROOT = resolve(__dirname, "..", "..", "..");
+const SCRIPT = "showcase/scripts/resolve-verify-matrix.ts";
+const SSOT_PATH = join(REPO_ROOT, "showcase/scripts/railway-envs.generated.json");
+
+interface SpawnResult {
+  status: number;
+  stderr: string;
+  output: string;
+}
+
+function runCli(env: Record<string, string>): SpawnResult {
+  const dir = mkdtempSync(join(tmpdir(), "rvm-cli-"));
+  const outputPath = join(dir, "gh_output");
+  writeFileSync(outputPath, "");
+  const fullEnv = {
+    ...process.env,
+    GITHUB_OUTPUT: outputPath,
+    ...env,
+  };
+  try {
+    execFileSync("npx", ["tsx", SCRIPT], {
+      cwd: REPO_ROOT,
+      env: fullEnv,
+      stdio: ["ignore", "pipe", "pipe"],
+    });
+    return {
+      status: 0,
+      stderr: "",
+      output: readFileSync(outputPath, "utf-8"),
+    };
+  } catch (e) {
+    // execFileSync throws on non-zero exit; the thrown error carries
+    // `.status` and `.stderr` (Buffer when stdio captures).
+    const err = e as {
+      status?: number;
+      stderr?: Buffer | string;
+    };
+    return {
+      status: typeof err.status === "number" ? err.status : 1,
+      stderr:
+        typeof err.stderr === "string"
+          ? err.stderr
+          : err.stderr
+            ? err.stderr.toString("utf-8")
+            : "",
+      output: readFileSync(outputPath, "utf-8"),
+    };
+  }
+}
+
+// Pull two real probe-eligible names off the actual SSOT so the test
+// stays in lock-step with the emitter (rather than hard-coding fixtures
+// that could drift).
+function realProbeEligibleNames(): string[] {
+  const raw = JSON.parse(readFileSync(SSOT_PATH, "utf-8")) as {
+    services: { name: string; probe: { staging: boolean } }[];
+  };
+  return raw.services
+    .filter((s) => s.probe.staging === true)
+    .map((s) => s.name)
+    .sort();
+}
+
+describe("resolve-verify-matrix CLI (end-to-end against real SSOT)", () => {
+  // The CLI spawns `npx tsx`. Cold-start can take >1s; bump timeout.
+  it(
+    "workflow_run + summary_present + empty ok_services → has_services=false (Issue A end-to-end)",
+    () => {
+      const r = runCli({
+        EVENT_NAME: "workflow_run",
+        SUMMARY_PRESENT: "true",
+        OK_FROM_REDEPLOY: "",
+        DISPATCH_SERVICE: "",
+      });
+      expect(r.status).toBe(0);
+      // Byte-for-byte: the workflow YAML compares against the literal
+      // strings 'true' / 'false'. Match the EXACT bytes (including the
+      // trailing newlines and key=value form).
+      expect(r.output).toBe("services_csv=\nhas_services=false\n");
+    },
+    30_000,
+  );
+
+  it(
+    "workflow_run + summary_present + two real ok_services → sorted CSV + has_services=true",
+    () => {
+      const probe = realProbeEligibleNames();
+      // Pick TWO probe-eligible names. Use a CSV order that is NOT
+      // already sorted so the sort-on-intersection assertion is real.
+      // probe[0] sorts before probe[1] (alphabetical); feed reversed.
+      const picked = [probe[0], probe[1]];
+      const reversedCsv = `${picked[1]},${picked[0]}`;
+      const sortedCsv = picked.join(",");
+      const r = runCli({
+        EVENT_NAME: "workflow_run",
+        SUMMARY_PRESENT: "true",
+        OK_FROM_REDEPLOY: reversedCsv,
+        DISPATCH_SERVICE: "",
+      });
+      expect(r.status).toBe(0);
+      expect(r.output).toBe(
+        `services_csv=${sortedCsv}\nhas_services=true\n`,
+      );
+    },
+    30_000,
+  );
+
+  it(
+    "workflow_dispatch + no summary → full probe-eligible set + has_services=true",
+    () => {
+      const probe = realProbeEligibleNames();
+      const r = runCli({
+        EVENT_NAME: "workflow_dispatch",
+        SUMMARY_PRESENT: "",
+        OK_FROM_REDEPLOY: "",
+        DISPATCH_SERVICE: "",
+      });
+      expect(r.status).toBe(0);
+      expect(r.output).toBe(
+        `services_csv=${probe.join(",")}\nhas_services=true\n`,
+      );
+    },
+    30_000,
+  );
+
+  it(
+    "workflow_dispatch + unknown service → non-zero exit with ::error::Unknown service",
+    () => {
+      const r = runCli({
+        EVENT_NAME: "workflow_dispatch",
+        SUMMARY_PRESENT: "",
+        OK_FROM_REDEPLOY: "",
+        DISPATCH_SERVICE: "totally-not-a-real-service",
+      });
+      expect(r.status).not.toBe(0);
+      expect(r.stderr).toMatch(
+        /::error::Unknown service 'totally-not-a-real-service'/,
+      );
+    },
+    30_000,
+  );
+});