Commit d4e7595
authored
chore(showcase): include failed job/step + error excerpt in validate Slack alert (CopilotKit#4068)
## Summary
- Bare `:x: Showcase validate: failed | View run` forces a click-through
to triage. Red alerts must carry triage-ready detail per the oss-alerts
policy.
- Adds a pre-notify step that extracts the failed step name and first
meaningful error line, then interpolates both into the Slack payload.
- Companion to CopilotKit#4065 (which quieted routine success noise); this PR
makes the remaining failure-only posts actionable at a glance.
## Before / after
**Before**
```
:x: Showcase validate: failed | View run
```
**After** (against tonight's run `24598654559`, the validate-parity
regression)
```
:x: Showcase validate: failed — Run validate-parity (MUST checks gating): [FAIL] ag2: demo 'hitl-in-chat' declared in manifest but no src/app/demos/hitl-in-chat/ directory | View run
```
## Extraction approach
New `Extract failure details for Slack` step (runs only on `failure() &&
push && env.SLACK_WEBHOOK != ''`):
1. **Resolve current job ID** via `GET
/repos/{repo}/actions/runs/{run_id}/jobs`. Match by `name == "Validate
Showcase"` (primary) with a fallback to the first job containing a
failed step (rename-drift tolerance).
2. **First failed step name** pulled from the same jobs response.
3. **First error line** via `gh run view --log-failed --job=<id>`.
Pipeline:
- `awk -F'\t'` to split TSV columns (`job \t step \t timestamp +
content`) and strip the leading ISO timestamp + optional BOM from the
content column.
- `sed` strips ANSI escape codes.
- `grep -vE` skips runner header noise (`##[group]`, `shell:`, `env:`,
`Run`, blank lines).
- `grep -m1 -E` grabs the first line matching `[FAIL]`, `[ERROR]`,
`Error:`, `error:`, or `::error`.
- `head -c 300` truncates to keep the payload under the ~800-char Slack
budget even after JSON escaping.
4. **Writes `failed_step` + `error_excerpt` to `$GITHUB_ENV`** using
heredoc delimiters (safe for values containing `=` or newlines).
5. **Never blocks the notify step**: `set +e`, stderr suppressed on all
extraction commands, explicit `exit 0`, and safe fallbacks (`"unknown
step"` / `"see workflow run for details"`).
The notify step's payload now uses `toJSON(format(...))` around all four
interpolated values (repo, run_id, failed_step, error_excerpt) so any
quotes/backslashes/newlines in extracted content are JSON-encoded
defensively — matches the pattern from `showcase_drift-report.yml`.
## Validation
- `actionlint .github/workflows/showcase_validate.yml` — no findings in
the edited region (lines 360-470). The only reported finding is the
pre-existing `depot-ubuntu-24.04-4` label info on line 47, unrelated.
- Dry-run of the extraction pipeline against run `24598654559`:
```
gh run view 24598654559 --repo CopilotKit/CopilotKit --log-failed
--job=71933395025 \
| awk -F'\t' 'NF>=3 { sub(/^[\xEF\xBB\xBF]?[0-9T:.\\-Z ]+/, "", $3);
print $3 }' \
| sed 's/\x1b\[[0-9;]*[a-zA-Z]//g' \
| grep -vE '^(##\[|shell: |env: |Run |[[:space:]]*$)' \
| grep -m1 -E '^\[(FAIL|ERROR)\]|^Error:|^error:|^::error' \
| head -c 300
```
Output:
```
[FAIL] ag2: demo 'hitl-in-chat' declared in manifest but no
src/app/demos/hitl-in-chat/ directory
```
Which is exactly the signal that was missing from tonight's bare alert.
- `oxfmt --check` and `oxlint` on the edited file: clean.
## Scope
- Only `.github/workflows/showcase_validate.yml` is touched.
`showcase_deploy.yml` and `showcase_smoke-monitor.yml` are untouched.
- Preserves the empty-webhook guard (`env.SLACK_WEBHOOK != ''`) added in
CopilotKit#4065 and mirrors that pattern on the new extraction step.
## Test plan
- [ ] Merge and wait for the next validate failure on `main` (or
temporarily force one on a push-path) to confirm the Slack payload
carries the expected `— <step>: <excerpt>` suffix.
- [ ] If a subsequent run fails in a step with no matching error marker,
confirm the fallback `"see workflow run for details"` renders cleanly
rather than an empty string.1 file changed
Lines changed: 97 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
361 | 361 | | |
362 | 362 | | |
363 | 363 | | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
364 | 451 | | |
365 | 452 | | |
366 | 453 | | |
367 | 454 | | |
368 | 455 | | |
369 | 456 | | |
370 | 457 | | |
371 | | - | |
372 | | - | |
373 | | - | |
374 | | - | |
375 | | - | |
376 | | - | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
377 | 467 | | |
378 | | - | |
| 468 | + | |
379 | 469 | | |
380 | 470 | | |
381 | 471 | | |
| |||
0 commit comments