[Feature] Ship LLM Auto-Correction & Goal Validation Loops

[vakrahul]

Status: Closed — superseded by #10

Auto-correction is already implemented inside EngineX step workers. This issue tracked building a separate EvaluationNode — that was not built and is not needed.

What exists (do not rebuild)

Capability	Code
Step judge (missing output_keys → RETRY)	core/engine/graph/event_loop/node.py → _evaluate()
Feedback to LLM on retry	[Judge feedback]: ... via add_user_message()
Optional Level 2 quality judge	conversation_judge.py + NodeSpec.success_criteria
Per-step retry limit	loop_config.max_iterations
Between-step loops (validate → fix)	Conditional EdgeSpec in graph/edge.py
Whole-job Goal scorecard	OutcomeAggregator — measurement only, no full-agent auto-retry

Track remaining work in #10: docs (GOALS.md), dashboard retry visibility, README link.

Original request (historical)

The original issue asked for a standalone EvaluationNode and retry loop wiring. The judge runs inside each EventLoopNode instead.

flowchart TD
 A[EventLoopNode] --> B{_evaluate / judge}
 B -->|ACCEPT| C[Next step]
 B -->|RETRY| A
 B -->|max iterations| D[Step fails]

Loading

[pravinmishra672]

test permission

[pravinmishra672]

Scope update (concise)

Auto-correction is already implemented inside EngineX step workers. This issue = audit, test, document — not build from scratch. See also #10.

---

What already exists

Capability	Code
Step checklist judge (missing output_keys → RETRY)	event_loop/node.py → _evaluate()
Feedback to LLM on retry	[Judge feedback]: ... via add_user_message()
Optional Level 2 quality judge	conversation_judge.py + NodeSpec.success_criteria
Per-step retry limit	loop_config.max_iterations
Between-step loops (validate → fix)	Conditional EdgeSpec in edge.py
Retry telemetry (partial)	ExecutionResult.total_retries, runtime logs
Whole-job Goal scorecard	OutcomeAggregator — tracks only, no full-agent auto-retry

Not built today: separate EvaluationNode type — judge runs inside each event_loop step.

---

What to add

• Audit judge path; document in docs/GOALS.md (with [P0][Phase 1][Docs] Goal vs node criteria, audit auto-correction, improvement gaps #10)
• Tests: RETRY on missing keys, max iterations, feedback injection
• One template step with success_criteria (Level 2 demo)
• Expose retry / execution_quality in dashboard if missing
• Document graph loop pattern for [P0][Phase 1][Agent] Build Enterprise-Grade Hourly Tracking Agent #2 Hourly Tracking

Out of scope: new EvaluationNode unless audit finds a gap.

---

Definition of done

• Docs: step judge vs Goal vs graph loops vs human review (pause_nodes)
• Tests pass for retry + limit failure
• One Level 2 example in templates

---

Suggested title: [Platform] Harden auto-correction: audit, test, and document judge loops

(Org admin: please replace issue description with this scope — edit permission required.)

[pravinmishra672]

Closing as duplicate/superseded.

All scope merged into #10 (docs + audit baseline + engineering deliverables for auto-correction).

Key point: Auto-correction already exists in EventLoopNode._evaluate() — #10 covers audit, tests, and docs rather than building a new EvaluationNode.

Track work there: #10

[pravinmishra672]

Superseded by #10 — please close this issue.

All scope (what exists vs what to add for auto-correction) has been merged into:
#10

#10 now includes:

• Auto-correction audit baseline (judge RETRY, Level 2 judge, telemetry)
• Engineering deliverables (docs, tests, one Level 2 example)
• Clear note: no new EvaluationNode unless audit finds a gap

(Maintainer: close #1 as duplicate of #10.)