List of bugs

[nimbly-dev]

Alias Map

• API Gateway Service: HTTP trigger entry service
• Entity Service: producer and authoritative SQL state owner
• Search Service: downstream consumer and search query endpoint
• Monitoring Service: downstream monitoring/count endpoint
• Regression Orchestrator: execution profile runner
• Artifact Store: persisted suite/run artifacts

---

Issue Table

Severity	Title	Summary	Evidence	Impact
Blocker	API Service consumer workflow does not converge to searchable state	End-to-end regression reaches the search-consumer plan, HTTP trigger steps pass, but downstream indexed visibility does not converge.	Suite 07-08-2026-01-16-42PM: plans 1-3 passed, plan 4 blocked. Watcher expected response.bodyJson.hits.total.value >= 500 but observed 0.	Prevents the full regression workflow from passing.
Blocker	Watcher can fail closed after long polling with actual_path_missing_persistent	After many successful HTTP 200 polling attempts, the watcher can terminate on a persistent path-resolution failure when the response shape changes.	Run 07-08-2026-01-27-14PM: 155 watcher attempts, final HTTP 500, reason watcher_configuration_invalid, cause actual_path_missing_persistent.	Long-running watcher plans can end as configuration failures instead of clean business-state failures.
Critical	SQL external verification requires runtime JDBC-driver support that is currently missing	SQL verification progressed from unresolved connection config to execution failure because the runtime lacked a suitable PostgreSQL JDBC driver.	Earlier run failed on missing sql.connection.postgresql.jdbc.url; later run failed with No suitable driver found for jdbc:postgresql://localhost:5432/<db>.	PostgreSQL-backed external verification cannot be relied on.
Critical	Regression Orchestrator tool call times out before long plan completion	Long-running suite execution exceeded the 300-second tool boundary, preventing synchronous visibility into completion.	Suite 07-08-2026-01-21-00PM timed out twice while artifact status remained in_progress at nextPlanOrder=4.	Operators cannot rely on the live call result for long plans.
High	Orchestrator lacks an outer bounded retry model for plan reruns	Execution supports resumable slices and watcher retry, but not a direct “rerun this plan N times” control.	Confirmed during execution-profile runs: retries are only available via watchers[].waitPolicy.retryMax.	Makes iterative diagnosis of flaky async plans slower.
High	Persisted progress can lag behind active long-running execution	During timeouts, the suite artifact may still show the same nextPlanOrder and no new run artifact.	Suite 07-08-2026-01-21-00PM remained at completed plans 1-3 after two timed-out resumes of plan 4.	Hard to distinguish active work from stalled execution.
Medium	Run-result artifact reads are strict and fail without explicit pagination	Reading watcher sections through the artifact tool requires exact pagination fields.	artifact_management rejected run_result watcher reads until query.watchers.offset and query.watchers.limit were supplied.	Adds friction for debugging.
Medium	Public SQL contract examples are insufficient for PostgreSQL authoring	Visible contract examples emphasized SQLite-like configuration and did not make effective PostgreSQL requirements obvious.	Initial host/port/database/user/password bindings were insufficient; runtime still required JDBC URL and then driver support.	Slows SQL verification adoption.
Minor	Final watcher failure reason can obscure the business-state failure	A watcher may surface transport/configuration failure at the end even when earlier attempts showed the underlying business condition never became true.	Search watcher earlier reported HTTP 200 with hits.total.value = 0, then later ended with transport/config/path reasons.	Investigation time increases.

---

Recommended Epic Grouping

1. Async Consumer Verification Reliability
2. SQL External Verification Runtime Support
3. Orchestrator Observability and Timeout Handling
4. Artifact Inspection Ergonomics

---

Blocker Issues

1. API consumer workflow does not converge to searchable state

Issue Description

API workflow execution reaches the downstream consumer plan and both trigger HTTP checks pass, but the expected indexed visibility does not converge. The effect is a blocked regression even though upstream authentication, trigger, and producer plans pass.

Observed Behavior

The downstream search plan repeatedly returned HTTP 200, but the watcher assertion on indexed result count remained false. In one terminal run the watcher saw hits.total.value = 0 instead of the expected >= 500. In a later run, long polling ended with a persistent missing path after the response shape changed under an HTTP 500.

Expected Behavior

The downstream search query should eventually expose the expected indexed document count after producer completion.

Steps to Reproduce

1. Execute the regression profile that runs authentication, trigger, producer, and search-consumer plans in order.
2. Allow the search-consumer plan to reach its HTTP watcher phase.
3. Observe repeated successful HTTP responses without convergence to the expected indexed count, followed by a blocked terminal state.

Supporting Evidence

• Suite run 07-08-2026-01-16-42PM: plans 1-3 passed, search plan blocked.
• Search watcher assertion expected response.bodyJson.hits.total.value >= 500 and observed 0.
• Later run 07-08-2026-01-27-14PM: 155 attempts, final HTTP 500, reason watcher_configuration_invalid, cause actual_path_missing_persistent.

Impact

Blocks end-to-end regression completion and leaves downstream indexing reliability unproven.

Sanitization Note

Class names, package names, enterprise identifiers, hosts, and credentials were anonymized or redacted.

---

2. Watcher can fail closed after long polling with actual_path_missing_persistent

Issue Description

The watcher performs long bounded polling successfully for many attempts, but when the response shape changes under failure conditions, it can terminate as a configuration/path failure instead of a clear business-state timeout.

Observed Behavior

The watcher recorded many HTTP 200 responses and then ended with an HTTP 500. The assertion path response.bodyJson.hits.total.value became unavailable and the watcher failed with watcher_configuration_invalid.

Expected Behavior

The watcher should either:

• continue reporting a clean unmet business assertion, or
• fail with a transport/business-state reason that preserves the real underlying condition.

Steps to Reproduce

1. Run the regression profile until the search-consumer watcher starts polling.
2. Let the watcher continue through repeated polling attempts.
3. Observe terminal failure after response-shape drift.

Supporting Evidence

• Run 07-08-2026-01-27-14PM
• attemptCount=155
• final response statusCode=500
• reason watcher_configuration_invalid
• cause actual_path_missing_persistent

Impact

Makes watcher failures harder to classify and debug.

Sanitization Note

Class names, package names, enterprise identifiers, hosts, and credentials were anonymized or redacted.

---

Critical Issues

3. SQL external verification requires runtime JDBC-driver support that is currently missing

Issue Description

SQL external verification was added to validate durable downstream state, but runtime execution failed first on missing connection context and then on missing PostgreSQL JDBC driver support.

Observed Behavior

The first SQL-backed run failed with missing runtime key sql.connection.postgresql.jdbc.url. After adding the JDBC URL binding, the next run failed with No suitable driver found for jdbc:postgresql://localhost:5432/<db>.

Expected Behavior

When the plan provides a valid PostgreSQL connection reference, SQL external verification should execute and evaluate the configured assertions.

Steps to Reproduce

1. Configure SQL external verification on the search-consumer plan with provider.type = sql.
2. Bind a PostgreSQL connectionRef through project context.
3. Run the regression profile and observe runtime failure in external verification.

Supporting Evidence

• Earlier run reason: external_verification_connection_unresolved
• Missing key: sql.connection.postgresql.jdbc.url
• Later run reason: external_verification_execution_failed
• Error: No suitable driver found for jdbc:postgresql://localhost:5432/<db>

Impact

Durable SQL verification for async workflow convergence cannot currently be trusted.

Sanitization Note

Class names, package names, enterprise identifiers, hosts, and credentials were anonymized or redacted.

---

4. Regression Orchestrator tool call times out before long plan completion

Issue Description

The regression orchestrator supports resumable suite execution, but long-running plans exceeded the 300-second tool boundary before a synchronous result was returned.

Observed Behavior

Multiple orchestration calls timed out after 300 seconds while the suite artifact still showed in_progress and nextPlanOrder=4.

Expected Behavior

Either the orchestration call should return progress more frequently, or the persisted state should advance clearly enough to distinguish active work from stalled execution.

Steps to Reproduce

1. Start the full regression execution profile.
2. Resume the suite into the long-running consumer plan.
3. Observe tool-call timeout at 300 seconds before terminal suite completion.

Supporting Evidence

• Suite run 07-08-2026-01-21-00PM timed out twice at the tool boundary.
• Persisted suite artifact remained in_progress with completed plans 1-3 only.

Impact

Operators must manually inspect persisted artifacts to understand whether the run is still progressing.

Sanitization Note

Class names, package names, enterprise identifiers, hosts, and credentials were anonymized or redacted.

---

High Issues

5. Orchestrator lacks an outer bounded retry model for plan reruns

Issue Description

Execution supports resumable slices and watcher retry, but not a direct operational “rerun this plan N times” control.

Observed Behavior

Retry behavior is only available at the watcher level through waitPolicy.retryMax, not through orchestration itself.

Expected Behavior

Operators should have a bounded rerun capability for flaky long-running plans without editing plan authoring state.

Steps to Reproduce

1. Execute the regression profile.
2. Attempt to configure orchestration to rerun a failing consumer plan 20 or 30 times.
3. Observe that only watcher-level retry exists.

Supporting Evidence

• Execution-profile orchestration supports suiteRunId and maxPlansPerCall
• Retry semantics are encoded in watchers[].waitPolicy.retryMax

Impact

Operational diagnosis is slower and more intrusive than it needs to be.

Sanitization Note

Class names, package names, enterprise identifiers, hosts, and credentials were anonymized or redacted.

---

6. Persisted progress can lag behind active long-running execution

Issue Description

During long-running timed-out orchestration calls, persisted suite state may not advance, making it difficult to determine whether the system is still processing or stalled before artifact persistence.

Observed Behavior

After repeated 300-second timeouts, the suite artifact still showed only plans 1-3 completed with nextPlanOrder=4.

Expected Behavior

Persisted progress should advance often enough to distinguish active execution from stalled execution.

Steps to Reproduce

1. Start the regression profile.
2. Resume into the long-running consumer plan.
3. Let the tool call hit the 300-second boundary and inspect persisted suite state.

Supporting Evidence

• Suite 07-08-2026-01-21-00PM remained on completed plans 1-3 after timed-out resumes.

Impact

Reduces confidence in operational visibility and slows incident triage.

Sanitization Note

Class names, package names, enterprise identifiers, hosts, and credentials were anonymized or redacted.

---

Medium Issues

7. Run-result artifact reads are strict and fail without explicit pagination

Issue Description

Reading watcher sections from run-result artifacts requires explicit pagination fields; otherwise the read fails closed.

Observed Behavior

Artifact reads failed until query.watchers.offset and query.watchers.limit were supplied.

Expected Behavior

The tool should either provide a clearer default for small result sets or emit simpler guidance inline.

Steps to Reproduce

1. Read a run_result artifact with watchers selected.
2. Omit watcher pagination.
3. Observe validation failure.

Supporting Evidence

• artifact_management rejected watcher reads without required pagination fields.

Impact

Adds friction to debugging and ad hoc investigation.

Sanitization Note

Class names, package names, enterprise identifiers, hosts, and credentials were anonymized or redacted.

---

8. Public SQL contract examples are insufficient for PostgreSQL authoring

Issue Description

The visible SQL contract examples did not make the effective PostgreSQL runtime requirements obvious.

Observed Behavior

Initial host/port/database/user/password bindings looked reasonable but were insufficient. Runtime still required a JDBC URL and then a PostgreSQL driver.

Expected Behavior

PostgreSQL requirements should be explicit in the visible contract/examples.

Steps to Reproduce

1. Author SQL external verification from the visible contract/examples.
2. Bind PostgreSQL connection fields without JDBC URL.
3. Run and observe unresolved/execution failures.

Supporting Evidence

• Contract examples emphasized SQLite-like concrete examples.
• Runtime required sql.connection.postgresql.jdbc.url.
• Runtime later failed on missing suitable JDBC driver.

Impact

Slows adoption of SQL verification and increases trial-and-error.

Sanitization Note

Class names, package names, enterprise identifiers, hosts, and credentials were anonymized or redacted.

---

Minor Issues

9. Final watcher failure reason can obscure the business-state failure

Issue Description

A watcher may surface a transport or configuration error at the end even when earlier attempts clearly showed the business condition never became true.

Observed Behavior

Earlier attempts returned HTTP 200 with unmet indexed-count assertions, but the terminal failure later changed to transport/path/configuration reasons.

Expected Behavior

Final watcher reporting should preserve both:

• the terminal failure reason
• the dominant business-state failure trend observed during polling

Steps to Reproduce

1. Run the search-consumer watcher through many polling attempts.
2. Allow the target state to remain unconverged.
3. Observe terminal failure after response-shape or transport change.

Supporting Evidence

• Earlier run: HTTP 200 with hits.total.value = 0
• Later run: HTTP 500 and actual_path_missing_persistent

Impact

Investigation time increases because the most actionable failure is not always the final one surfaced.

Sanitization Note