Skip to content

MCPJVM-367: Transitional legacy correlation index backfill for SQLite state-store #367

Description

@nimbly-dev

Description

During migration away from shared correlation-index.json, it may be useful to support a temporary compatibility path that imports legacy correlation-oriented index data into the SQLite-backed state-store. This should be treated as transitional migration support only, not as the long-term recovery path.

This story adds transitional legacy backfill/import behavior for correlation-oriented SQLite state-store data where migration support is still needed.

Scope

In scope:

  • Support temporary import/backfill from legacy shared correlation-index.json
  • Limit legacy import support to correlation-oriented state, and only any partial run linkage needed by that migration path
  • Position legacy import as transitional compatibility rather than long-term source-of-truth behavior
  • Keep backfill behavior deterministic and fail-closed when legacy data is missing or invalid
  • Keep implementation aligned with the existing artifact_management action-plus-shared helper structure

Out of scope:

  • Treating legacy shared index data as the long-term recovery path
  • Full rebuild of all SQLite state surfaces from legacy index data alone
  • New public MCP query/read contracts beyond the migration capability entrypoint
  • Performance Suite integration

Expected Tables / Detailed Schema Direction

Primary legacy import target:

correlation_runs

Column Type Required from legacy source Purpose
run_id TEXT yes imported run linkage
project_name TEXT yes owning project
plan_name TEXT yes owning plan linkage
correlation_session_id TEXT no imported session grouping if present
status TEXT yes imported correlation verdict
reason_code TEXT yes imported deterministic reason
key_type TEXT yes imported correlation key type
key_value TEXT no imported resolved key value
window_start_epoch_ms INTEGER no imported window start if present
window_end_epoch_ms INTEGER no imported window end if present
max_window_ms INTEGER yes imported/derived max window
probe_ids_json TEXT yes imported probe id array summary
generated_at_epoch_ms INTEGER no imported sortable timestamp

Optional partial linkage target only when needed:

suite_runs

Column Type Required from legacy source Purpose
run_id TEXT yes linkage only
project_name TEXT yes linkage only
plan_name TEXT yes linkage only
status TEXT no partial inferred status if defensible
generated_at_epoch_ms INTEGER no partial timestamp if present

Legacy backfill should not imply honest reconstruction of watcher_runs or external_verifications from correlation-index-only data.

Acceptance Criteria

  • SQLite-backed state-store supports transitional legacy correlation-index import where needed for migration
  • Legacy import scope is explicitly limited and does not overstate coverage for watcher or external verification state
  • Legacy backfill is positioned as temporary compatibility support rather than long-term recovery behavior
  • Backfill fails closed when required deterministic legacy data is missing or invalid
  • The implementation remains aligned with the existing artifact_management action-plus-shared helper structure

Implementation Notes

  • Treat this as a migration bridge with removal potential in a later release rather than a permanent capability.
  • Avoid expanding legacy import semantics to state surfaces that cannot be reconstructed honestly from legacy correlation index data.

Testing

  • UT for legacy index parsing and backfill mapping behavior
  • UT for fail-closed handling on malformed or incomplete legacy index data
  • IT for transitional backfill into an empty SQLite state-store
  • IT for deterministic skip/insert behavior on repeated backfill runs

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions