Skip to content

Use walk-forward training windows for bounded parameter selection #18

Description

@sybapp

Follow-up from review

Walk-forward validation currently records separate training and scoring windows, but each fixed Strategy Spec is evaluated on both windows. Search/template parameter selection happens outside the training window loop.

What to build

Make the walk-forward loop use training/search windows to select Strategy Spec parameters or bounded template choices, then evaluate selected candidates on later scoring windows.

Acceptance criteria

  • For each walk-forward split, candidate selection or parameter choice is based only on the training/search window.
  • The scoring window remains out-of-sample and is not used to choose candidates.
  • The Run Registry records training-window selection inputs, selected candidate/spec, scoring-window result, and final ranking inputs.
  • Tests prove scoring-window-only features/results cannot influence selection.
  • Existing fixed-spec walk-forward validation remains available for diagnostics or smoke checks, but is not presented as full search-window optimization.

Context

Review finding: run_walk_forward_backtest validates the same fixed Strategy Spec on training and scoring slices, which proves separation but does not yet perform training-window selection.

Related PRD line: "The evaluator should split data into search/training windows and later scoring windows."

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified and ready for an AFK agent

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions