Skip to content

Capture reviews on PRs created outside the coverage window #18

Description

@gkanitz

Follow-up to #3 (PR #15).

The v1 GitHub adapter scans reviews only for PRs created inside the coverage window (fetchPulls filters on created_at before review fetching). Consequences:

  • Reviews the subject gave during the window on PRs opened before the window are not counted in reviews_given.
  • Review comments are unaffected (the repo-level pulls/comments listing is walked independently and filtered by comment timestamp).

This was accepted as a documented v1 approximation during review of PR #15. A later slice should widen the PR scan (e.g. also list PRs updated in the window, or use the search API) so reviews-given covers all in-window review activity regardless of when the reviewed PR was created.


Amendment (2026-07-04) — pulled into "Launch readiness"

A known data-correctness gap is disqualifying for a product whose pitch is
"these numbers are trustworthy", so this ships before the public push.
Precise criteria added for the TDD workflow:

Approach

Widen the PR scan to PRs updated in the coverage window (GitHub: pulls
list supports sort=updated; GitLab: updated_after filter on MRs), then
attribute each review by its submission timestamp against the window.
Authored-PR stats keep their created-in-window semantics (document this in
the metric definitions).

Success criteria (write these tests first; all must pass)

  1. Fixture: PR created before since, review submitted inside the window →
    counted in reviews_given (and in the correct trend bucket).
  2. Fixture: PR created before the window, review submitted before the
    window → not counted.
  3. Fixture: PR both created and updated inside the window → its reviews are
    counted exactly once (no double counting from the widened scan).
  4. Pagination fixture with mixed update order: an old PR updated recently
    appears on a late page and is still scanned (no early-termination loss).
  5. Cross-adapter parity: equivalent GitHub/GitLab fixtures produce identical
    reviews_given totals.
  6. Authored-PR stats (pull_requests, pr_size once merged) are unchanged
    by this widening — asserted against existing goldens.
  7. Review comments logic is untouched (already timestamp-filtered) —
    existing tests stay green.

Failure — QA red flags (any one observed → reject the PR)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready-for-agentTriage complete; ready for an agent to pick up

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions