Add local coverage tooling

.github/workflows/ci.yml

@@ -43,8 +43,8 @@ jobs:
         run: uv run ruff format --check .
       - name: Lint
         run: uv run ruff check .
-      - name: Run tests
-        run: uv run pytest -q
+      - name: Run tests with coverage
+        run: uv run pytest --cov=copilot_experiments --cov-report=term-missing:skip-covered
       - name: Check installed dependency metadata
         run: uv pip check
 

.gitignore

@@ -8,6 +8,9 @@ dist/
 .venv/
 .pytest_cache/
 .ruff_cache/
+.coverage
+coverage.xml
+htmlcov/
 
 # Experiment results (filesystem artifacts + SQLite index are regenerable)
 results/

README.md

@@ -103,6 +103,7 @@ uv run ruff check --fix .
 uv run ruff format .
 uv run ruff check .
 uv run pytest -q
+uv run pytest --cov=copilot_experiments --cov-report=term-missing:skip-covered
 ```
 
 Install the local hooks once to make Ruff formatting/lint fixes automatic on commit and tests run
@@ -113,4 +114,7 @@ uv run pre-commit install --install-hooks
 uv run pre-commit install --hook-type pre-push
 ```
 
+The pre-push hook runs plain pytest for speed. Run the coverage command explicitly when you need
+branch coverage and missing-line details; CI also runs it for every push and pull request.
+
 See [`AGENTS.md`](AGENTS.md) for contributor guidance.

docs/adr/0018-adopt-pytest-cov-for-local-coverage-analysis.md

@@ -0,0 +1,37 @@
+# 0018. Adopt pytest-cov for local coverage analysis
+
+- **Status:** Accepted
+- **Date:** 2026-06-21
+- **Deciders:** project owner, Copilot
+
+## Context
+
+The package already uses pytest for an offline test suite, Ruff for formatting and linting, uv for
+dependency management, and GitHub Actions for quality checks. The project has no existing coverage
+configuration, dependency, reporting command, or CI coverage checkpoint.
+
+Coverage analysis should stay local and GitHub Actions-only. A hosted service would add account,
+permission, badge, and upload concerns that are not needed for the current workflow. The chosen tool
+should also feel lightweight and current rather than adding a separate clunky reporting system.
+
+## Decision
+
+We will use `pytest-cov` as the pytest integration layer for `coverage.py`.
+
+- `pytest-cov` will live in the development dependency group.
+- Coverage collection will be scoped to `src/copilot_experiments`.
+- Branch coverage will be enabled so reports catch missing decision paths, not just missing lines.
+- GitHub Actions will run the test suite with coverage reporting.
+- The pre-push hook will continue to run plain pytest so local pushes stay fast; developers can run
+  coverage explicitly when they need the report.
+- We will not set an initial hard coverage threshold. The first step is to establish a reliable
+  baseline before deciding whether a gate is useful.
+
+## Consequences
+
+- Contributors can get coverage feedback with the same pytest workflow they already use.
+- CI records coverage output without introducing Codecov, Coveralls, or another hosted dependency.
+- `coverage.py` remains available underneath `pytest-cov`, so the project can later add XML, HTML,
+  branch-specific, or diff-oriented reporting without changing tools.
+- Without an initial threshold, coverage regressions are visible but not automatically blocked. A
+  future ADR or follow-up change can introduce a threshold once the baseline is understood.

docs/adr/README.md

@@ -34,3 +34,4 @@ We follow the lightweight format popularized by
 | [0015](0015-adopt-pier-for-sandboxed-agent-evals.md) | Adopt Pier for sandboxed agent evaluations | Accepted |
 | [0016](0016-use-deepswe-for-large-benchmark-protocols.md) | Use DeepSWE for large benchmark protocols | Accepted |
 | [0017](0017-import-deepswe-as-pier-dataset.md) | Import DeepSWE as a Pier dataset config | Accepted |
+| [0018](0018-adopt-pytest-cov-for-local-coverage-analysis.md) | Adopt pytest-cov for local coverage analysis | Accepted |

pyproject.toml

@@ -33,6 +33,7 @@ packages = ["src/copilot_experiments"]
 dev = [
     "pre-commit>=4.0",
     "pytest>=8.0",
+    "pytest-cov>=6.0",
     "ruff>=0.5",
 ]
 
@@ -55,3 +56,16 @@ docstring-code-format = true
 [tool.pytest.ini_options]
 testpaths = ["tests"]
 addopts = "-q"
+
+[tool.coverage.run]
+branch = true
+source = ["copilot_experiments"]
+
+[tool.coverage.report]
+show_missing = true
+skip_covered = true
+exclude_also = [
+    "if TYPE_CHECKING:",
+    "if __name__ == .__main__.:",
+    "raise NotImplementedError",
+]