Need much better testing infracture on local host

[jonfroehlich]

We need a much better testing infrastructure on localhost. This is something we have never had but would likely reduce number of bugs that get on to -test or production. Claude Code CLI previously wrote this:

I want to build out test infrastructure for this Django project. Current state: 49 SimpleTestCase tests with MagicMock, no DB. We've shipped several fixes (per codebase-analysis.md) that lacked automated coverage because they needed DB-touching tests — query-count regressions for the prefetch_related perf wins, view-level tests for the null-author guards, the delete_unused_files guards, and the member.py field reference fix. Read codebase-analysis.md, the memory files in ~/.claude/projects/-Users-jonfroehlich-Git-makeabilitylabwebsite/memory/, and the existing website/tests.py first.

Goals: (1) a clean base TestCase class with fixture helpers for Person / Publication / Project (handle Person.save()'s Star Wars image side-effects via mocking), (2) at least one assertNumQueries-style perf regression test pinning the /publications/ prefetch_related (so we'd catch it if anyone removed it), (3) backfill the missing view-level tests for the null-author guards in views/news_item.py and news_listing.html. Pa11y CI exists for accessibility — we don't need to touch that.

[jonfroehlich]

Strongly +1 on this direction — and the scope I drafted in the issue body is a good first slice.

A few additional thoughts now that we've shipped 2.3.4 and 2.3.5 with several fixes that would have been caught by DB-touching tests:

Concrete gaps the current SimpleTestCase + MagicMock style misses:

1. Query-count regressions. The prefetch_related batches in 2.3.4 (d4f6d65, 1cae31c) cut /publications/ from 617 → 60 queries. There's nothing stopping a future contributor from removing those prefetches; an assertNumQueries-bounded test would catch it.
2. Null-FK guards. The 1c0d6c0 fix (authorless news items) and the 202b1bb fix (Person.modified_date typo on the MultipleObjectsReturned branch in views/member.py) both went out without regression tests because they required real querysets to exercise.
3. delete_unused_files .path guards (202b1bb, c7abcf7) — same shape, also untestable without DB.
4. View-level smoke. We have zero end-to-end view tests. A handful of Client().get('/publications/')-style tests asserting 200 + key elements would catch broad regressions cheaply.

Suggested first slice (one PR, one session):

• Add a DatabaseTestCase base (Django's TestCase with a few fixture helpers — make_person, make_publication, make_project). Use model_bakery or just plain Model.objects.create() — both work.
• Mock the Person.save() Star Wars / file-I/O side effects via a single class-level @patch.
• Add one assertNumQueries regression test pinning the /publications/ prefetch_related batch.
• Add one view-level test for /news/<id>/ with a null author (regression for the bug closed in Receiving Server Error on a news item #1013).

That's a small, contained PR that proves out the pattern without committing to a full backfill. Once the harness exists, future fixes can include DB-touching regression tests as part of the same surgical-batch workflow we already use (memory: feedback-workflow).

One thing to decide: do we run DB tests against SQLite (fast, in-memory, lowers the bar to running tests) or against the actual Postgres dev DB (slower, but matches prod)? Probably SQLite for speed, with a separate --postgres flag for full-fidelity runs before tagging a release. Worth a 5-min discussion before starting.

Happy to take a first pass at the harness in a future session.

[jonfroehlich]

Continuation tracked in #1272 (2026-06-14)

First DB-backed slice landed in 4fdf6d6 (DatabaseTestCase + make_person / make_publication / make_news_item helpers). Next slice — replacing those helpers with proper factory_boy factories, adding curated seed media so file-field code paths get exercised, and a seed_dev_data management command for realistic local dev — is now tracked in #1272.

Leaving this open as the umbrella issue. #1272 can close independently once the factories + seed_dev_data work ships; further test-infra needs (perf regression tests, view-level coverage for the null-author guards, etc., all mentioned in the original brief) continue under this issue.

[jonfroehlich]

Closing as effectively complete and superseded by the testing roadmap in #1278.

The three specific goals from this issue have all landed:

1. Base TestCase with fixtures — DatabaseTestCase + make_person/make_publication/make_talk/make_news_item (now in website/tests/base.py).
2. Query-count perf regression test — PublicationsViewQueryCountTests pins the /publications/ prefetch_related batch.
3. Null-author view-level tests — NewsItemNullAuthorViewTests exercises the URL→view→template stack.

Beyond the original ask, #1280 (merged 2026-06-15) added GitHub Actions CI, a test-settings shim (MIGRATION_MODULES={'website': None} — the durable fix for the column already exists flakiness this issue mentioned), and split the suite into a website/tests/ package.

Remaining testing work (coverage, backfill, Pa11y-in-CI) is tracked in #1278; realistic fixtures in #1272.