Skip to content

ci: separate generation from type checking in CI pipeline #123

Description

@emilyoram

Context

The lint CI job previously ran make typecheck (pyright) against the committed generated code in generated/. This broke when examples referenced types from new upstream API operations (e.g. ResourceSearchQuery from searchResources) that hadn't been regenerated into the committed code yet.

What was done (PR #122)

Moved make typecheck from the lint job into the test job, gated to python-version == '3.12'. The test job runs make itest, which depends on make generate, so pyright now checks freshly generated code.

The lint job retains:

  • make lint (ruff)
  • make sync-readme-check
  • make config-reference-check
  • make docs-link-check

Tradeoffs of current approach

  • Slower feedback — type errors now wait for the full test pipeline (Docker, Node.js, spec fetch, generation, integration tests) instead of the fast lint job.
  • Fragile dependency chain — if Docker, spec fetch, or tests fail, pyright never runs.
  • Redundant pyright runsmake generate already runs uv run pyright and uv run pyright examples/ internally, then the explicit make typecheck step runs uv run pyright again.
  • Docker required for type checking — purely static analysis is now gated behind Docker Compose.

Future improvement: separate generation job

Long-term, consider splitting CI into three jobs:

lint  (fast, no generation)  →  ruff, readme checks, config checks, docs links
gen   (medium, no Docker)    →  bundle-spec + generate + pyright + pyright examples/ + acceptance tests
test  (slow, Docker)         →  integration tests + coverage checks

The gen job would:

  1. Set up Node.js + Python
  2. Run make bundle-spec (fetch upstream spec)
  3. Run generation + formatting + linting of generated code
  4. Run make typecheck (pyright on generated code + examples)
  5. Run acceptance tests

This gives fast type-checking feedback (~2-3 min) without needing Docker, while keeping the heavy integration tests separate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions