Add CI coverage for assistant modules

[alexeygrigorev]

Add CI coverage for assistant modules

Status: pending
Tags: assistant, podcast, infra, testing, P0
Depends on: #7
Blocks: None

Scope

Add GitHub Actions coverage for assistant modules under assistants/**, with the initial required coverage focused on the canonical Podcast Assistant at assistants/podcast/.

Current gap from the #7 on-call context: commit 245a276 canonicalized Podcast Assistant under assistants/podcast/, but existing workflows do not run for assistants/** changes. The current workflows are:

• .github/workflows/deploy-dataops-v1.yml, which is scoped to the V1 app, content, Lambda, scripts, tests/docs_app, and work-engine paths.
• .github/workflows/validate-dataops-content.yml, which is scoped to content/** and the content validation workflow.

This issue should add or update CI so assistant changes get a deterministic, non-credentialed test signal. The expected baseline command from #7 is:

uv run --project assistants/podcast pytest

The workflow must be safe for a clean GitHub Actions checkout. It must not require real Telegram delivery, TELEGRAM_BOT_API_KEY, TELEGRAM_CHAT_ID, GROQ_API_KEY, live Groq calls, live Heru engine runs, Codex, or Claude. If the current assistants/podcast dependency on a local Heru checkout (../../../heru) prevents this from working in Actions, resolving that CI dependency boundary is part of this issue, but only to the extent needed for safe unit CI.

Acceptance Criteria

• GitHub Actions runs assistant CI when files under assistants/** change.

• Assistant CI also runs when its workflow file changes, and can be started with workflow_dispatch.

• The workflow runs on push to main; adding pull_request coverage for assistants/** is acceptable and preferred if it matches existing repo conventions.

• Podcast Assistant unit tests run in CI with the safe baseline command, or a documented equivalent that executes the same default assistants/podcast/tests suite from a clean checkout:

  uv run --project assistants/podcast pytest

• CI does not set or require Telegram, Groq, Heru engine, Codex, or Claude credentials for the default assistant test job.

• CI does not run live Telegram delivery, Groq transcription/image analysis, real Heru processing, Codex, or Claude by default.

• Opt-in real-engine tests under assistants/podcast/tests_integration/ remain excluded from normal CI, or are collected only in a way that proves they skip when PODCAST_ASSISTANT_INTEGRATION_ENGINES is unset.

• CI is able to install/resolve the assistant test environment from a clean GitHub Actions checkout. Any handling of the Heru dependency is explicit and does not depend on Alexey's local ../heru directory being present on the runner.

• Workflow path filters include assistant package metadata and lock/config files that affect assistant tests, such as assistants/podcast/pyproject.toml, assistants/podcast/uv.lock, and assistant test files.

• CI failure messages are attributable to the assistant job rather than being hidden inside the DataOps V1 deploy workflow.

• Existing DataOps V1 app and content workflows keep their current purpose; this issue should not accidentally deploy, refresh content caches, or require AWS credentials for assistant-only test changes.

Test Scenarios

Scenario: Podcast unit tests run for assistant code changes

Given: A change touches assistants/podcast/main.py, assistants/podcast/session_retrier.py, or a file under assistants/podcast/tests/
When: The change is pushed to main or evaluated through any configured pull request event
Then: GitHub Actions runs the assistant CI job and executes the Podcast Assistant unit suite without external credentials.

Scenario: Workflow-only change can validate assistant CI

Given: A change touches the assistant CI workflow file
When: The workflow runs through path filters or workflow_dispatch
Then: The assistant CI job runs and reports a pass/fail status for the Podcast Assistant test suite.

Scenario: No live integrations in default CI

Given: The Actions runner has no TELEGRAM_BOT_API_KEY, TELEGRAM_CHAT_ID, GROQ_API_KEY, Codex CLI, Claude CLI, or live Heru credentials
When: The assistant CI job runs
Then: The default unit suite still passes or fails based on code behavior only, not missing external credentials.

Scenario: Real-engine tests remain opt-in

Given: PODCAST_ASSISTANT_INTEGRATION_ENGINES is unset
When: Any integration-test collection/check is run for assistants/podcast/tests_integration/
Then: Codex/Claude real-engine tests are skipped and do not consume external quota or require CLIs.

Scenario: Assistant-only changes do not trigger app deployment

Given: A change only touches assistants/** and the assistant CI workflow
When: The change reaches GitHub Actions
Then: The assistant test workflow runs, and the DataOps V1 deploy/content refresh workflows are not required for assistant-only validation unless their own path filters are intentionally changed in a separate issue.

Out of Scope

• Implementing or changing assistant product behavior.
• Running real Telegram, Groq, Heru, Codex, or Claude checks in normal CI.
• Adding production assistant deployment, AWS resources, DynamoDB tables, S3 artifact storage, or portal UI.
• Closing the [HUMAN] Telegram smoke check left open on Import Podcast Assistant into assistants/podcast #7.
• Modifying source repos outside dataops, including ../podcast-assistant, ../dtc-operations, ../datatasks, or ../heru, unless a separate issue explicitly scopes that work.
• Refactoring all assistant packaging beyond what is required for a clean, safe CI test environment.

Dependencies

• Import Podcast Assistant into assistants/podcast #7 provides the canonical assistants/podcast/ package location and current safe local test command.
• The engineer should verify the workflow from a clean checkout assumption, not from local-only directories available on Alexey's machine.
• If Heru must be available for import-time unit tests, the solution should make that dependency explicit in dataops CI without requiring live Heru engine execution.

[alexeygrigorev]

Software Engineer handoff for #52

Summary:

• Added a standalone assistant CI workflow at .github/workflows/assistant-ci.yml for assistants/** changes, workflow-file changes, push to main, pull_request, and manual workflow_dispatch.
• Kept assistant CI separate from the DataOps V1 deploy/content workflows, with only contents: read permissions and no AWS/secrets setup.
• Removed the default clean-checkout dependency on local ../../../heru from assistants/podcast/pyproject.toml and uv.lock.
• Made heru_runner.py import Heru lazily only for live processing, with a clear runtime error when Heru is missing.
• Added assistants/podcast/sitecustomize.py so the exact baseline command uv run --project assistants/podcast pytest runs the assistant unit suite from repo root in a clean uv environment.
• Updated integration tests so assistants/podcast/tests_integration/ skips cleanly when PODCAST_ASSISTANT_INTEGRATION_ENGINES is unset, without requiring Heru/Codex/Claude at collection time.
• Updated README docs for safe unit tests, opt-in real-engine tests, and explicit local Heru installation for live /process use.
• No commit made.

Changed files:

• .github/workflows/assistant-ci.yml
• assistants/podcast/README.md
• assistants/podcast/heru_runner.py
• assistants/podcast/pyproject.toml
• assistants/podcast/sitecustomize.py
• assistants/podcast/tests/test_heru_runner.py
• assistants/podcast/tests_integration/test_real_engines.py
• assistants/podcast/uv.lock

Verification:

• uv lock --project assistants/podcast: PASS.
• UV_PROJECT_ENVIRONMENT=/tmp/dataops-podcast-ci-py313-final-venv uv run --project assistants/podcast --python 3.13 pytest: PASS, 29 passed.
• UV_PROJECT_ENVIRONMENT=/tmp/dataops-podcast-ci-py313-final-int-venv uv run --python 3.13 pytest tests_integration -q from assistants/podcast/: PASS, 2 skipped as expected with PODCAST_ASSISTANT_INTEGRATION_ENGINES unset.
• UV_PROJECT_ENVIRONMENT=/tmp/dataops-podcast-ci-unit-venv-2 uv run --project assistants/podcast python - <<'PY' ...: PASS, confirmed heru is absent and heru_runner still imports.
• ruby -e 'require "yaml"; Dir[".github/workflows/*.yml"].sort.each { |path| YAML.load_file(path); puts path }': PASS for all workflow YAML files.
• git diff --check -- .github/workflows/assistant-ci.yml assistants/podcast/...: PASS.

Acceptance criteria status:

• All Add CI coverage for assistant modules #52 acceptance checkboxes are satisfied and marked complete.
• Assistant CI path filters include assistants/**, covering package metadata, lock/config files, code, and tests.
• Default CI runs uv run --project assistants/podcast pytest without Telegram, Groq, Heru engine, Codex, Claude, AWS, or deployment credentials.
• Default CI does not run live Telegram delivery, Groq transcription/image analysis, real Heru processing, Codex, or Claude.
• Real-engine tests are collected separately only to prove they skip by default.
• Clean-checkout Heru boundary is resolved by removing the local editable dependency from the default package and making Heru a documented explicit install only for live processing.

Skipped checks / reasons:

• Real Telegram delivery, Groq calls, live Heru processing, Codex, and Claude were not run because Add CI coverage for assistant modules #52 requires default CI to avoid live integrations and credentials.
• DataOps V1 app/content/deploy checks were not run because this issue is scoped to assistant CI and assistant packaging; existing V1 workflows were not modified.

Worktree note:

• I did not modify frontend/**. Existing local changes in frontend/src/app.js and frontend/src/styles.css were present during final status review and were left untouched.

[alexeygrigorev]

TEST PASS

Tester verification for #52 is complete. I kept the review scoped to the assistant CI/package files from the SWE handoff and did not evaluate or modify the unrelated active frontend/** / tests/docs_app/test_frontend_routes_and_links.py worktree edits.

Evidence and commands:

• Read AGENTS.md, _docs/PROCESS.md, .claude/agents/tester.md, gh issue view 52 --repo DataTalksClub/dataops --json title,body,labels,state,assignees,comments, and the SWE handoff comment at Add CI coverage for assistant modules #52 (comment).
• Inspected scoped Add CI coverage for assistant modules #52 diff including untracked files: .github/workflows/assistant-ci.yml, assistants/podcast/sitecustomize.py, and tracked assistant package/test/lock/docs changes. Local unrelated changes remain in frontend/src/app.js, frontend/src/styles.css, and tests/docs_app/test_frontend_routes_and_links.py; those are not part of Add CI coverage for assistant modules #52.
• git diff --check -- assistants/podcast/README.md assistants/podcast/heru_runner.py assistants/podcast/pyproject.toml assistants/podcast/tests/test_heru_runner.py assistants/podcast/tests_integration/test_real_engines.py assistants/podcast/uv.lock -> exit 0.
• git diff --check --no-index /dev/null .github/workflows/assistant-ci.yml; test $? -eq 1 -> exit 0, no whitespace errors in untracked workflow file.
• git diff --check --no-index /dev/null assistants/podcast/sitecustomize.py; test $? -eq 1 -> exit 0, no whitespace errors in untracked helper file.
• ruby -e 'require "yaml"; Dir[".github/workflows/*.yml"].sort.each { |path| YAML.load_file(path); puts path }' -> exit 0; all workflow YAML files parsed.
• Static workflow assertion for .github/workflows/assistant-ci.yml push/main, pull_request paths, workflow_dispatch, contents: read, and no credential setup/secrets references -> exit 0.
• uv run --project assistants/podcast pytest -> exit 0; 29 passed.
• env -u PODCAST_ASSISTANT_INTEGRATION_ENGINES uv run pytest tests_integration -q from assistants/podcast/ -> exit 0; 2 skipped.
• UV_PROJECT_ENVIRONMENT=$(mktemp -d /tmp/dataops-52-clean-heru-check.XXXXXX) uv run --project assistants/podcast python - <<'PY' ... -> exit 0; fresh environment had no heru installed and heru_runner imported successfully.

Acceptance criteria:

• PASS: Assistant CI runs for assistants/** via .github/workflows/assistant-ci.yml push and pull_request path filters.
• PASS: Assistant CI runs when .github/workflows/assistant-ci.yml changes and supports workflow_dispatch.
• PASS: Workflow runs on push to main; pull_request coverage is also present for the same assistant/workflow paths.
• PASS: Podcast Assistant unit tests run with the exact safe baseline command uv run --project assistants/podcast pytest; result was 29 passed.
• PASS: Default assistant CI job does not set or require Telegram, Groq, Heru engine, Codex, Claude, AWS, or deployment credentials; workflow permissions are only contents: read.
• PASS: Default CI does not run live Telegram delivery, Groq transcription/image analysis, real Heru processing, Codex, or Claude. The only integration step skips when the opt-in engine env var is unset.
• PASS: assistants/podcast/tests_integration/ remains opt-in; with PODCAST_ASSISTANT_INTEGRATION_ENGINES unset, both tests skipped without invoking CLIs or live engines.
• PASS: Clean-checkout Heru boundary is satisfied. The assistant lock/project no longer includes editable ../../../heru; in a fresh UV_PROJECT_ENVIRONMENT, heru was absent and heru_runner still imported because Heru is lazily imported only for live processing.
• PASS: Workflow path filters include assistants/**, which covers assistant package metadata, lock/config files, code, and tests including assistants/podcast/pyproject.toml and assistants/podcast/uv.lock.
• PASS: CI failures are attributable to a standalone Assistant CI / Podcast Assistant tests job rather than the DataOps V1 deploy workflow.
• PASS: Existing DataOps V1 deploy and content workflows keep their current purpose and path filters; assistant-only changes do not trigger deployment/content refresh workflows or AWS credential setup.

Skipped by design: no live Telegram, Groq, Heru, Codex, Claude, credentialed, or UI screenshot checks were run. #52 has no UI changes, and the issue explicitly requires default verification to avoid live integrations.

[alexeygrigorev]

ACCEPTED

Final PM acceptance for #52 passes from the product/process perspective.

Reasons:

• The implementation adds a standalone .github/workflows/assistant-ci.yml with clear assistant ownership: it runs for assistants/**, its own workflow file, push to main, pull_request, and workflow_dispatch.
• The default assistant job is deterministic and non-credentialed. It has only contents: read, does not configure AWS, does not reference repository secrets, and does not require Telegram, Groq, Heru engine, Codex, or Claude credentials.
• The workflow runs the safe baseline command uv run --project assistants/podcast pytest; I reran it from the repo root and got 29 passed.
• Real-engine tests remain opt-in. With PODCAST_ASSISTANT_INTEGRATION_ENGINES unset, uv run pytest tests_integration -q from assistants/podcast/ produced 2 skipped and did not invoke live engines.
• The clean-checkout Heru boundary is resolved for CI: heru is removed from the default assistant package/lock dependency graph, heru_runner.py imports Heru only for live processing, and the README documents explicit local Heru installation for /process use.
• Existing DataOps V1 deploy/content workflows keep their current purpose and path filters; assistant-only changes are not routed through app deploy or content refresh workflows and do not require AWS credentials.
• Import Podcast Assistant into assistants/podcast #7 remains open with the human label and its manual Telegram smoke check is not closed by this issue, which matches Add CI coverage for assistant modules #52's out-of-scope boundary.
• I kept this acceptance scoped to the Add CI coverage for assistant modules #52 assistant CI/package files and did not evaluate or modify the unrelated active frontend/** / tests/docs_app/test_frontend_routes_and_links.py worktree edits.

No UI screenshots were needed because #52 has no UI change. No live Telegram, Groq, Heru, Codex, Claude, or credentialed checks were run because the accepted behavior is that normal CI avoids those integrations.

[alexeygrigorev]

ON-CALL REPORT

Commit monitored: 7547293 (7547293) - Add assistant CI coverage

Runs observed:

• Assistant CI: PASS - https://github.com/DataTalksClub/dataops/actions/runs/28302019166
  • Job: Podcast Assistant tests PASS - https://github.com/DataTalksClub/dataops/actions/runs/28302019166/job/83851862706

Checks performed:

• Read AGENTS.md and _docs/PROCESS.md before monitoring.
• Ran gh run list --repo DataTalksClub/dataops --limit 20; the latest run for this push is Assistant CI, completed successfully.
• Queried runs/check-runs for head SHA 754729385c48b52179afd2ab63baa4c3b37446fd; GitHub reports only the Assistant CI workflow and its Podcast Assistant tests check for this commit.
• Reviewed the commit file list. The push changed .github/workflows/assistant-ci.yml and assistants/podcast/** files only.

Path-filter status:

• Assistant CI triggered as expected because the commit changed .github/workflows/assistant-ci.yml and assistants/**.
• Deploy DataOps V1 Lambda did not trigger for this commit, as expected. Its push path filters cover deploy/app/content/work-engine paths, not assistants/**.
• Validate DataOps Content did not trigger for this commit, as expected. Its push path filters cover content/** and its own workflow file, not assistant files.

Classification:

• No failures observed.
• Current status: healthy. No CI/deployment infrastructure, code/test, content/search validation, external account/secret, or flake issue to route.

Next step:

• None required for Add CI coverage for assistant modules #52. Leave the approved commit as-is; no force-push, amend, PR, or check bypass performed.