Add operator follow-up actions and history controls

[alexeygrigorev]

Add operator follow-up actions and history controls

Status: pending
Tags: enhancement, portal, work-engine, frontend, backend, testing, data, P0
Depends on: #50
Blocks: None

Scope

Add first-class operator controls for waiting-task follow-ups inside the unified DataOps task/workflow flow. This builds on the existing waiting task fields and follow-up-due reminder behavior from #50; it should not create a separate follow-up tool.

The operations manager must be able to open a waiting task from Operations Home, a task panel, or workflow detail and record the next real operational action:

• mark a task as waiting with waitingFor, followUpAt, contact channel, and a short note;
• record that a follow-up was sent without completing the task;
• choose the next follow-up date after sending a follow-up;
• record that a response was received and move the task back to active todo work;
• resolve the wait and mark the task done only when normal proof/completion rules pass;
• see follow-up/waiting history from both task and workflow context;
• preserve follow-up history in audit/export data.

The implementation should keep waiting work visible as part of ordinary tasks and workflow bundles. A waiting task should remain active, should appear in follow-up-due surfaces when its follow-up date arrives, and should stop appearing as waiting/follow-up work only after the wait is explicitly resolved or the task is completed through the approved flow.

API coverage is required. The UI must not be the only place where follow-up semantics exist. Add authenticated work-engine action endpoints, or equivalent atomic task action routes, for mark-waiting, follow-up-sent, response-received/unblocked, and resolve-to-done. These actions must perform server-side validation and history/audit recording instead of relying on the browser to construct timestamped comment text as the durable record.

Acceptance Criteria

• Waiting task actions are available from the task detail/panel and from workflow detail waiting/checklist context; Operations Home follow-up-due items open into the same action flow.
• Marking a task waiting requires waitingFor, followUpAt, contact channel, and a short note; invalid or missing values return clear API/UI validation errors.
• Recording follow-up sent requires a channel, note, and next follow-up date, keeps the task in waiting, updates the active followUpAt, and refreshes dashboard/workflow follow-up state.
• Recording response received or unblocked moves the task out of active waiting/follow-up state into todo without losing the previous waiting/follow-up history.
• Resolving a waiting task as done is possible only through an explicit resolve/complete action and still enforces required link, required file, required artifact, comment, external-status, and bundle-link proof gates.
• Generic task completion controls cannot silently complete a waiting task without resolving the wait and writing history.
• Task and workflow detail show a readable history of waiting started, follow-up sent, next follow-up date, response received/unblocked, resolved, reopened, and completed events with timestamp, actor when available, channel when relevant, and note.
• Follow-up history is structured enough for export and audit: use durable audit events when practical, or an equivalent typed task-history structure with stable IDs/relationships. Existing free-text comments may remain visible, but they must not be the only machine-readable follow-up record.
• Follow-up-due reminders from Generate dashboard reminders from waiting follow-up dates #50 remain idempotent per task/follow-up date and do not reappear for a resolved wait unless a new waiting/follow-up cycle is created.
• Waiting/follow-up behavior works for standalone tasks and tasks linked to workflow bundles, including migrated or intake-created tasks when they already carry waiting metadata.
• Portable export includes current waiting state and follow-up history/audit records with stable task/bundle/user relationships, validates successfully, and omits secrets, session data, signed URLs, credentials, and binary payloads.
• Existing task, bundle, notification, export, and portal/work-engine tests still pass; new tests cover the follow-up action lifecycle and history/export behavior.
• Tester captures screenshots for Operations Home follow-up due entry, task follow-up action controls, workflow waiting/history section, validation errors, and resolved/unblocked state.

Test Scenarios

Scenario: Operator records a follow-up sent

Given: a waiting task is due for follow-up and appears on Operations Home
When: the operator opens it, records follow-up sent with channel email, a short note, and next follow-up date 2026-07-02
Then: the task remains waiting, followUpAt becomes 2026-07-02, the old due reminder is no longer active, and history shows the sent follow-up event with timestamp, channel, note, actor if available, and next date.

Scenario: Response received returns work to active todo

Given: a task is waiting for a sponsor response and has prior follow-up history
When: the operator records response received with a note
Then: the task is no longer in waiting/follow-up lanes, appears as active todo work where appropriate, and task/workflow history still shows who/what it had been waiting for and the response event.

Scenario: Waiting task cannot be silently completed

Given: a waiting task has a required Luma link and no link saved
When: the operator tries to complete it through a generic checkbox or direct task status update
Then: the app/API blocks completion with a clear message; resolving as done succeeds only after the wait is resolved and the required proof is present.

Scenario: Workflow history shows follow-up lifecycle

Given: a workflow bundle has one task that was marked waiting, followed up twice, then unblocked
When: the operator opens workflow detail
Then: the waiting/history section shows the active waiting state when applicable, the done/history section preserves completed items, and the task history can be inspected without leaving workflow context.

Scenario: Export preserves follow-up audit data

Given: local test data includes standalone and bundled tasks with waiting started, follow-up sent, response received, and resolved/done history
When: portable export and validation run
Then: tasks.jsonl contains the current waiting fields where active, audit/history records export with stable task/bundle/user references, relationship validation passes, and no secrets or temporary signed URLs are present.

Scenario: Imported or intake-created waiting task uses the same controls

Given: a task created by migration or intake has status=waiting, waitingFor, and followUpAt
When: the operator opens it from the dashboard or workflow context
Then: the same follow-up sent, next date, response received, and resolve controls are available, and no source-specific follow-up UI is required.

Out of Scope

• Actually sending email, Telegram, Slack, sponsor, guest, or publisher messages from DataOps. This issue records that the operator sent a follow-up elsewhere.
• Building external integrations, OAuth setup, webhooks, or provider-specific message templates.
• Replacing the whole task state model beyond changes needed for follow-up actions/history.
• Migrating production Trello/spreadsheet/intake data; Implement raw intake inbox for operational inputs #31 and Migrate active Trello cards into integrated operations-manager work #41 own their source-specific import/intake work.
• Historical backfill of old follow-up notes beyond preserving existing comments and future action history.
• Modifying ../dtc-operations, ../datatasks, ../podcast-assistant, or any source repo outside dataops.

Dependencies

• Generate dashboard reminders from waiting follow-up dates #50 is the hard dependency because follow-up-due reminders must already be generated from waiting tasks.
• Implement raw intake inbox for operational inputs #31 and Migrate active Trello cards into integrated operations-manager work #41 are related consumers, not blockers. Their intake/migration tasks should use the same follow-up controls when they create tasks with waiting metadata.
• docs/operations-manager-platform-jtbd.md is the product source of truth, especially JTBD 4 and the reminder/task status model.
• docs/v1-execution-state-schema.md and docs/v1-execution-data-safety.md define runtime-state, audit, and portable export expectations.

Affected Areas

• work-engine/src/types.ts for any typed follow-up history/action or audit-event relationship additions.
• work-engine/src/router.ts, task routes, and related DB helpers for authenticated atomic follow-up actions and validation.
• work-engine/src/db/notifications.ts for reminder idempotency/resolution behavior if active follow-up dates or resolved waits change notification semantics.
• work-engine/src/export/portable.ts and export/dry-run validation tests for follow-up history/audit preservation.
• frontend/src/app.js, frontend/src/styles.css, and any work-engine public UI still used by the operator flow for task/workflow controls and history display.
• Existing task, bundle, notification, export, dashboard, and Playwright tests.

Data Safety, Export, And Restore Implications

• Follow-up notes may contain private operational context. Keep notes bounded, redact or reject detectable secrets where existing export safety supports it, and never store API keys, OAuth tokens, cookies, signed URLs, session data, or credentials as follow-up metadata.
• Store structured metadata and references only. Do not store message binaries, raw email bodies, Telegram payloads, or external attachment contents in DynamoDB as part of follow-up history.
• Portable exports must represent both active waiting state and historical follow-up events using stable application IDs and explicit relationships. Do not make exports depend on DynamoDB PK/SK values.
• If new durable fields or event types are added, update export validation and dry-run import validation so unknown enums, invalid dates, missing task references, and bad user references fail clearly.
• Production restore drills or destructive restore/import checks remain out of scope for agent verification unless a later issue explicitly authorizes them.

Verification Commands

Run from repo root:

npm --prefix work-engine test
npm --prefix work-engine run typecheck
npm --prefix work-engine run build
npm --prefix work-engine run test:e2e
npm --prefix work-engine run export:data -- .tmp/exports/follow-up-actions
npm --prefix work-engine run validate:export -- .tmp/exports/follow-up-actions
npm --prefix work-engine run dry-run:import -- .tmp/exports/follow-up-actions
git diff --check

If implementation changes the Python portal broker, protected portal route handling, or docs-app tests fixtures, also run:

uv run --project lambda-functions --extra search --with pytest python -m pytest tests/docs_app

Tester must include screenshot evidence for the changed operator flows listed in the acceptance criteria.

[alexeygrigorev]

Software Engineer handoff for issue #56

Implemented operator follow-up actions inside the existing task/workflow flow:

• Added task history/follow-up fields and action endpoints for mark waiting, follow-up sent, response received, unblocked, and resolve done.
• Blocked direct waiting -> todo/done generic status updates so operators must record the proper action/history.
• Added Operations Home and workflow controls for channel, note, next follow-up date, response received, follow-up sent, resolve done, and visible follow-up history.
• Extended portable export/validation for follow_up_channel and typed task_history with secret-safety validation.
• Added/updated API, dashboard, workflow, notification, export, and E2E coverage.

Changed files:

• work-engine/src/types.ts
• work-engine/src/router.ts
• work-engine/src/export/portable.ts
• work-engine/src/public/api.js
• work-engine/src/public/app.js
• work-engine/src/pages/index.html
• work-engine/tests/api-tasks.test.ts
• work-engine/tests/export-portable.test.ts
• work-engine/tests/notifications.test.ts
• work-engine/tests/home-dashboard.test.ts
• work-engine/tests/api-podcast-slice.test.ts
• work-engine/e2e/follow-up-actions.spec.js
• work-engine/e2e/podcast-operator-slice.spec.js

Verification run:

• npm --prefix work-engine install
• npm --prefix work-engine run typecheck
• NODE_ENV=test SKIP_AUTH=true node --import tsx --test tests/api-tasks.test.ts tests/notifications.test.ts tests/export-portable.test.ts tests/home-dashboard.test.ts
• npm --prefix work-engine test
• npm --prefix work-engine run build
• npm --prefix work-engine run test:e2e -- e2e/follow-up-actions.spec.js
• npm --prefix work-engine run test:e2e -- e2e/home-dashboard.spec.js e2e/podcast-operator-slice.spec.js
• npm --prefix work-engine run test:e2e -- e2e/podcast-operator-slice.spec.js
• npm --prefix work-engine run test:e2e -- e2e/home-dashboard.spec.js -g "Dashboard task rows save comment evidence from the queue"
• npm --prefix work-engine run test:e2e
• npm --prefix work-engine run export:data -- .tmp/exports/follow-up-actions
• npm --prefix work-engine run validate:export -- .tmp/exports/follow-up-actions
• npm --prefix work-engine run dry-run:import -- .tmp/exports/follow-up-actions
• git diff --check

Final full E2E result: 306 passed. Export/validate/dry-run import passed on the local execution export.

Screenshots captured in the worktree:

• .tmp/screenshots/issue-56-operations-home-follow-up-due.png
• .tmp/screenshots/issue-56-validation-error.png
• .tmp/screenshots/issue-56-workflow-waiting-history.png
• .tmp/screenshots/issue-56-resolved-unblocked-state.png

Remaining gaps: none known. One full E2E run had a timeout in an existing dashboard proof test; the focused rerun passed, and the subsequent full E2E run passed completely.

Ready for Tester.

[alexeygrigorev]

TEST PASS

Tester gate for #56 completed in /home/alexey/git/dataops/.tmp/worktrees/issue-56 on branch worktree-issue-56.

Acceptance criteria verification:

• PASS: Waiting/follow-up actions are exposed from Operations Home task rows and workflow detail waiting context. Verified by focused E2E and screenshots.
• PASS: Mark waiting, follow-up sent, response received/unblocked, and resolve-done are server-side action endpoints with validation and structured taskHistory records. Verified in api-tasks.test.ts and focused API subset.
• PASS: Follow-up sent requires channel, note, and next date; keeps status waiting; updates active followUpAt; history records previous/next dates. Verified by API tests and E2E.
• PASS: Response received/unblocked returns task to todo, clears active waiting/follow-up fields, and preserves history. Verified by API tests and E2E.
• PASS: Resolve-done uses explicit action and still enforces required link/file/artifact/comment/external-status/bundle-link gates. Verified by API proof-gate tests.
• PASS: Generic waiting task completion or waiting-to-todo update is blocked. Verified by API tests and disabled checkbox screenshots/flows.
• PASS: Task/workflow history is readable with action, timestamp, actor when available, channel, note, waitingFor, previous/next follow-up date. Verified in screenshot and tests.
• PASS: Portable export maps follow_up_channel and task_history, validates stable task/bundle/user relationships, rejects bad enums/dates/user refs/secrets. Verified by export unit tests plus validate/dry-run commands.
• PASS: Follow-up-due notification idempotency remains per task/date and does not reappear after resolving wait until a new cycle. Verified by notifications tests.
• PASS: Standalone task lifecycle and bundled workflow task lifecycle both covered. Migrated/intake-style waiting task with existing waiting metadata uses same controls in E2E.

Commands/evidence:

• node --check work-engine/src/public/api.js -> exit 0.
• node --check work-engine/src/public/app.js -> exit 0.
• node --check work-engine/e2e/follow-up-actions.spec.js -> exit 0.
• node --check work-engine/e2e/podcast-operator-slice.spec.js -> exit 0.
• git diff --check -> exit 0.
• npm --prefix work-engine test -> exit 0; 677 tests, 134 suites, 677 pass, 0 fail.
• npm --prefix work-engine run typecheck -> exit 0.
• npm --prefix work-engine run build -> exit 0.
• NODE_ENV=test SKIP_AUTH=true node --import tsx --test tests/api-tasks.test.ts tests/notifications.test.ts tests/export-portable.test.ts tests/home-dashboard.test.ts from work-engine/ -> exit 0; 145 tests, 26 suites, 145 pass, 0 fail.
• npm --prefix work-engine run export:data -- .tmp/exports/follow-up-actions-tester -> exit 0; local export valid shape, entity counts all 0 in the local table.
• npm --prefix work-engine run validate:export -- .tmp/exports/follow-up-actions-tester -> exit 0; valid: true, errors: [].
• npm --prefix work-engine run dry-run:import -- .tmp/exports/follow-up-actions-tester -> exit 0; valid: true, totalRecords: 0.
• npm --prefix work-engine run test:e2e -- e2e/follow-up-actions.spec.js -> exit 0; 1 passed.
• npm --prefix work-engine run test:e2e -- e2e/home-dashboard.spec.js e2e/podcast-operator-slice.spec.js -> exit 0; 35 passed.
• npm --prefix work-engine run test:e2e first run -> exit 1; 305 passed, 1 timeout in existing Dashboard task rows save comment evidence from the queue.
• npm --prefix work-engine run test:e2e -- e2e/home-dashboard.spec.js -g "Dashboard task rows save comment evidence from the queue" -> exit 0; 1 passed.
• npm --prefix work-engine run test:e2e final run -> exit 0; 306 passed.

Screenshots inspected:

• .tmp/screenshots/issue-56-operations-home-follow-up-due.png
• .tmp/screenshots/issue-56-validation-error.png
• .tmp/screenshots/issue-56-workflow-waiting-history.png
• .tmp/screenshots/issue-56-resolved-unblocked-state.png

Screenshot result: no 404/error page or broken follow-up flow observed. The resolved/unblocked screenshot includes the sticky header in the full-page capture, but the task state and history are readable and correct.

Artifact/status notes:

• package-lock.json is unchanged.
• Expected implementation source still untracked: work-engine/e2e/follow-up-actions.spec.js; it should be added when committing Add operator follow-up actions and history controls #56.
• Generated test artifacts are untracked and should not be committed: work-engine/e2e/.auth-state.json, work-engine/test-results/.last-run.json.
• .tmp/screenshots/ and .tmp/exports/follow-up-actions-tester are scratch evidence, not commit artifacts.

Ready for PM acceptance.

[alexeygrigorev]

ACCEPTED

PM acceptance for #56 from the operations-manager/user perspective.

Accepted because the implemented flow matches the groomed scope:

• Operations Home follow-up-due rows expose the same waiting action flow used in workflow detail, including channel, note, next follow-up date, response received, follow-up sent, and resolve done.
• Follow-up sent keeps the task waiting, updates the next follow-up date, refreshes the visible task/workflow state, and writes structured history.
• Response received/unblocked clears active waiting fields, returns the task to todo, and preserves history in the workflow/task context.
• Waiting tasks cannot be silently completed through generic completion; resolve done is explicit and still enforces the existing proof gates.
• History controls are visible in task/workflow flow and preserve typed events for audit/export.
• Portable export includes current waiting state and task_history/follow_up_channel, validates relationships and enum/date/user references, and rejects secret-like payloads.
• Tester provided PASS with full command evidence and required screenshots. I also spot-checked the API/export/notification subset locally: 81 tests passed.

Artifact/commit note: do not commit generated scratch artifacts such as work-engine/e2e/.auth-state.json, work-engine/test-results/, or .tmp outputs. The new implementation test work-engine/e2e/follow-up-actions.spec.js is source and should be included when committing.

Blockers: none.

Branch may be committed with Closes #56 once the source changes are staged and generated artifacts are excluded.