Import DataTasks into work-engine

[alexeygrigorev]

Import DataTasks into work-engine

Status: pending
Tags: migration, work-engine, testing, P0
Depends on: #12
Blocks: #6, #8, #9, #15

Scope

Import the existing DataTasks codebase from ../datatasks into work-engine/ in this repository while preserving its current TypeScript service, local development commands, tests, fixtures, templates, bundle behavior, recurring work behavior, file handling, user handling, notifications, and Playwright coverage.

The implementation should treat work-engine/ as a transitional standalone Node/TypeScript subsystem inside the merged DataOps repo. Make only the path, package, configuration, and documentation changes needed for it to install, build, run, and test from work-engine/ without requiring changes to ../datatasks.

Record the imported source revision according to the import-log/source-commit policy from #12. If #12 is not complete yet, this issue should wait rather than invent a competing import-log convention.

Acceptance Criteria

• DataTasks source is copied into work-engine/ from ../datatasks without modifying the source repository.
• work-engine/ includes the runtime source, package files, TypeScript configs, tests, Playwright config, scripts, docs, fixtures, templates, and local development assets needed for standalone operation.
• Node package scripts still work from inside work-engine/, including npm test, npm run typecheck, npm run build, npm run start, npm run dev, and npm run test:e2e where local dependencies are available.
• Path, package, and environment assumptions are updated so the imported code does not require hard-coded references back to ../datatasks.
• The import records the source repository and source commit/revision according to Define import log and source commit policy #12.
• Existing task, template, bundle, recurring, file, user, notification, storage, and cron test coverage remains present after the import.
• Focused Playwright coverage for task and bundle flows passes from work-engine/, or any environment-only blocker is documented with the exact failing command and reason.
• The implementation notes clearly identify any follow-up work deferred to portal integration issues instead of mixing it into the import.

Test Scenarios

Scenario: Engineer validates the imported work engine

Given: a fresh checkout of DataTalksClub/dataops with DataTasks imported into work-engine/
When: an engineer runs npm install, npm test, npm run typecheck, and npm run build from work-engine/
Then: the commands complete successfully without reading or writing files in ../datatasks.

Scenario: Engineer exercises task and bundle browser flows

Given: the imported work engine has its local dependencies installed and its test server can start
When: an engineer runs the focused Playwright specs for task lists, bundle detail, templates, and instantiation from work-engine/
Then: the flows pass against the imported code and use fixtures/configuration that live inside work-engine/.

Scenario: Operator starts the local work engine

Given: the imported work engine has local environment defaults configured for development
When: an operator starts it with the preserved local start or dev command from work-engine/
Then: the DataTasks UI/API starts locally and exposes the existing task, template, bundle, recurring, user, file, and notification behavior.

Out of Scope

• Rewriting DataTasks into Python or merging it into the portal backend.
• Designing the final DataOps workflow data model; that belongs to Define the first workflow data model #15.
• Adding task-to-process-doc IDs or process-doc resolver behavior; that belongs to Add task-to-process-doc links #6.
• Building Podcast workflow behavior or the first end-to-end Podcast slice; those belong to Define the Podcast workflow #8 and Build the Podcast end-to-end slice #9.
• Changing deployment infrastructure beyond what is necessary to keep the imported work engine runnable locally.
• Modifying ../datatasks or any other source repository.

[alexeygrigorev]

Grooming complete. Scope is limited to importing DataTasks into work-engine/ and preserving its standalone TypeScript build, local dev flow, unit tests, and focused Playwright coverage. Follow-on portal model/linking work remains in the dependent issues.

[alexeygrigorev]

PM triage / acceptance decision: #4 is satisfied by existing work and current verification.

Evidence checked:

• work-engine/ now contains the imported DataTasks runtime source, package files, TypeScript configs, docs, scripts, tests, fixtures, Playwright config, local development assets, and Lambda packaging files.
• _docs/import-log.md records the DataTasks source path ../datatasks, source repo git@github.com:alexeygrigorev/datatasks.git, and source commit 9adbc21da3a4db3990a58965950513c618abcda0 per the Define import log and source commit policy #12 policy.
• The source repo was not modified by this triage; ../datatasks is still at 9adbc21da3a4db3990a58965950513c618abcda0 with only the already-known local e2e/.auth-state.json source-worktree modification.
• Search for hard-coded ../datatasks references in work-engine/ found none. Remaining datatasks strings are package/localStorage/source-doc identity references, not source-repo path dependencies.
• Existing coverage remains present for tasks, templates, bundles, recurring work, files, users, notifications, storage, cron, export/import safety, and auth under work-engine/tests/, plus Playwright coverage under work-engine/e2e/.

Verification run from the current repo state:

• npm --prefix work-engine test passed: 621 tests, 621 pass, 0 fail.
• npm --prefix work-engine run typecheck passed.
• npm --prefix work-engine run build passed.
• npm --prefix work-engine run test:e2e -- e2e/task-list.spec.js e2e/bundle-detail.spec.js e2e/api-templates.spec.js e2e/api-instantiation.spec.js passed: 54 tests, 54 pass.
• timeout 8s npm --prefix work-engine run start started successfully and printed Dev server listening at http://localhost:3000; timeout is expected because this is a long-running server command.
• timeout 8s npm --prefix work-engine run dev started successfully and printed Dev server listening at http://localhost:3000; timeout is expected because this is a long-running watch server command.

One local false alarm during verification: I first ran start and dev concurrently, which made dev fail on the embedded .data/LOCK. Rerunning dev alone passed startup, so this is not a product blocker.

Follow-on portal model/linking work remains correctly out of scope for this issue and is tracked by dependent issues such as #6, #8, #9, and #15. Closing #4 as completed.