PRD: CodeMesh workspace control plane roadmap

[BramVR]

Problem Statement

CodeMesh is trying to become one coherent code workspace across machines and agents without replacing Git. The current MVP already has a local Project Registry, readiness checks, explicit hydration, and Agent Prep, but the next roadmap needs to turn that local control plane into a trustworthy agent and multi-machine workspace foundation.

From the user perspective, the pain is simple: every laptop, server, cloud agent, and temporary agent workspace has a slightly different view of the code tree, env setup, repo freshness, and local state. Agents are especially sensitive to this drift. If an agent starts from the wrong base, misses required env, or receives too much secret scope, the system has failed even if the clone technically succeeded.

The product must keep the core promises sharp: never lose a local change, never leak a secret, make stale state obvious before work starts, and keep existing Git/local filesystem tools working.

Solution

Build CodeMesh forward as a workspace control plane, not a sync engine. Deepen the Agent Prep wedge first, then add shared topology, safe env materialization, and Git-native lazy hydration in phases.

The immediate direction is to make agent handoff green-or-not-run: a prepared agent workspace must have a versioned Agent Run Contract, checkout provenance, readiness diagnostics, handoff docs, allowed env scopes, and declared run boundaries. Warnings such as dirty source checkout remain visible but do not make a temp-clone handoff unsafe by themselves.

Architecture should move toward fewer, deeper modules. The strong seams are Readiness Decision, Git Operations, Agent Run Contract, Command Result/Presentation, Workspace Manifest, Reconciliation, Machine Registry, Env Binding, Env Provider, Clone Strategy, Bootstrap, and Toolchain Readiness. These modules should provide leverage through small interfaces and locality for stale-start, secret-safety, topology, and CLI automation rules.

After the agent contract is solid, add a Git-backed Workspace Manifest that stores desired topology only, backed by local SQLite observed state on each machine. Then add private env bindings and agent-scoped env bundles, followed by explicit lazy clone/partial clone/sparse checkout mechanics. Virtual filesystem behavior remains last and opt-in.

User Stories

1. As a developer, I want codemesh agent prepare to produce a rich codemesh-run.json, so that an agent harness can verify exactly what workspace it received.
2. As a developer, I want the run contract to include checkout provenance, so that I can prove the agent started from the intended remote base commit.
3. As a developer, I want source checkout dirtiness to be visible as a warning, so that local work is not hidden from me.
4. As a developer, I want dirty source checkout warnings not to block temp-clone agent handoff, so that unrelated local changes do not prevent safe remote-base agent work.
5. As a developer, I want missing base, origin mismatch, invalid policy, fetch failure, path conflict, and secret-scope denial to hard-stop agent prep, so that agents never start from unsafe or ambiguous state.
6. As a developer, I want no broad --force escape hatch for agent prep, so that critical safety blockers cannot be casually bypassed.
7. As a developer, I want narrow readiness overrides when appropriate, so that exceptional cases are explicit and auditable.
8. As an automation author, I want stable --json output for agent-driving commands, so that harnesses can drive CodeMesh without scraping human output.
9. As an automation author, I want stable exit-code classes, so that scripts can distinguish warnings, readiness blockers, usage errors, and internal failures.
10. As a developer, I want doctor to preview the same readiness rules as Agent Prep, so that I can check handoff safety without creating a run.
11. As a stricter automation user, I want a mode where warnings fail, so that CI or scheduled checks can enforce tighter local hygiene.
12. As a developer, I want base selection to prefer explicit request, project policy, remote default, then main fallback, so that agents do not silently start from the wrong branch.
13. As a developer, I want Agent Prep to eventually work without a local source checkout, so that missing projects and fresh machines can still hand agents a correct remote workspace.
14. As a developer, I want Project identity to remain the normalized Git remote, so that the same repo can be recognized across paths and machines.
15. As a developer, I want clone URLs to remain separate from Project identity, so that each machine can fetch through the transport that works there.
16. As a developer, I want a Git-backed Workspace Manifest for desired topology, so that my canonical workspace can follow me using tools I already trust.
17. As a developer, I want local SQLite to remain observed machine state, so that readiness, scans, runs, and local facts do not pollute shared topology.
18. As a developer, I want manifest reconciliation to be explicit, so that CodeMesh never silently mutates working trees to match a remote manifest.
19. As a developer, I want manifest entries to be small per-project records, so that unrelated topology changes do not constantly conflict in Git.
20. As a developer with multiple machines, I want each machine to have a persistent local machine ID, so that hostname changes and ephemeral cloud hosts do not break identity.
21. As a developer, I want workspace roots to be machine facts, so that the same Desired Path can resolve under different local roots.
22. As a developer, I want bootstrap to create workspace structure and local registry entries without cloning everything by default, so that new-machine setup is fast and predictable.
23. As a developer, I want content hydration to remain explicit unless I opt in, so that CodeMesh does not surprise me with disk or network usage.
24. As a developer, I want path conflicts to block automation, so that CodeMesh never overwrites local files or untracked work.
25. As a developer, I want env requirements to be logical project requirements, so that repos can declare what they need without committing provider-specific secret locations.
26. As a developer, I want private env bindings to map requirements to providers, so that secret references stay outside repo-local project policy by default.
27. As a developer, I want agent-scoped env materialization first, so that agents receive only the env they are allowed to use.
28. As a developer, I want env bundles to live outside the Git working tree or be injected into a launched process, so that prepared checkouts do not become secret containers.
29. As a security-conscious user, I want run metadata to record env presence, scopes, and status without values, so that audits do not leak secrets.
30. As a developer, I want deterministic test providers before live secret providers, so that the env model can be proven without depending on external auth.
31. As a developer, I want 1Password, SOPS, and other providers to be integrations behind an interface, so that CodeMesh does not become a secrets manager.
32. As a developer, I want toolchain readiness to report and delegate, so that CodeMesh does not become an environment builder.
33. As a developer, I want clone strategy to be explicit, so that full clone, partial clone, sparse checkout, and object cache behavior are not surprising global magic.
34. As a developer, I want controlled hydration stubs only when an activation mechanism exists, so that fake directories do not masquerade as real checkouts.
35. As a developer, I want Git-native lazy clone and sparse checkout before a virtual filesystem, so that structure-before-content arrives without high data-loss risk.
36. As a future team user, I want shared topology without raw secrets, so that a team workspace can be shared safely.
37. As a future cloud user, I want workspace targets to receive manifests and scoped env policy, so that Coder, DevPod, Daytona, and cloud agents can boot into the same canonical workspace.
38. As an agent harness author, I want one command result shape behind human and JSON output, so that automation can rely on CodeMesh without coupling to terminal prose.
39. As a maintainer, I want Git process behavior behind one Git Operations module, so that base selection, clone strategy, checkout provenance, remote identity, and redaction do not drift across callers.

Implementation Decisions

• Keep the existing deep-module direction: Project Registry, Readiness, Agent Prep, and Env Readiness remain core behavior modules with thin CLI adapters.
• Deepen Readiness into a Readiness Decision module. It owns project readiness, agent handoff readiness, base selection inputs, blocker/warning semantics, strict mode, narrow readiness overrides, source-checkout warning treatment, and handoff green computation.
• Make status, tree, doctor, and Agent Prep consume the same Readiness Decision interface. Agent Prep should not reimplement path, dirty, remote, fetch, base, policy, or env diagnostics.
• Add a Git Operations module with process and test adapters. It owns Git command ordering, remote inspection, fetch, clone, checkout, remote default discovery, checkout provenance, clone URL redaction, and Git error normalization.
• Keep Project Registry responsible for Project identity and canonical workspace membership. Move duplicated remote normalization, clone URL cleanup, and credential redaction toward a single project identity/Git Operations seam so manifest work does not add a third implementation.
• Add or deepen an Agent Run Contract module that owns contract versioning, producer version, run identity, project identity, checkout provenance, readiness diagnostics, handoff docs, env requirement/scope metadata, policy source, toolchain status when checked, declared run boundaries, validation, redaction, JSON encoding/decoding, file writing, and list projections.
• Treat Agent Prep as the producer of an Agent Run Contract, not the owner of the contract schema. Agent Runs and State Store adapters should read the same contract shape rather than private subsets.
• Keep prepared-workspace freshness separate from source-checkout freshness. Source checkout dirty/stale signals are warnings unless strict mode promotes them.
• Add a Doctor command surface as a preflight over the same readiness rules as Agent Prep.
• Add a Command Result module that carries exit class, diagnostics, and command payload independent of presentation.
• Add Presentation adapters for human output and JSON output. Human output can evolve; JSON output is a stable contract.
• Establish stable exit-code classes for success, readiness/user-action failure, usage/config error, and internal/tool failure.
• Update base selection to use explicit request, repo-local policy, discoverable remote default branch, then main fallback.
• Keep Project identity as normalized Git remote. Treat clone URL, alias, manifest row ID, and local path as supporting fields, not identity.
• Introduce a Workspace Manifest module for desired shared topology only. The manifest stores project identity, alias, relative desired path, clone hints, and grouping.
• Keep Workspace Manifest storage/parse separate from Reconciliation. Workspace Manifest should be dumb desired topology; Reconciliation should decide drift and safe changes.
• Add a Reconciliation module that compares desired topology with observed local state and returns a drift plan before applying changes. Apply behavior must remain explicit.
• Keep observed state in the local State Store. Readiness results, dirty/stale state, agent runs, machine facts, project-intrinsic policy, env values, and secret values do not belong in the shared topology manifest.
• Store manifest projects as small per-project entries to reduce Git conflicts.
• Add a Machine Registry module around locally minted persistent machine IDs, mutable machine facts, workspace roots, optional target specs, and machine secret scopes.
• Define Bootstrap as an adapter over Workspace Manifest, Reconciliation, Machine Registry, and Project Registry. Bootstrap should output a plan first, create parent structure and local rows, and never clone content unless the user opts in.
• Preserve explicit hydration and path-conflict refusal. Do not create fake project directories without daemon, shell hook, or mount semantics.
• Add an Env Binding module that maps project-declared logical env requirements to private provider-specific secret references.
• Make Env Binding, not providers, the seam that owns scope intersection, missing binding diagnostics, materialization status, and no-secret-values guarantees.
• Add an Env Provider interface with deterministic fake provider support before live provider integrations.
• Make agent-scoped env bundles the first materialization target. Treat .env.local writing as an explicit opt-in feature.
• Record env materialization status and scope in the Agent Run Contract without raw values.
• Add a Clone Strategy module or policy shape that makes full clone, partial clone, sparse checkout, and object cache behavior explicit. Git Operations adapters execute the selected strategy.
• Add Toolchain Readiness as reporting/delegation only. CodeMesh integrates with environment tools but does not install or build environments itself.
• Define Toolchain Readiness declaration and result shapes before adding provider-specific detection. doctor and the Agent Run Contract should be able to include toolchain status.
• Keep VFS behavior out of the implementation path until Git-native lazy hydration has proven insufficient.
• Keep command docs and CLI help as the boundary for runnable features. Roadmap and ADR direction is not real command surface until the Command Catalog and help agree.

Testing Decisions

• Test module behavior through stable public interfaces and CLI behavior rather than implementation details.
• Extend unit tests around Readiness Decision for blocker/warning classification, strict mode, narrow overrides, base selection order, source-checkout warning behavior, missing source checkout behavior, and handoff green computation.
• Add tests proving status, tree, doctor, and Agent Prep consume the same readiness semantics.
• Add Git Operations tests using process and fake adapters. Cover remote inspection, fetch, clone, checkout, remote default discovery, checkout provenance, clone URL redaction, Git error normalization, and clone strategy selection.
• Extend unit tests around Project Registry for normalized remote identity, clone URL separation, manifest import/export, alias conflicts, local path movement, path conflict detection, and duplicate project identity locality.
• Extend Agent Run Contract tests for schema versioning, producer version, rich contract fields, validation, encoding/decoding, redaction, path-only handoff doc metadata, env scope metadata without values, list projection, and file/SQLite parity.
• Extend Agent Prep tests for producing the contract through the Agent Run Contract module, no source checkout support where manifest data is sufficient, no run creation on blockers, and correct cleanup on clone/write failures.
• Add Command Result and Presentation tests for JSON output schemas, human output, exit-code classes, strict mode, and readiness/user-action failure behavior across status, tree, hydrate, agent prepare, and doctor once implemented.
• Extend Env Readiness and Env Binding tests with fake providers, logical requirements, private bindings, allowed scopes, denied scopes, missing bindings, env bundle metadata, and secret leak checks.
• Add State Store tests for machine IDs, machine facts, manifest-related local state, migrations, and run contract metadata storage.
• Add Machine Registry tests for locally minted persistent IDs, mutable facts, workspace roots, target specs, and scope updates without manifest leakage.
• Add Workspace Manifest and Reconciliation tests for per-project entry round trips, relative desired paths, drift detection, no observed-state leakage, explicit apply behavior, and path conflict refusal.
• Add Bootstrap integration-style tests on a fresh temp machine root, including plan output, parent directory creation, registry row creation, missing projects, explicit hydrate, and no default cloning.
• Add Toolchain Readiness tests with fake declarations and fake detection adapters before live integration with mise/devbox/devenv/devcontainer checks.
• Extend e2e with isolated local Git fixtures, fake env values, fake providers, and no live network dependency by default.
• Preserve the existing secret-safety posture: command output, JSON reports, run metadata, local state bytes, and public docs must not contain fake or real secret values.
• Add tests that prove command docs and CLI help stay in sync when new commands or flags become runnable.

Out of Scope

• Replacing Git, Git history, source merges, or conflict resolution.
• Generic Dropbox-style arbitrary file sync.
• Syncing .git, dependency directories, build output, or caches through a generic transport.
• Becoming a secrets manager or storing raw secret values.
• Becoming Nix, devcontainer, mise, devbox, or devenv.
• Virtual filesystem behavior before inventory, status, manifest, env materialization, and Git-native lazy hydration are excellent.
• Hosted team service, SSO, audit/compliance product, or billing layer in this PRD.
• Fully enforcing agent run boundaries without an agent harness or future sandbox integration.
• Live provider e2e as the default test path; live checks can be added only when a provider integration needs proof.
• Final manifest file format details beyond small per-project entries; choose YAML/JSON/layout when module interface tests exist.
• Live 1Password/SOPS provider work before fake provider and Env Binding depth are proven.
• VFS integrity matrix work beyond preserving the VFS-last decision.

Further Notes

This PRD captures the roadmap clarified in the grill session and the accepted ADR direction through the Agent Run Contract, machine-readable CLI contracts, topology manifest, env bindings, machine identity, controlled hydration stubs, toolchain boundary, manifest entry layout, base selection, source-checkout independence, and VFS-last decisions.

An architecture review against the current codebase accepted these strong deepening opportunities: Readiness Decision, Git Operations, Agent Run Contract, Command Result/Presentation, Workspace Manifest plus Reconciliation, Machine Registry, Env Binding/Provider, Clone Strategy, Bootstrap, and Toolchain Readiness. The top recommendation is Readiness Decision because it unlocks doctor, strict mode, narrow overrides, JSON exit classes, and safer Agent Prep without waiting for manifest or env providers.

The PRD is intentionally broad. The next useful step is to split it into independently grabbable implementation issues, likely starting with Readiness Decision, Agent Run Contract v1, and machine-readable CLI contracts before manifest sync or env materialization.

Current tests before publishing the original PRD were green with make test and make e2e after the glossary and ADR pass.