research: can webjs db/test/typecheck run under bun zero-install?

[vivek7405]

Question

Can webjs db / test / typecheck run under bun zero-install (no node_modules), and how? This is the deep research for #704 after its first approach (a runner that imports the tool bin + a spawn pin preload) was found broken.

Verdict

The spawned dev tools (drizzle-kit, the bun test runner, tsc) fundamentally need a materialized node_modules; bun's auto-install + global cache + plugins cannot substitute. The reconciling fix is to auto-materialize node_modules via a fast bun install (cheap because the deps are already in bun's global cache) and run the tool from the app context (the existing resolveBin path), NOT the runner / preload / bun x approach. The server (dev / start) stays true zero-install via the pin shim; only the spawned tooling materializes.

Findings (verified empirically on bun 1.3.14)

1. bun x <tool> runs the tool from an ISOLATED temp dir (/tmp/bunx-.../node_modules/<tool>/), so drizzle-kit cannot resolve the APP's drizzle-orm. This fails EVEN WITH a full bun install present, so bun x was always the wrong tool, independent of zero-install. drizzle-kit must run from the app's own node_modules context.

2. The db failure is a version mismatch surfaced by auto-install. drizzle-kit@1.0.0-rc.3 requires drizzle-orm/_relations, which exists in drizzle-orm@1.0.0-rc.3 but NOT in drizzle-orm@0.45.2. npm latest is 0.45.2 (the rc line is a prerelease under a non-latest tag), so bun's bare auto-install resolves drizzle-orm to 0.45.2 and the subpath is missing.

3. Bun plugins (Bun.plugin onLoad / onResolve) do NOT intercept CommonJS require() resolution, only ESM imports. drizzle-kit's bin.cjs is CommonJS, so the pin shim cannot reach its internal require('drizzle-orm/_relations'). A bunfig preload DOES run under bun x (verified), but its onResolve never fires for the CJS require. onLoad on a cached CJS file also breaks it (returning { contents, loader } forces ESM parsing of a CJS module).

4. package.json overrides and bunfig [install.overrides] do NOT reach the bun x temp process, so they cannot pin the version there.

5. The fix works end to end. With the deps already in bun's global cache (zero-install DOES download to the global cache on disk), bun install materializes node_modules in ~57ms (measured). Running drizzle-kit from the app context then fully generates a real migration ([✓] Your SQL migration -> drizzle/<ts>_init/migration.sql). bun test (which has no auto-install at all) and tsc also work once node_modules exists.

Recommendation for #704 (separate implementation issue)

Drop the runner + the keystone-preload-for-tooling + the bin-import wiring (it cannot work). Instead: webjs db / test / typecheck under bun zero-install auto-materialize node_modules via a fast bun install, then run the existing resolveBin path. This is "no MANUAL install" (webjs does it transparently and fast), the honest reconciliation of the zero-install goal with the reality that CommonJS + esbuild tools and bun test need node_modules.

Tradeoff to document: after the first tool invocation node_modules exists, so the app is "installed" thereafter (everything keeps working; the user never typed an install command). The seamless progression is: start zero-install (instant dev), and the first time you need tooling, webjs materializes in tens of milliseconds.

Relates to #704, #684/#698 (the pin shim, which stays for the server), #669 (the zero-install vision), #718 (lockfile reproducibility, which the materialized lock would also serve).

[vivek7405]

Deep-dive refinement: the start.before cold-cache problem, and migrate-vs-generate

A key correction to "auto-materialize": webjs db migrate runs in start.before, BEFORE the app boots. On a FRESH deploy the global cache is COLD, so a bun install there is a slow network install before the server even starts, which defeats zero-install exactly where it matters (deploy). So auto-materialize is fine for DEV tools but wrong for the deploy-critical migrate.

This splits the db commands by what they actually need:

• webjs db migrate (deploy-critical, start.before): should NOT use drizzle-kit at all. Applying COMMITTED migrations is a runtime job that needs only drizzle-orm (which the server already auto-installs + pins under zero-install), not the drizzle-kit dev tool. Verified: drizzle-orm's runtime migrator RUNS under bun zero-install with no resolution error (drizzle-orm auto-installs, the pin resolves it). So migrate can be zero-install. OPEN DETAIL: drizzle-orm 1.0.0-rc.3 changed the migration on-disk format (a per-folder drizzle/<ts>_init/migration.sql + snapshot.json, NO meta/_journal.json), so the classic drizzle-orm/<dialect>/migrator (which reads _journal.json) applies nothing against rc.3 output (confirmed even WITH node_modules). The implementer needs rc.3's matching migrator API, OR webjs ships a tiny native applier (read the committed migration.sql files, track applied ones in a table, execute new ones via the db driver) which is fully zero-install and format-controlled.
• webjs db generate (dev): needs drizzle-kit (esbuild + the schema), so it materializes node_modules on demand (dev-time, where a one-time bun install is acceptable and the cache is usually warm).
• webjs db push / studio (dev): drizzle-kit, same as generate (materialize).
• webjs db seed: runs the app's own db/seed.server.ts under the runtime, so it is already zero-install via the server pin shim (no drizzle-kit).
• webjs test / typecheck (dev / CI): materialize, or run tsc self-contained.

Net: the ONE command that must work zero-install on a cold deploy (migrate) is exactly the one that does NOT need drizzle-kit. The dev tools that need drizzle-kit are exactly the ones where a fast materialize is acceptable. So the honest #704 design is: migrate via the runtime migrator (or a native SQL applier), dev tools via on-demand materialize.