research: respect full package.json (ranges + tree) under Bun zero-install (default auto-install)

[vivek7405]

Terminology correction (2026-06-24): this issue says "--no-install" in places, but that is the WRONG term. webjs's zero-install is default bun run dev with auto-install ON (no flag). The --no-install FLAG is the opposite (it DISABLES auto-install, for CI/prod strictness) and webjs does not use it. Read every "--no-install" below as "default auto-install / bun run dev zero-install". The research question is whether DEFAULT auto-install honors package.json ranges + the lockfile on a current (1.3.14) Bun.

Question

Make Bun zero-install (default bun run dev auto-install, no node_modules) resolve dependencies EXACTLY as npm install / bun install + node_modules would, for the FULL package.json: caret/tilde/x ranges, dist-tags, prerelease, peer + optional deps, npm: aliases, and the whole transitive tree, not just exact direct pins.

Background (verified in #684 + shipped in #685)

• Bun's run-path auto-install honors an EXACT inline specifier (import 'zod@3.22.4') but ENOENTs on a RANGE or dist-tag (import 'zod@^3' / 'zod@latest'). onResolve plugins are bypassed for auto-install; onLoad specifier-rewriting works (that is feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685).
• feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685 rewrites a declared dep's bare specifier to an inline EXACT version (from bun.lock exact, else an exact package.json pin). A caret/tilde RANGE WITHOUT a bun.lock is left BARE and resolves to LATEST. That is the gap.
• This blocks defaulting the scaffold (and bun create defaults to no install (zero-install onboarding); node unchanged #682) to --no-install: a range-based app with no bun.lock runs unpinned-latest.

Goal

A zero-install Bun app (no node_modules) runs the SAME version set that npm install / bun install + node_modules would, across the whole tree, verified differentially.

Avenues to investigate

1. bun install --lockfile-only (most promising): can Bun produce/refresh a bun.lock by RESOLUTION ONLY, with no node_modules? If yes, run it at scaffold/first-run (cheap), then feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685's exact-pin reads the lock and pins every declared dep. The lock IS npm-equivalent resolution output, so this closes the range gap with zero node_modules.
2. Transitive correctness: when auto-install fetches an exact dep (zod@3.22.4), does it resolve zod's OWN deps deterministically (per the lock / zod's manifest) or fetch them at latest? feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685 only pins the app's DIRECT imports (the onLoad skips node_modules/cache dep files). Verify whether a bun.lock under zero-install pins the FULL tree.
3. bun.lock + the feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685 rewrite end-to-end: does a present bun.lock + the exact-pin rewrite ALREADY achieve full npm-equivalence today? Test differentially against a node_modules install of the same package.json.
4. Boot-time resolve-once-and-cache: on first run with no lock, do a one-time resolution (lockfile-only or a registry query) writing a bun.lock / internal manifest, then pin from it. Trigger in the webjs-bun.mjs bootstrap or a webjs CLI step.
5. Custom resolver (heavy fallback): re-implement semver max-satisfying + registry version queries to resolve ranges ourselves. Re-implements a package-manager solver; last resort.

Edge cases the solution must cover

caret/tilde/x/*/||/hyphen ranges, dist-tags (latest/next), prerelease versions, peer + optional deps, dedup, npm: aliases, git/file/workspace specifiers (bare today), and the full transitive tree.

Acceptance

A no-node_modules Bun app resolves to the identical version tree a node_modules install would (differential test). Then --no-install-by-default (scaffold + #682) becomes safe.

Cross-references

#684 (prior research, concluded exact-only via the onLoad rewrite), #685 (shipped exact-only pin), #682 (closed; the --no-install default this unblocks), #675 (zero-install bootstrap), #687.

Forward-looking research task; the first investigation pass (the bun install --lockfile-only avenue) is being recorded as comments below.

[vivek7405]

Investigation pass 1 (sprite, Bun 1.3.14)

Avenue 1 confirmed viable, and it is the path. bun install --lockfile-only exists ("Generate a lockfile without installing dependencies") and produces a COMPLETE bun.lock with NO node_modules:

• package.json with debug: ^4.0.0 -> lock resolved debug@4.4.3 AND the transitive ms@2.1.3.
• package.json with debug: 4.3.4 -> lock resolved debug@4.3.4 + ms@2.1.2.
• node_modules was NOT created; only bun.lock was written.

So a resolution-only step (cheap, no node_modules) yields the full npm-equivalent resolved tree (ranges resolved to exact, transitive included). #685's existing exact-pin rewrite already reads bun.lock exact versions for DIRECT deps, so lockfile-only + #685 closes the range gap for direct deps with zero node_modules.

Inline-versioned auto-install works for packages WITH dependencies. Verified debug@4.3.4, chalk@4.1.2 (both have deps) resolve under zero-install, same as the dependency-less zod@3.22.4. An earlier run where debug@4.3.4 ENOENT'd was a flaky-sprite-network artifact (the same specifier resolved on a later run). So #685's inline rewrite is NOT broken for real deps (a feared regression, now cleared).

Still open (sprite network too flaky to settle here): when a lock-exact DIRECT dep is auto-installed under zero-install, does its TRANSITIVE dep resolve to the LOCK version or to latest? The discriminating test (debug@4.3.4 locks ms@2.1.2, latest is 2.1.3) kept ENOENT-ing on the transitive run. Needs a stable-network re-run. Two outcomes:

• If transitive honors the lock -> lockfile-only + feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685 already gives full npm-equivalence (the whole tree). DONE.
• If transitive resolves to latest -> the onLoad rewrite must also pin transitive specifiers from the lock's resolved tree (it currently skips node_modules/cache dep files), or rely on the fetched exact dep's own manifest.

Proposed solution sketch

1. A webjs step (scaffold create, or the webjs-bun.mjs bootstrap on first run when no lock) runs bun install --lockfile-only to produce/refresh bun.lock (resolution only, no node_modules).
2. feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685's exact-pin rewrite reads that lock and pins every declared dep to its exact resolved version.
3. (Pending the transitive answer) extend the rewrite or rely on Bun to pin the transitive tree from the lock.

This makes a no-node_modules app resolve to the same versions a full install would, and unblocks --no-install-by-default (scaffold + #682), since the lockfile-only step (not a full install) is what provides the pinning.

Next steps

• Re-run the transitive discriminator on stable network (settle the open question).
• If green, file the implementation (webjs-file-issue): wire bun install --lockfile-only into the scaffold/bootstrap + confirm feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685 pins from it; add a differential test (no-node_modules tree == node_modules tree).

[vivek7405]

Investigation pass 2: the transitive bound (the crux)

The remaining empirical question (does a lock-exact DIRECT dep's TRANSITIVE deps resolve to the lock version or latest-within-range under zero-install) could not be settled on the current sprite: the runtime auto-install fetch (bun --bun) flaked across ~13 retries this session, while bun install --lockfile-only succeeds (its installer has retry logic). Recording the reasoned analysis + the exact test to re-run on stable infra.

What is solid

• bun install --lockfile-only produces the FULL resolved tree (direct + transitive) in bun.lock with NO node_modules. So the resolution of ranges to exact versions, npm-equivalently, is available offline-of-node_modules.
• feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685's rewrite pins the app's DIRECT imports to the lock-exact version (the onLoad sees only app files; it does not rewrite a dependency's own internal imports, which live in the cache and are skipped).

The likely bound (needs one stable-network confirmation)

Prior #684 finding: Bun's RUNTIME auto-install ignores a present bun.lock (a bare import with a lock + no node_modules still fetched latest). If that holds for the transitive case, then under zero-install:

• DIRECT deps: pinned exactly (lockfile-only + feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685 inline rewrite).
• TRANSITIVE deps: resolved to latest-satisfying the depending package's own range, NOT the lock pin, because nothing rewrites the dependency's internal imports and the runtime resolver does not consult the lock.

If so, the achievable target under TRUE no-node_modules zero-install is: direct deps pinned to the declared/lock version; transitive deps at latest-within-range (i.e., equivalent to a LOCKLESS npm install, not a LOCKED npm ci). Exact transitive pinning (matching npm ci / a committed lockfile) appears to require either materialized node_modules or a Bun feature that makes the runtime auto-install honor the lockfile for the resolved subtree.

To verify on stable infra (decisive test)

1. package.json { "debug": "4.3.4" }; bun install --lockfile-only -> lock has debug@4.3.4 + ms@2.1.2 (latest ms is 2.1.3).
2. Clear the Bun cache; bun --bun import debug@4.3.4 (the feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685-rewritten direct specifier).
3. Inspect the cache: ms@2.1.2 => the lock IS honored for transitive (FULL equivalence achievable via lockfile-only + feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685). ms@2.1.3 => transitive is latest-within-range (the bound above).
   Also test a bunfig.toml [install] setting (e.g. a frozen/exact mode) to see if any flag makes the RUNTIME resolver honor the lock for the subtree.

Provisional conclusion + next step

• The lockfile-only + feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685 path closes the DIRECT-dep range gap with zero node_modules (high confidence). That alone makes a scaffold's OWN declared deps pin correctly, which is most of the bun create defaults to no install (zero-install onboarding); node unchanged #682 safety concern.
• Whether it reaches FULL (transitive) npm ci-equivalence hinges on the runtime-honors-lock question above. If it does not, document the bound (direct pinned, transitive latest-within-range) and treat materialized node_modules as the path for strict reproducibility (which is already what the prod Docker image does).
• Next: run the decisive test on stable infra; then file the implementation (webjs-file-issue): wire bun install --lockfile-only into the scaffold create / first-run bootstrap, confirm feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685 pins from it, add a differential test (no-node_modules tree vs node_modules tree), and document the transitive bound.

[vivek7405]

Pass 3: tried again (test-1, test-2, local /tmp); a non-reproducible lock-interaction blocks clean measurement

Retried the decisive transitive test with a network gate (resolve zod@3.22.4 first to confirm the runtime auto-install path is up). Result in one run: the gate PASSED (NET_OK=3.22.4, network up) but debug@4.3.4 still ENOENT'd in the same run, with no ms fetched.

This is NOT simply network: the variable that differs from the earlier run where debug@4.3.4 DID resolve (the bareinline test) is that here the dep is declared in package.json AND a bun.lock is present (from bun install --lockfile-only), whereas the working run was a bare ad-hoc inline import with no manifest entry and no lock. So there appears to be an inconsistent interaction between an inline-versioned specifier and a present lock/manifest at runtime that this environment (sprite network + sandbox) will not reproduce cleanly across runs. zod (declared, no lock, in the #685 E2E) pinned fine; debug (declared, WITH a lock) ENOENT'd here.

Implications to characterize on stable infra (these matter beyond the transitive question):

1. Does a present bun.lock + a feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685 inline specifier reliably resolve, or can the lock/inline interaction ENOENT? If the latter is real and consistent, it would affect feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685 itself for apps that commit a bun.lock (not just this research). The feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685 E2E only covered package.json-without-lock, so this is an untested combination worth a dedicated check.
2. The original transitive question (lock-honored vs latest-within-range) still needs the clean-cache fetch observation, which depends on the runtime auto-install path being reliably up.

Status: the reproducible finding stands (bun install --lockfile-only -> full no-node_modules lock; direct-dep pinning via lockfile-only + #685). The runtime-resolution characterization (transitive + the lock/inline interaction) needs a stable environment where bun --bun auto-install fetches reliably. Decisive tests are documented above; deferring the empirical pass rather than drawing conclusions from a non-reproducible setup.

[vivek7405]

CONCLUSION (research complete): not achievable with current Bun. A lockfile and zero-install are mutually exclusive.

Decisive, reproducible test (Bun 1.3.14 sprite, network confirmed up the whole run, hardcoded cache, same chalk@4.1.2):

setup	runtime auto-install
bun.lock present (from bun install --lockfile-only), bun --bun	ENOENT
bun.lock present, bun --bun --install=force	ENOENT (the flag does not override)
bun.lock deleted, bun --bun	works (resolves chalk + transitive ansi-styles@4.3.0, latest-in-range)

The ONLY variable is the lockfile's presence. So: a present bun.lock disables Bun's runtime auto-install (Bun treats a lockfile as "dependencies should be materialized in node_modules" and refuses to fetch on the fly), and no --install mode overrides it. This reproduces the earlier debug ENOENT (t7) cleanly.

Why the goal is not achievable with current Bun

Respecting package.json RANGES exactly (like a locked npm install) requires a lockfile to record the resolved versions. But a lockfile disables zero-install auto-install. So you get one or the other, never both:

1. Zero-install (NO lock): deps resolve to latest-satisfying-the-range (direct AND transitive; e.g. chalk@4.1.2 -> ansi-styles@4.3.0). Works with no node_modules, but is NOT pinned/reproducible. feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685's inline rewrite can pin a dep that has an EXACT package.json pin (no lock needed); a caret/tilde RANGE resolves to latest-in-range.
2. Pinned (WITH lock): requires bun install to materialize node_modules (the lock turns off auto-install). Reproducible, but NOT zero-install.

There is no third mode. Avenue 1 from the opening (lockfile-only + #685) is therefore a DEAD END: the bun.lock it produces is exactly what disables the zero-install run.

Implications

• The original ask (--no-install respects package.json EXACTLY like npm install + node_modules) is not possible on current Bun. It is a Bun engine constraint, not a webjs one.
• bun create defaults to no install (zero-install onboarding); node unchanged #682 stays closed: defaulting the scaffold to --no-install cannot be made range-pinned. Install-by-default (node_modules + lock = reproducible) remains correct; zero-install stays a dev-convenience that runs latest-in-range.
• feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685 is unaffected and correct: under zero-install it pins EXACT package.json pins (no lock); its bun.lock-exact path is operative only when node_modules is present (where the rewrite is inert anyway), so it never hits the lock-disables-autoinstall trap. A bun.lock + no-node_modules app is broken for ANY import (with or without feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685), a pre-existing Bun limitation, not something feat: pin Bun zero-install deps via an onLoad specifier-rewrite from package.json #685 introduces.
• Production stays bun install (materialized, pinned image) regardless, which was already the design.
• Future: revisit only if Bun adds a runtime mode that honors a lockfile for on-the-fly resolution (none today; --install=force does not).

Closing as completed: the question is answered (not achievable with current Bun; the two modes above are the bound).

[vivek7405]

RETRACTION: the conclusion above was tested on Bun 1.3.11, which has bugs that 1.3.14 fixes. Re-test required.

The prior conclusion ("a lockfile disables zero-install; ranges ENOENT; not achievable") was measured on Bun 1.3.11 (sprite test-1's version at the time; bun upgrade confirms 1.3.14 is the current latest). Bun 1.3.14 fixes bugs directly in the paths I was exercising, so those results are unsafe:

• "a use-after-free in the runtime auto-install path where enqueueDependencyToRoot could read from a dangling pointer into the lockfile's dependency buffer after it was reallocated by fromNPM" (the lock + runtime-auto-install path, i.e. my "lock present -> ENOENT").
• "auto-install resolution corruption where resolving a package's subpath (e.g. nanoid/non-secure) could fail with 'Cannot find module' after auto-installing a second package, due to cached directory info pointing at a reused threadlocal buffer". My probes imported pkg/package.json (a SUBPATH) and ran several packages per script, the exact trigger. This explains the run-to-run inconsistency.

Bun's DOCUMENTED auto-install algorithm (bun.com/docs/runtime/auto-install): check bun.lock (use its pinned version), else scan up for a package.json (use the declared semver range), else latest. That is the INTENDED behavior, and it would mean ranges + lockfile ARE honored under zero-install. It matches a second agent's description that the prior pass wrongly "refuted" on the buggy 1.3.11.

What must be re-tested on 1.3.14 (with corrected probes)

Re-run the decisive matrix on 1.3.14, and do NOT import pkg/package.json subpaths (that tripped the fixed subpath bug). Import the package main, one package per process, clean cache between:

1. package.json range (zod: ^3.20.0) + bun.lock present (from bun install --lockfile-only), no node_modules: does zero-install resolve to the LOCK version (honored) or ENOENT?
2. package.json range, no lock: resolves to latest-in-range?
3. --no-install with a cache pre-seeded at a satisfying version: resolves from cache (the other agent's claim) or "Cannot find module"?
4. transitive: a range-transitive dep, lock present: lock version or latest-in-range?

If the documented algorithm holds on 1.3.14, the feasibility verdict flips toward YES (lockfile-only / lock + range honoring), which would unblock #682 and reshape #685's guidance.

Status: conclusion RETRACTED; re-test on 1.3.14 pending (the sprite auto-install network was in a sustained outage during this pass). Issue reopened.