cdktn-prhead provisioning can execute public npm release instead of PR-head artifacts

[sakul-learning]

Bug description

cdktn-prhead can report a successful PR-head sandbox while actually installing and executing the public npm release of cdktn-cli/cdktn instead of the freshly built local monorepo tarballs.

This makes PR-head e2e validation unsafe: tests can pass or fail against the already-published release, not the PR code under test.

There are two related failure modes:

1. Provisioning installs the wrong packages. The monorepo provisioning path publishes the local dist/js/*.tgz packages to the in-process Verdaccio registry, but the sandbox install can still resolve cdktn-cli and cdktn from public npm.
2. Sandbox execution does not fail closed. The generated manifest still labels the sandbox as cdktn-prhead, and the harness executes manifest.binPath, so tests silently exercise whatever package landed in node_modules. If the sandbox is manually repaired by installing local tarballs, manifest.json can also remain stale and continue to report the old public-release version.

Reproduction notes

While validating open-constructs/cdk-terrain PR #293, I provisioned a PR-head sandbox from the cdk-terrain worktree:

cd /data/repos/cdktn-e2e
CDKTN_MONOREPO=/data/repos/hermes-pr-reviewer/worktrees/cdk-terrain/open-constructs__cdk-terrain/pr-293 \
  pnpm provision cdktn-prhead

The provisioner built and packaged the PR monorepo, but the resulting sandbox manifest showed public npm versions:

{
  "id": "cdktn-prhead",
  "binPath": "/data/repos/cdktn-e2e/.sandboxes/cdktn-prhead/node_modules/cdktn-cli/bundle/bin/cdktn",
  "binName": "cdktn",
  "version": "0.23.3+prhead",
  "libPackage": "cdktn",
  "libVersion": "0.23.3"
}

That is not a PR-head install. In this case it meant the sandbox was executing the published cdktn-cli@0.23.3 bundle instead of the local 0.0.0 PR tarball.

Running the OpenTofu/provider-registry regression through that sandbox reproduced the old failure:

TERRAFORM_BINARY_NAME=/home/vincent/.local/share/mise/installs/opentofu/1.10.1/tofu \
node .sandboxes/cdktn-prhead/node_modules/cdktn-cli/bundle/bin/cdktn \
  get --force --output .gen --language typescript

with fixture config:

{
  "terraformProviders": ["hashicorp/awscc@~> 1.0"]
}

Actual error from the incorrectly provisioned sandbox:

Error: Could not find provider with constraint {"constraint":{"name":"awscc","fqn":"awscc","version":"~> 1.0","source":"hashicorp/awscc","namespace":"hashicorp"} ...}

After manually installing the locally built PR tarballs into the same sandbox:

npm install --no-fund --no-audit --legacy-peer-deps \
  /data/repos/hermes-pr-reviewer/worktrees/cdk-terrain/open-constructs__cdk-terrain/pr-293/dist/js/*.tgz \
  constructs@^10.6.0

node_modules/cdktn-cli/package.json became 0.0.0, the bundle contained the PR fix, and the same cdktn get command succeeded and generated .gen/providers/awscc/index.ts.

That confirms the e2e scenario is valid, but the default cdktn-prhead provisioning/execution path did not reliably run the PR-head artifacts.

Expected behavior

For cdktn-prhead:

• pnpm provision cdktn-prhead should install the freshly built local monorepo packages, not the public npm release.
• The installed cdktn-cli and cdktn versions should be the local PR-head package versions, currently 0.0.0.
• If the installed versions or package-lock resolutions are not local/PR-head, provisioning should fail immediately.
• Test execution should not silently run a cdktn-prhead manifest whose installed packages are public npm releases.

Actual behavior

• Provisioning completed successfully.
• manifest.json reported version: "0.23.3+prhead" and libVersion: "0.23.3".
• The harness would execute the cdktn-prhead manifest bin path even though it pointed at the public release installed in the sandbox.
• Manually overlaying local tarballs fixed the executed code but left the manifest stale, so the manifest is not a reliable source of truth after repair/debugging.

Suspected area

In scripts/provision.mjs, provisionMonorepo() publishes local tarballs and then installs bare package names:

await shAsync(NPM, ["install", "--no-fund", "--no-audit",
  "cdktn-cli", entry.libPackage, "constructs@^10.6.0"], sandbox)

The code comments expect Verdaccio to serve local 0.0.0 as latest, but observed behavior did not match that expectation.

Possible causes to check:

• the Verdaccio package rule for "cdktn**" may not fully prevent uplink/public npm resolution for cdktn-cli,
• npm may prefer latest metadata from the uplink/public release when the install spec is bare,
• the local registry may need explicit install specs such as cdktn-cli@0.0.0 and cdktn@0.0.0,
• or the provisioner should bypass registry resolution entirely and install the generated local tarballs directly.

Suggested fix

Make the PR-head path fail closed and self-verifying:

1. Install exact local packages, for example:

   npm install --no-fund --no-audit cdktn-cli@0.0.0 cdktn@0.0.0 constructs@^10.6.0

   or install the local dist/js/*.tgz tarballs directly.

2. After install, assert:

   • node_modules/cdktn-cli/package.json.version === "0.0.0",
   • node_modules/cdktn/package.json.version === "0.0.0",
   • package-lock.json resolutions for cdktn-cli/cdktn are local/Verdaccio or file tarballs, not https://registry.npmjs.org/...,
   • manifest.version and manifest.libVersion match the installed packages.

3. Consider adding a harness/runtime guard for id === "cdktn-prhead" so tests fail loudly if the manifest points at non-PR-head packages.

Why this matters

This bug can invalidate PR review/e2e results. In the PR #293 validation, the incorrectly provisioned sandbox reproduced a provider-source-matching bug that was already fixed in the PR. Only after installing the local tarballs did the same sandbox exercise the PR code and pass the OpenTofu shorthand-provider regression.