Skip to content

Browser-native Bitcoin node: architecture and plan #1

@melvincarvalho

Description

@melvincarvalho

Summary

Grow bitcoin-kernel from a consensus test runner into a browser-native Bitcoin node: validate Bitcoin's consensus rules in a tab, persist the chain to OPFS, and sync through a thin local bridge. Usable as a wallet in minutes; fully validated in the background within hours.

The hard part, the consensus engine, already exists in this repo: it validates headers, difficulty, blocks, scripts and SPV proofs against real test vectors, with real ECDSA/Schnorr. This issue captures the architecture, an honest performance model, and the milestones to get from here to "node".

Product idea: usable first, trustless later

Start at the chain tip and work backwards so the user has a working wallet almost immediately, then harden to full validation in the background. This stacks two proven designs:

  • Neutrino (BIP 157/158) for the instant wallet: headers-first to tip, then walk backwards pulling compact block filters and only downloading the few blocks that match your addresses.
  • assumeutxo for the background hardening: load a hash-checked UTXO snapshot, be usable in minutes, and validate genesis to snapshot in the background.

Trust ladder (the UI MUST be honest about which rung the user is on):

  1. Headers only, trust accumulated proof of work (most-work chain)
    • compact filters / SPV, light-wallet trust: find and verify your own coins
    • assumeutxo snapshot, trust one hash temporarily, usable now
    • background backfill complete, fully validated and trustless

Note: the wallet scan goes backwards (cheap via filters), but consensus validation goes forwards from a snapshot, because the UTXO set only exists by applying blocks in order. Two motions, stacked.

Architecture: three pillars

  1. Storage, OPFS. The enabler is createSyncAccessHandle() in a Web Worker (synchronous file I/O). Holds the ~650 GB of flat block/undo files, headers, filter data, and periodic UTXO snapshots. Requires navigator.storage.persist() so the browser does not evict it. IndexedDB is a trap (async, per-op overhead) and must not be in the hot path.
  2. Compute, WASM. WASM-SIMD libsecp256k1 for signature verification, one verifier per core via Web Workers, ideally zero-copy over SharedArrayBuffer (needs COOP/COEP headers, which we control). Multi-buffer SIMD SHA256 for hashing. The UTXO set lives in RAM (a custom compact hash map), not OPFS, with OPFS used only for durable snapshots.
  3. Networking, a local shim. A browser tab cannot open raw TCP to port 8333. A small local WebSocket-to-TCP bridge relays bytes to real peers; the tab still validates everything itself, so it stays trustless. The engine already speaks p2p over WebSocket (ws.js/p2p.js).

What already exists (engine)

  • Consensus validation: headers, difficulty (incl. testnet4 timewarp), blocks, transactions, scripts (Bitcoin Core's vectors), SPV merkle proofs.
  • Real verifyEcdsa / verifySchnorr (not stubs).
  • p2p.js, node.js, ws.js (p2p over WebSocket), filters.js (BIP158 compact filters), storage, mine.js, wallet.js.
  • The whole thing is pure ESM, zero-dependency, importable as bitcoin-kernel, and already runs validation client-side (the demo).

What is missing / to build

  • WASM-SIMD secp256k1: the engine's crypto is pure JS today (fine for the 1,191 test vectors, ~20-100x too slow for IBD). This swap is the prerequisite for everything below.
  • OPFS storage backend: flat block/undo store + UTXO snapshot format + crash recovery.
  • In-RAM UTXO set: compact hash map in WASM/SharedArrayBuffer, plus a signature cache.
  • Local TCP shim: ~100-line WebSocket-to-TCP bridge, runnable on the same machine.
  • assumeutxo: snapshot load, then background backfill genesis to snapshot.
  • Wallet path: headers-first sync + backwards filter scan + your-coins UTXO/history.
  • Long-run host: installed PWA / persistent worker so a multi-hour sync is not a casual tab.

Performance model (modeled, treat as +/- 2x until benchmarked)

All numbers assume the WASM-secp swap is done and the chain data is sourced fast. Hardware target: a high-end desktop (16-32+ cores, 64 GB RAM, 4 TB NVMe).

Mode Time to usable Trust
Light wallet (Neutrino) minutes PoW headers + filters
assumeutxo snapshot minutes, backfill behind one hash, temporarily
assumevalid (skip ancient sigs) ~2-3 h a recent checkpoint
Full verify-everything, ~1.3x native (future WASM wide-arith) ~3-6 h nothing
Full verify-everything, ~2x native (today's WASM-SIMD) ~5-10 h nothing

Why: ~2.5 billion signature verifications dominate, but parallelized across cores they are well under ~1.5 h; once sigs are fast the bottleneck shifts to parsing 650 GB, hashing, and UTXO apply (~2-4 h). 64 GB RAM keeps the whole UTXO set resident, removing the I/O bottleneck that usually dominates native IBD.

Speed ceiling, honest: a browser node is ~1.3-2x slower than native and cannot beat it at equal work. Native libsecp uses MULX/ADX and SHA-NI instructions that WASM cannot reach; the only future lever that closes most of the gap is the WASM wide-arithmetic proposal (add-with-carry, 128-bit multiply), which would take secp toward ~1.1-1.3x. We win by doing less work (assumevalid/assumeutxo), not by out-running C.

Milestones

  • M0, benchmark: WASM-SIMD secp256k1 verify throughput in workers on real blocks. Confirms the whole model. Nothing else is worth building until this number is real.
  • M1, headers in OPFS: headers-first sync to tip via the local shim, validate all PoW + difficulty, persist to OPFS. Honest, impressive, sidesteps signatures.
  • M2, light wallet: BIP158 filter scan backwards from tip, build a wallet's UTXO/history, verify inclusion via SPV. Usable wallet.
  • M3, assumeutxo: load a snapshot, full block validation forward from it (sigs via WASM workers, UTXO in RAM). Usable immediately, validating new blocks live.
  • M4, backfill: validate genesis to snapshot in the background; flip the UI from "trusted snapshot" to "fully validated".
  • M5, durability/PWA: persistent storage, installable, survives restarts, resumes.

Open questions / unknowns

  • OPFS at ~650 GB is the least battle-tested part: quotas, eviction, fragmentation, behaviour at scale.
  • Real WASM-secp throughput in workers (the model's biggest uncertainty).
  • TCP shim: peer discovery, eclipse resistance, trust framing when blocks come from few sources.
  • Tab/worker lifetime across a multi-hour sync.
  • Timeline of the WASM wide-arithmetic proposal in V8.

Non-goals (at least initially)

  • Archival node (we prune; we do not serve historical blocks to others).
  • Mining, mempool/relay policy.
  • Matching native speed (the honest target is a small constant factor off native, in a tab).

Honest headline: "usable in minutes via assumeutxo, fully validated from genesis in an afternoon, in a browser tab." Not "instant full node," not "replaces Bitcoin Core."

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions