Editor enhancements without CodeMirror — design reference and tracking

[BorisTyshkevich]

Design reference and tracking issue for evolving the SQL editor without adopting an editor library. Implementation is split across the child issues below, each a separate PR. Counterpart to #21 (CodeMirror 6 decision track).

What is the editor

src/ui/editor.js is hand-rolled and zero-dependency: a <textarea> overlaid on a syntax-highlighted <pre> with a line gutter, fed by a pure tokenizer in src/core/sql-highlight.js. It carries the 100% per-file coverage gate and adds nothing to the single served dist/sql.html artifact.

The <textarea> surface is the defining constraint — one caret, one selection, no in-text rendering (highlighting is faked by the <pre> underneath). That cleanly sorts features into buildable vs. not:

Feature	Textarea-friendly?
In-editor search/replace	✅ yes
Bracket matching + auto-close	✅ yes
Autocomplete (keywords/functions/schema)	✅ yes — data is the work, not the surface
Code folding	❌ can't hide lines in a textarea
Multi-cursor	❌ textarea has exactly one caret

Code folding and multi-cursor require a real editor surface and belong to #21.

Child issues

Issue	Title	Phase
#23	In-editor search/replace	1a
#24	Bracket matching and auto-close	1b
#25	Reference data loader + tokenizer dynamic-keyword API	2a
#26	Autocomplete dropdown	2b

#25 must land before #26. #23 and #24 are independent of each other and of Phase 2.

Design principles

The keystroke rule: never run SQL on the keystroke path.

1. Static reference data (system.keywords, system.functions, system.completions) → fetch once per connection, cache in memory, complete client-side. No per-keystroke SQL.
2. Discrete user actions → one network call per explicit action (Run, Format already work this way).
3. Live validation → debounced if ever wanted; validate-at-run is likely sufficient for a query console. No child issue filed — see "Open questions" below.

Overlay strategy for in-editor visuals (match highlights, bracket pair):
The <pre> overlay carries SQL token spans from renderHighlightInto. Rather than splitting those spans at highlight positions, a second absolutely-positioned <pre> with color: transparent carries only mark/highlight spans. Same scroll sync, same font metrics, zero interference with the token render path.

Resolved design decisions

Settled here so implementers don't have to re-derive them:

• Overlay highlights: second transparent <pre>, not span-splitting. renderHighlightInto stays untouched.
• Cmd+F registration: on the textarea keydown listener — the browser fires native find before a global handler in shortcuts.js can intercept it.
• Tokenizer API for dynamic keywords: tokenize(sql, { keywords, funcs } = {}) — backward-compatible optional second argument; existing callers are unaffected.
• Bracket wrap-selection: typing ( with text selected wraps it: (selected).
• {/} auto-close: excluded from Phase 1b (niche ClickHouse JSON context; defer).
• Double-quote auto-close: included in Phase 1b, same logic as single-quote.
• Column completions strategy: only already-loaded columns (table.columns !== null). No on-demand column fetch from the autocomplete path.
• Reference data cache: in-memory per session only in Phase 2a. localStorage deferred until server-version-keyed invalidation is designed.

Open questions

• Live squiggles (Phase 3): validate-at-run + local unbalanced-bracket check is probably enough for a query console. Optional live EXPLAIN AST debounced validation has not been filed as a child issue — needs a team call on whether it's wanted before implementing.
• system.completions context/belongs columns: enable position-aware completion. Worth exploiting but not required for the first autocomplete ship.
• system.documentation loading strategy for Phase 2c: load upfront alongside Phase 2a (increases connect time), or fetch on first hover per entity?

Relationship to #21

The main argument for CodeMirror 6 in #21 was schema-aware autocomplete. Phase 2 (issues #25–#26) delivers that on the current editor, server-powered and version-correct, with no second bundled dependency. CM6 still wins on editing mechanics a textarea cannot do (folding, multi-cursor); if those become priorities, adopt CM6 and retire issues #23 and #24 (CM6 provides them as extensions — don't double-build if CM6 is imminent).