Version-exact reference docs in the editor from ClickHouse 26.6 embedded docs

[BorisTyshkevich]

Context

ClickHouse 26.6 makes reference documentation queryable from the running server. That lets SQL Browser show docs that match the exact ClickHouse binary the user is connected to, without fetching the public docs site and without version drift.

Relevant server-side sources:

1. system.functions structured docs

   • Modern ClickHouse exposes structured documentation fields on functions: description, syntax, arguments, parameters, returned_value, examples, introduced_in, categories, plus function metadata such as case_insensitive, alias_to, deterministic, and higher_order.
   • Today SQL Browser bulk-loads only name, is_aggregate, and optionally syntax for reference data/signature help, then fetches the first line of description lazily for hover docs.

2. system.documentation full embedded docs

   • Shape is 4 columns: name, type, description, source.
   • type is an enum with the documented entity kind. Current ClickHouse source/docs define 20 kinds, including functions, aggregate functions, table functions, table/database engines, data types, settings, formats, compression codecs, metrics, and system tables.
   • description is a complete Markdown document assembled from embedded docs: description, syntax, arguments, examples, introduced version, related items, setting type/default, system table columns, etc.
   • source is the repo-relative source file path when available.
   • Do not hard-code row counts or type counts as product logic. Counts vary by ClickHouse build, registered factories, runtime metrics, and server version.

3. Self-documenting per-kind system tables

   • Examples include system.table_engines, system.database_engines, system.data_type_families, system.formats, system.dictionary_layouts, system.dictionary_sources, system.disk_types, and aggregate-function-combinator docs.
   • These expose structured columns where supported, commonly description, syntax, examples, introduced_in, and related.
   • Important exception: system.formats does not have syntax; use description, examples, introduced_in, and related.

This issue upgrades editor documentation from a one-line hover to a real in-app, version-exact reference surface.

Decision: version-exact source of truth

Use the connected ClickHouse server as the primary docs source. The docs are embedded in the running binary, so they match the server the SQL Browser is actually querying. This also preserves the app's self-contained/airgapped deployment story: no external fetches and no dependency on clickhouse.com being reachable.

A public docs link may be shown only as a secondary latest-docs link when it can be derived safely. It is never the source of truth.

UX: progressive disclosure

Use glance -> depth:

1. Hover stays small

   • Signature/syntax when available.
   • One-line summary.
   • since vX badge when introduced_in is available.
   • Hint: Click for docs & examples.
   • The hover must remain transient and compact.

2. Click or shortcut opens a right-side docs pane

   • Full description.
   • Syntax.
   • Arguments / parameters.
   • Returned value.
   • Syntax-highlighted, copyable examples using existing src/core/sql-highlight.js.
   • introduced_in badge.
   • related / categories chips where available.
   • In-app navigation for related entries where resolvable.
   • Optional footer link: View latest on clickhouse.com ↗ when a safe URL is available.

3. Reuse existing UI patterns

   • The docs pane should follow the side-drawer/detail-pane pattern already used by cell detail and schema detail.
   • It should be closeable, keyboard-accessible, and not block editor typing unexpectedly.

Data-loading rules

Preserve the existing keystroke rule: no SQL while typing.

Reference data is loaded once per connection. Full documentation bodies and examples are loaded lazily and cached per connection.

Capability detection

Probe capabilities once per connection and degrade gracefully:

• system.functions rich docs columns available?
• system.documentation available with name, type, description, source?
• Per-kind structured docs available on tables such as system.table_engines, system.formats, system.data_type_families, etc.?

Implementation may probe via system.columns or by optimistic best-effort SELECT ... LIMIT 1 calls, but every probe must use the existing tryQueryData style: missing tables, missing columns, or denied privileges must not surface as user-visible query errors.

On older servers or denied access, fall back to today's behavior: function signature plus first-line hover where available, otherwise built-in reference data.

Loader contracts

Add explicit loader shapes instead of letting UI code know table details.

Suggested shape:

// Query failure / denied / transient error: null, not permanently cached.
// Successful no-match: { found: false }, cacheable for this connection.
// Successful match: { found: true, ... }.

loadHoverDoc(ctx, { kind, name }, sqlString) -> Promise<null | HoverDoc>

HoverDoc = {
  found: true,
  kind,           // 'function' | 'aggregate-function' | 'setting' | 'format' | ...
  name,
  title,
  signature,      // syntax or name() fallback where useful
  summary,        // first non-empty description line or compact derived summary
  introducedIn,   // optional
  categories,     // optional array
}

loadDocEntry(ctx, { kind, name }, sqlString) -> Promise<null | DocEntry>

DocEntry = {
  found: true,
  kind,
  name,
  title,
  source,         // optional system.documentation source
  summary,
  description,    // plain/markdown depending on path
  syntax,
  arguments,
  parameters,
  returnedValue,
  examples,
  introducedIn,
  related,        // array
  categories,     // array
  latestUrl,      // optional, derived safely
  renderMode,     // 'structured' | 'markdown-subset'
}

The app-level cache should key by connection + kind:name, and clear on reconnect just like the current hover-doc cache.

Entity routing

Add a pure token/context classifier in src/core/ so UI code can ask for docs without embedding SQL grammar heuristics.

Minimum routing:

• Function call / known function token -> function or aggregate-function.
• FORMAT <name> context -> format.
• ENGINE = <name> / schema tree engine inspect -> table-engine.
• Column type / schema tree type inspect -> data-type.
• SETTINGS <name> context -> setting.
• MergeTree settings context, where known -> merge-tree-setting.
• Ambiguous names should either prefer the strongest SQL context or open a small disambiguation state in the docs pane.

This classifier must be pure and unit-tested. It must not issue SQL.

Rendering strategy

Structured path first

Use structured columns where they exist:

• Functions / aggregate functions: system.functions.
• Table engines: system.table_engines.
• Database engines: system.database_engines.
• Data types: system.data_type_families.
• Formats: system.formats (no syntax column).
• Dictionary layouts/sources, disk types, combinators, etc. when supported.

Structured entries render directly; no Markdown parser required for those fields except where the server field itself contains Markdown fragments.

Markdown-subset path

Use system.documentation.description for broad coverage and fallback depth.

Implement a small renderer in src/core/, not a new dependency. It must build DOM/spec nodes, not HTML strings.

Supported subset:

• Paragraphs.
• ATX headings.
• Bold labels such as **Syntax**.
• Bulleted lists.
• Inline code.
• Fenced code blocks: sql, text, response, unknown language.
• Markdown links.

Safety rules:

• No innerHTML.
• No raw HTML rendering.
• No images.
• Escape all text by construction.
• External links must be explicit anchors with target="_blank" and rel="noopener noreferrer".
• Doc-relative links may be converted to latest clickhouse.com links, but they must not trigger fetches.

Files likely touched

File	Change
src/net/ch-client.js	Capability probes; richer system.functions row shaping; new loadHoverDoc / loadDocEntry; structured-table loaders; system.documentation fallback; graceful degradation.
src/core/completions.js or new src/core/doc-context.js	Pure token/context classifier for doc routing.
src/core/doc-render.js	Markdown-subset renderer to DOM/spec nodes; structured-doc normalization helpers.
src/ui/editor-intel.js	Keep hover compact; add since badge and click affordance; route click/shortcut to docs pane.
src/ui/doc-pane.js or src/ui/results.js	Right-side docs pane; highlighted/copyable examples; chips; related navigation; optional latest-docs footer link.
src/ui/app.js	App-level doc cache and openDocEntry wiring; clear cache on reconnect.
src/styles.css	Docs pane, hover since badge, chips, copy buttons.
tests/unit/*	Capability/row-shaping tests, classifier tests, Markdown-subset renderer tests.
README.md	Note richer docs require ClickHouse >= 26.6; older servers degrade gracefully.

Acceptance criteria

• On ClickHouse >= 26.6, hovering a documented function shows compact signature, one-line summary, optional since vX, and a click hint.
• Clicking a documented function opens a docs pane with full syntax, arguments, returned value, examples, introduced version, and categories where available.
• system.formats docs work without querying a nonexistent syntax column.
• system.documentation is handled as 4 columns: name, type, description, source.
• The code does not assume fixed row counts or a fixed subset of 15 documentation kinds.
• On ClickHouse < 26.6, missing columns/tables, or denied system.* privileges, the browser does not show errors and falls back to current behavior.
• No full docs/examples are loaded on the keystroke path.
• Full docs are loaded lazily and cached per connection.
• Markdown is rendered safely without innerHTML or a new runtime dependency.
• Examples use existing SQL highlighting and are copyable.
• Tests cover loader row shaping, fallback behavior, token/context classification, and Markdown-subset rendering.
• The single-file/self-contained artifact property is preserved.

Phased implementation

1. Phase 1: functions

   • Add rich function hover/pane using system.functions structured columns.
   • Keep old fallback behavior.

2. Phase 2: structured non-functions

   • Add docs for formats, table engines, database engines, and data types.
   • Add schema-tree inspect hooks for engine/type docs.

3. Phase 3: system.documentation breadth

   • Add Markdown-subset rendering.
   • Cover settings, MergeTree settings, server settings, table functions, codecs, metrics, system tables, and remaining documented kinds.

Non-goals

• Do not add a general Markdown dependency.
• Do not fetch clickhouse.com as the docs source.
• Do not replace autocomplete/signature-help architecture; extend its version-exact reference source.
• Do not run SQL per keystroke.
• Do not expand hover into a large scrollable tooltip.

[BorisTyshkevich]

This issue spans two layers, sequenced after CM6 (#21):

• Hover (compact signature/summary/since badge) → a CM6 hover extension in the editor.
• Right-side docs pane → as the issue says, reuse the side-drawer/detail-pane pattern. Extract a shared Drawer primitive here (cell detail + schema detail + this docs pane = its 3rd consumer) with focus/Esc/backdrop/ARIA, so the three stop being separate overlay systems.