Unnest: formula expression parser (AST)

[jimmytacks]

Part of #269 — spec 0009 (specs/0009-unnest-to-let.md).

The foundation and the main risk area: the repo's first recursive-descent Excel-formula parser, producing an expression tree (AST). Pure engine, no UI. Every existing tool (RefactorEngine, CellRefExtractor, LetParser) is token/regex-based and works only because none of them need structure; step extraction does.

Parse the canonical US formula that Formula2 returns (comma separators, US function names) — we dodge locale ;-separator variants. We never evaluate: structure need only be good enough to identify call/operator nodes and keep leaves opaque. Be conservative on genuinely ambiguous corners (union/intersection) as long as lexing of strings/brackets/refs is rock-solid.

Must handle:

• Function calls with comma-separated arg lists; empty args SUM(A1,,A3); nested calls.
• Operators with correct Excel precedence/associativity: ^, unary +/-, * /, + -, &, comparisons (= <> < <= > >=), reference operators (: range, space intersection), postfix % and #, prefix @.
• Atomic leaves kept opaque (never descended, never a step): cell refs, structured table refs, named ranges, literals, array constants.

Grammar source-of-truth

• XLParser (Spreadsheet Lab / Felienne Hermans) — closest C# reference grammar for parse-not-evaluate; use as grammar + edge-case reference. Check licence before vendoring any of it.
• ECMA-376 / ISO 29500 Part 1, Formulas section — formal ABNF.
• Microsoft "Using structured references with Excel tables" — authoritative for the bracket syntax.

Edge-case coverage checklist (tests must cover)

Structured / square-bracket refs (priority — brackets are NOT operators, must stay opaque):

• t[City] single column
• t[[X-Coordinates]:[Y-Coordinates]] column range
• [@Column] / t[@Column] current-row specifier
• t[[#Headers],[City]] special-item + column
• t[#All] t[#Data] t[#Headers] t[#Totals] t[#This Row]
• column names with spaces / escaped specials ('[ '] '# '@) inside brackets

Cell refs:

• $A$1, mixed $A1/A$1
• Sheet1!A1, quoted 'My Sheet'!A1
• 3-D Sheet1:Sheet3!A1
• external [Book.xlsx]Sheet1!A1
• full col/row A:A, 1:1
• spill A1#

Operators / structure:

• comma overload: argument separator vs union (A1:A3,B1:B3) (resolved by paren context)
• intersection (space), postfix %, prefix @ implicit intersection
• precedence/associativity chains (2^3^2, a-b-c, -A1^2)

Literals / constants:

• array constants {1,2;3,4} with mixed number/boolean/string/error
• error literals #N/A #REF! #DIV/0! #VALUE! #NAME? #NULL! #NUM! #SPILL! #CALC!
• numbers incl. scientific 1.5E-3, leading-dot .5
• strings with "" escapes; single-quoted sheet/workbook qualifiers skipped wholesale

Functions / misc:

• _xlfn.-prefixed future functions; LET / LAMBDA calls
• embedded newlines / whitespace inside formulas

Golden round-trip corpus

• Parse every formula in the lambda library (*.tests.yaml / lambda definitions) and assert the AST re-serialises identically — strongest guard against real-world surprises.