engine

Overview ¶

Package engine's argument-shaping layer: how a call's operands are reduced to the values a builtin receives — a scalar slot, a flattened range, or the mixed shapes the recurring parameter specs describe.

The clipboard block: how a copied range travels as plain TSV text. ParseBlock is the one definition of how that text becomes a grid again; the paste half of the model lives in paste.go.

The text folds: CONCAT and TEXTJOIN, whose operands flatten from ranges, so a single argument may be a whole column. Split from builtins_text.go for size; they share the byte budget that keeps a fold from building what it would then refuse.

Package engine's regex family: the three REGEX* builtins over Go's RE2 syntax, kept apart from the plain text builtins because a bad pattern is a value error rather than a text operation.

Comparison, where the two languages part company three ways.

This language compares by value and by type: text against text by code point, a boolean as 1 or 0, and a mixed comparison not at all. Excel compares everything, in a total order that puts every number below every text and every text below every boolean, and its text comparison ignores case.

So the same three-operator expression can read differently for three unrelated reasons, and each is announced only where the two orderings actually produce different answers — which is far less often than the shapes suggest. `=TRUE<0` is FALSE under 1/0 folding and FALSE under Excel's ranking; the reasons differ and the answer does not, so there is nothing to tell anybody.

Package engine's compute entry points: the public Compute family, and the rendering that turns a computed value matrix back into a text grid.

Windowed evaluation (spec 016 area 6b): ComputeRows runs the one resolver over a sparse budgeted memo with dependencies read lazily through the block cache. The capability type and its census policy live in windowed.go.

View directives: the `#.` lines that declare what a viewport does with the grid — which rows and columns are hidden, which carry headers, which stay anchored while the rest scrolls (SPECIFICATION §3).

A directive is a projection of the grid, never an input to it: nothing here is reachable from a formula, and deleting every directive leaves the data byte-identical and computing identically. The line is a TSV row, so a key and its value are ordinary fields; only the value is a language, parsed by internal/tsvt against the shared grammar.

Package engine's lazy-dispatch families: the calls whose arguments are NOT evaluated eagerly — clock readings, conditionals that must not evaluate the untaken branch, kind inspectors, and the text family.

The Excel divergences, announced.

Five of this language's behaviours differ from Excel deliberately (SPECIFICATION §5.2): `^` binds tighter than a unary sign and associates to the right, where Excel does the opposite of both; arithmetic and comparison refuse text where Excel coerces it; and text comparison is case-sensitive where Excel's ignores case. Each is a defensible choice, and each is a place a spreadsheet user's muscle memory produces a value they did not mean.

So the language says so. Check reports every occurrence at authoring time, naming what Excel would do and what to write instead. A divergence a tool announces is a lesson; one it hides is a bug.

Two constraints keep this honest, because an advisory nobody trusts is worse than none:

• Nothing is reported unless it is provable from the text alone. Whether a *reference* holds text is a runtime fact, so `A1="yes"` is not flagged — it diverges only for some contents, and a checker that fired on the most common formula in any spreadsheet would train its reader to ignore it. Where the operands are literals, the divergence is proven, not guessed.
• Authored parentheses silence the precedence findings. `-(A1^2)` is the remedy this file recommends; flagging it would make the advice impossible to take. See tsvt.Binary.IsGrouped.

The findings are advisory: the sheet computes exactly as it did before they existed. The point is that the author is told once, while writing, instead of by a wrong total months later.

The source-preserving document layer: a Document is a Sheet plus the file's physical line layout, so comment and shebang lines — which ReadTSV drops from the grid — survive editing and serialization. Every frontend that writes a .tsvt back out goes through Document.Text; no frontend serializes a grid (capability engine-doc-text).

Directives on a document: reading the view a sheet declares, and keeping the declarations true through a structural edit (SPECIFICATION §3, work order 009).

Only a directive line whose value actually moved is rewritten. Every other line — prose, a comment, a directive the edit did not touch — comes back byte-identical, because a rewrite must say exactly what the edit did and no more.

Duplicating a row or a column.

A duplicate is a fill of one line onto a new one, plus the grid shift an insert performs — so it shares fill.go's rebasing machinery and differs in what it does to the grid's shape. Keeping it here makes that difference the subject of its own file: fill.go changes cells inside the grid it was given, this one changes how many lines the grid has.

The sheet edit language (work order 011): an edits document is a TSV stream of semantic operations applied to a Document as a pure left fold. Comment lines are metadata (`#.base`) or prose; every data line is one op mirroring a Document method, its arguments verbatim TSV fields in the language's own address forms. An Edits value round-trips verbatim: Text returns the exact bytes parsed.

The op table: one parser per wire op.

edits.go owns the envelope — how a batch is framed, revision-checked and folded over a document. This file owns what each line MEANS: the operand grammar of every op, and the table that is the edit surface itself. The two change for different reasons, which is why they are separate files: adding an op touches only this one, and changing how a batch is applied touches only the other.

Fill and duplicate: the copy-time half of the reference model (spec 007). Fill copies one cell across a target span with Excel fill semantics — each unpinned reference coordinate shifts by the target's offset from the source, `$`-pinned coordinates hold — and DuplicateRow/DuplicateCol compose the existing insert with a one-line fill, so a whole line duplicates with its references rebased and the rest of the grid shifted exactly as inserts already shift it. Like the structural quartet (structural.go), everything is an AST transform re-serialized through the canonical renderer.

Writing a computed number.

Every number a formula produces passes through here on its way into a cell, and the cell is a document: what is written gets committed, diffed, pasted back in, and read by the next tool. So the rule is not "make it look nice" — it is "write text that names the value it came from".

Binary floating point makes those goals pull apart. 0.1+0.2 is genuinely 0.30000000000000004, and printing all seventeen digits reports an artefact of the representation as though it were information. Trimming to fifteen significant digits fixes that, and is what every spreadsheet does. But the trimming has to be applied as a decimal PLACE count: rounding the value and re-expanding it writes 1152921504606850000 for 2^60, which is a different number wearing four digits the value never had. Losing a fraction is a bounded, explicable loss; inventing an integer digit is not.

TSV serialization for the sheet engine: reading a .tsvt grid into a Grid of raw cell strings and writing a computed Grid back out. The A1 reference resolution and formula evaluation live in model.go and eval.go; this file is just the tab-separated line format. (The package doc is in model.go.)

Authored parentheses.

The parser records that an author wrote parentheses (tsvt.Binary.IsGrouped), and this is where that record is honoured. It is not a formatting preference: grouping is how the author said which reading was meant, and the Excel-divergence checker treats a grouped power as the explicit form that needs no diagnostic. If a rewrite dropped the parentheses, the checker would re-report a divergence the author had already resolved — the tool undoing the fix it asked for. So parentheses survive parsing, rewriting, and rendering.

Package engine's import shaping: the strict rectangle rules each IMPORT* result must satisfy. A shape mismatch is #IMPORT!, never a best-effort salvage — a silently reshaped import would put wrong numbers in a grid that looks right.

Package engine's JSON decoder: a document text becomes the immutable jsonNode tree the JSON family navigates. Member order is preserved because jsonset must round-trip a document without reordering it.

Package engine's JSON path layer: the dotted/indexed path syntax (a.b[0].c) compiled to steps, and the walk that resolves them against a document.

Package engine's JSON renderer: a jsonNode tree back to compact JSON text, preserving member order and number literals exactly as they were read.

The lazy cell backing a windowed evaluation reads through (spec 016 6b): on-demand block reads with a shared failure latch, and the sparse-memo computer construction that carries it.

Materialization: the one scan-and-read path every document size shares (spec 016). A source is indexed with the engine's own single definitions — scanLine terminates lines, IsCommentLine classifies them — and its grid rows are parsed into cells through the strided block reader. A fully materialized sheet is simply this path with an unbounded cache; the windowed and lazy modes to come are the same path under policy, never a second implementation.

The compute pass's memoization: phases for cycle detection, the memo seam, and its two forms — dense slabs for the resident pass, a budget-bounded sparse map for the windowed pass (spec 016 area 6b).

Package engine is the tsvsheet spreadsheet engine: a .tsvt file IS the sheet — a TAB-separated grid whose cells are literal values or `=formulas`, and whose formulas address other cells by A1 reference (`B2`, `D2:D4`) exactly like a conventional spreadsheet. It parses the grid, evaluates every formula in dependency order (memoized, with cycle detection), and emits the computed grid. The expression sublanguage and its AST come from internal/tsvt (the ANTLR-generated parser); this package resolves A1 references and evaluates.

This is the implementation behind the thin public github.com/tsvsheet/go-tsvsheet facade, which re-exports engine's surface unchanged; callers use that package.

Package engine's operator layer: the arithmetic and comparison semantics behind the §5.2 binary operators, kept apart from the function registry that dispatches named calls.

What the other spreadsheet would compute.

A divergence is only worth announcing when the two languages actually answer differently, and for an expression built from literals that is decidable: evaluate it under this language's reading and under Excel's, and compare. The alternative — matching the shape of the expression — announces `=+2^2`, `=-2^3` and `=-0^2`, all of which read the same in both, and a checker that cries wolf is one whose reader stops reading.

Two details make the comparison honest rather than merely arithmetic:

• An overflow is an error, not a number. `=200^200` is #NUM! here and in Excel, so two readings that both overflow AGREE even though the floats they overflow to (+Inf and -Inf) differ. Comparing raw floats reported `=-200^200` as divergent when both languages show the same error.
• The error propagates from the inside out. `=0^200^200` is #NUM! here because 200^200 overflows before the outer power is reached, while Excel groups it as (0^200)^200 and gets 0 — a real divergence that comparing only the final values missed, because 0^(+Inf) is 0.

Paste: the paste half of the copy-time reference model (spec 010). Paste places a clipboard block (block.go) with every formula rebased by the single delta target−origin, using the same AST transform Fill applies per target (fill.go). Like the rest of the family, everything re-serializes through the canonical renderer.

Pasting into a selection.

paste.go places a block once, at a point. This file answers the other question a spreadsheet asks of a clipboard: what happens when the target is a SELECTION rather than a cell. The block tiles the span when it divides it exactly, lands once at the top-left when it does not, and the span itself is bounded — the work and the allocation are O(area) while the request is four integers, so an unbounded selection would be an out-of-memory the caller has no way to catch.

Package engine's structural-edit algebra: how a row or column insert/delete maps every existing line index to its new one, expressed as pure transforms so the reference rewriting and the grid surgery share one definition.

The census policy and the windowed document (spec 016 area 6): a source of any size opens through the one indexed path; the census decides which capability comes back. At or under the resident budget the caller gets the same fully resident Sheet Parse builds — editable, computable, byte-for-byte today's contract. Over it the caller gets a WindowedSheet: view/compute-only by ruling, serving bounded row windows through the block cache while the source stays on disk. One implementation either way; the type is the policy made visible.