markdown

package
v0.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 13, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Package markdown is the lossless markdown import/export surface over the editor_blocks dataset (internal/editor). It exists for LLM tools, "Export as .md" / "Import .md" UI flows, and programmatic API users that don't want to walk the block tree manually.

The package contains:

  • Split / Join: a CommonMark-ish block splitter and its inverse joiner. Pure string→[]string transforms, no SDK access.
  • ParseBlock / RenderBlock: the typed view — convert a raw markdown block into the {type, style, text} shape stored on the editor_blocks dataset, and the inverse rendering. Inline-only `text` (no block-level syntax inside).
  • Set / Get / List: helpers that read existing top-level blocks via the blocks package, diff against the supplied markdown, and emit per-record create/update/delete ops on the editor_blocks dataset. PUT /v1/.../markdown is one HTTP wrapper around Set; other clients can call it directly.

All splitting, parsing, and diffing is caller-side — the SDK's per-record CRDT handler (in internal/editor) is what enforces the block-shape on apply.

Index

Constants

View Source
const DefaultSimilarityThreshold = 0.5

DefaultSimilarityThreshold is the cutoff for "edit vs delete + insert" inside a gap. Below this score the pair is treated as unrelated. Tuned to be permissive — small typo corrections should be matched as edits — without being so loose that wholly different blocks merge.

Variables

This section is empty.

Functions

func Join

func Join(blocks []string) string

Join is the inverse of Split: it renders block entries back to one canonical markdown document.

Content blocks are separated by a single blank line. An empty entry is an empty paragraph and adds one blank line beyond that separator; at either edge of the document, where there is no separator to build on, it adds one blank line of its own — and a trailing run needs one extra newline to survive splitLines eating the terminator.

Round-trip is idempotent — Split(Join(blocks)) yields back the same blocks (whitespace inside a block is preserved).

func RenderBlock

func RenderBlock(b ParsedBlock) string

RenderBlock produces the canonical markdown bytes for a ParsedBlock. Inverse of ParseBlock for the canonical-output case — input `"+ foo"` parses as `{list_item}` and renders as `"- foo"`; setext headings parse as `{heading, level}` and render as ATX. Round-trip equivalence holds for the common cases used by the e2e suite (paragraphs and ATX headings).

func Split

func Split(md string) []string

Split breaks raw markdown into block-records.

A "block" is one of:

  • Front matter: line 1 exactly `---`, YAML-shaped lines (fmScan), a line exactly `---` or `...`; kept verbatim, at the very start of a document only. Otherwise those lines read as the blocks below.
  • ATX heading (`# foo` … `###### foo`) — single line.
  • Setext heading (text line followed by `===` or `---` underline).
  • Fenced code block (``` … ``` or ~~~ … ~~~), opening fence to matching closing fence inclusive. Content kept verbatim.
  • Blockquote: any run of consecutive `>`-prefixed lines, including blank `>` lines that only have the marker. The whole run is one block.
  • List item (`-`, `*`, `+`, `\d+.`, `\d+)`): each item is its own block. Continuation/lazy lines indented under the marker are part of the same item.
  • HTML block (line starting with `<`): consecutive non-blank lines.
  • Table (line starting with `|`): consecutive non-blank lines.
  • Paragraph: anything else, until a blank line or a different block opens.
  • Empty paragraph: a blank line not acting as a block separator, returned as an empty entry.

One blank line between two content blocks is the plain separator; every blank line beyond it is one empty paragraph. A run at either edge of the document has no separator duty, so all of its lines are empty paragraphs — including a document that is blank throughout. splitLines still treats a single trailing newline as a terminator, so content that merely ends in "\n" gains nothing.

The returned strings carry no trailing newline. Join reconstructs a canonical markdown document; round-trip is idempotent — Split(Join(Split(x))) == Split(x).

Types

type AmbiguousMatchError

type AmbiguousMatchError struct {
	Index       int
	Occurrences int
}

AmbiguousMatchError: an edit's OldText occurs more than once without ReplaceAll.

func (AmbiguousMatchError) Error

func (e AmbiguousMatchError) Error() string

type BlockTooLargeError added in v0.3.0

type BlockTooLargeError struct {
	Index int // the block's position in the parsed document
	Bytes int
}

BlockTooLargeError: a block's text is over the editor's per-block cap. The markdown writes refuse it before writing anything; the editor would reject it mid-change and leave the rest of the save applied.

func (BlockTooLargeError) Error added in v0.3.0

func (e BlockTooLargeError) Error() string

type ConflictError added in v0.3.0

type ConflictError struct {
	Current Document
}

ConflictError is Set's answer to an ifVersion the document has moved past. Current is the latest body and version read.

func (ConflictError) Error added in v0.3.0

func (e ConflictError) Error() string

type DiffResult

type DiffResult struct {
	NewSeq  []NewOp
	Deletes []int
}

DiffResult is the full plan returned by Diff. NewSeq aligns 1:1 with the new block slice. Deletes is the set of old indices that were neither kept nor updated, in ascending order.

func Diff

func Diff(oldBlocks, newBlocks []string) DiffResult

Diff plans the per-block ops needed to transform old into new.

Two passes:

  • Pass 1 anchors: Myers-LCS-on-xxh3-hashes finds blocks that are byte-identical between old and new. Those become Keep ops at their new positions.

  • Pass 2 gap-fill: in each gap between anchors (and at the edges), pair unmatched old blocks with unmatched new blocks by similarity score (sergi/go-diff Levenshtein-like ratio). Pairs above the threshold become Update ops; leftover olds go to Deletes; leftover news go to Insert ops at their new positions.

func DiffWithThreshold

func DiffWithThreshold(oldBlocks, newBlocks []string, threshold float64) DiffResult

DiffWithThreshold is Diff with a caller-supplied similarity cutoff for the gap-fill phase. Useful for tests.

type Document added in v0.3.0

type Document struct {
	Content string
	Version string
}

Document is a rendered body and the version it was read at.

Version is opaque to callers: the space's rebuild generation, the highest _applySeq across the object's records in the collection (tombstones and nested blocks included), and a hash of the body. Any write to the collection moves it. A rebuild of the SDK store renumbers _applySeq under a new generation, and the hash keeps a version from matching a different body if the sequence ever rewinds without one, as after restoring an older store backup.

func Get

func Get(ctx context.Context, sp space.Space, objectId, collection string) (Document, error)

Get returns the full markdown content of objectId by rendering each top-level block and joining with "\n\n", plus the version it was read at. Round-trip is canonical: blank lines between two content blocks are exactly one plus one per empty paragraph between them, and block-type canonicalisation (setext → ATX, `+` → `-`) applies the same way as on Set.

type Edit

type Edit struct {
	// OldText must be non-empty and, unless ReplaceAll is set, occur
	// exactly once in the rendered document.
	OldText string
	// NewText replaces OldText. Empty deletes the matched text.
	NewText    string
	ReplaceAll bool
}

Edit is one targeted replacement against the rendered markdown — the caller quotes text, never lines or block ids.

type NewOp

type NewOp struct {
	Kind   OpKind
	OldIdx int
}

NewOp is one entry in DiffResult.NewSeq, walked in new-document order. OldIdx is the index into the old block slice for Keep and Update, and -1 for Insert.

type NoMatchError

type NoMatchError struct {
	Index int
}

NoMatchError: an edit's OldText was not found, even fuzzily.

func (NoMatchError) Error

func (e NoMatchError) Error() string

type OpKind

type OpKind uint8

OpKind classifies the per-position outcome of a diff. The new document is described by a NewSeq of length len(newBlocks); each entry says how that position's text is sourced.

const (
	// OpKeep — old block kept verbatim; same lexid, no DB write.
	OpKeep OpKind = iota
	// OpUpdate — same lexid as the matched old block, text replaced.
	OpUpdate
	// OpInsert — new block, needs a fresh lexid allocated.
	OpInsert
)

type OverlapError

type OverlapError struct {
	IndexA int
	IndexB int
}

OverlapError: two edits matched intersecting regions.

func (OverlapError) Error

func (e OverlapError) Error() string

type ParsedBlock

type ParsedBlock struct {
	Type  string
	Style map[string]any // nil when no style metadata applies
	Text  string         // inline markdown only (no block-level syntax)
}

ParsedBlock is the typed view of one raw markdown block produced by Split. It mirrors the on-the-wire shape of a editor_blocks record (minus the SDK-managed id and _ver) so the markdown.Set path can diff and emit ops directly.

func ParseBlock

func ParseBlock(raw string) ParsedBlock

ParseBlock classifies a raw block from Split into a typed ParsedBlock with the splitter's own recognisers. Front matter depends on position, so parseBlocks types it.

type SetResult

type SetResult struct {
	// Inserted is the id of every newly created block, in document
	// order.
	Inserted []string
	// Updated is the id of every block whose fields were replaced.
	Updated []string
	// Deleted is the id of every block that was tombstoned.
	Deleted []string
	// Unchanged is the count of blocks that were kept verbatim — no
	// DB write touched them.
	Unchanged int
	// Version is the document version of the body as saved (see
	// Document). Set fills it; EditContent and Append leave it empty.
	Version string
}

SetResult bundles the per-call summary returned by Set, so callers that want to log / surface what changed can inspect it without re-running the diff.

func Append

func Append(ctx context.Context, sp space.Space, objectId, collection, content string) (SetResult, error)

Append parses content into blocks and appends them all after the object's current last top-level block, in a single ModifyBatch. It never reads the existing block bodies and never diffs — only the tail position is looked up (one indexed `-nav.pos` query via editor.MaxPos) — so the cost is O(appended content), independent of how large the document already is.

This is the append-only fast path for whole-document round-trips: callers like the agent debug-log collector grow a page turn-by-turn and would otherwise pay Set's O(document) GET+diff on every append, making a full run O(N²) in page size. Append makes each call O(chunk) and the run O(N).

Semantics differ from Set in two ways the caller must accept:

  • No diffing. Append is purely additive: every parsed block becomes a new record. It cannot update or delete existing blocks, and it will happily create a block identical to an existing one.
  • No separator control. A fragment is positioned by the append itself: its blocks simply follow the current last one, and blank lines wrapping the fragment are framing, not content, so Split's leading/trailing empty paragraphs are dropped here. Empty paragraphs BETWEEN blocks of the fragment are kept, as they are for Set.

Empty (or blank-only) content is a no-op that returns a zero result.

func EditContent

func EditContent(ctx context.Context, sp space.Space, objectId, collection string, edits []Edit) (SetResult, error)

EditContent applies targeted replacements to the rendered markdown of objectId — the surgical alternative to Set for callers that know the text they want changed but not the block ids. Edits resolve against the current rendering and the result goes through Set's diff pipeline, so a one-block change lands as one record op and a stale quote fails (NoMatchError) instead of clobbering concurrent edits. All-or-nothing: any unresolvable edit aborts before any write; byte-identical output is a no-op. The write is conditional on the read the edits resolved against, like Set's, is redone against a fresh read when another writer landed in between, and is written unconditionally when other writers keep landing first.

func Set

func Set(ctx context.Context, sp space.Space, objectId, collection, content, ifVersion string) (SetResult, error)

Set replaces the markdown content of objectId with content, operating over the editor_blocks dataset.

Pipeline: parse content as a document (front matter on line 1), diff it against the stored top-level blocks by each block's own rendering (planDiff), and write creates, updates and tombstones as one change.

content is stored as INLINE markdown inside each block's `text` field — no full-block markdown bytes survive on disk. Empty content tombstones every existing top-level block. Blank lines beyond the one separating two blocks become empty paragraph records, so a document's vertical spacing survives the round trip (see Split).

A non-empty ifVersion makes the write conditional: unless the document is still at that version, Set writes nothing and returns a ConflictError carrying the current body and version.

The diff is written as one change, conditional on the collection being unchanged since the read it was computed from, so no other writer — a markdown write, a …/blocks call, another device — lands in between; the reported Version is the saved body's. Without ifVersion, a write that lost that race is redone against the new state, and written unconditionally when other writers keep landing first (writeAgainstRead).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL