Documentation
¶
Overview ¶
Package markdown is the lossless markdown import/export surface over the editor_blocks dataset (internal/editor). It exists for LLM tools, "Export as .md" / "Import .md" UI flows, and programmatic API users that don't want to walk the block tree manually.
The package contains:
- Split / Join: a CommonMark-ish block splitter and its inverse joiner. Pure string→[]string transforms, no SDK access.
- ParseBlock / RenderBlock: the typed view — convert a raw markdown block into the {type, style, text} shape stored on the editor_blocks dataset, and the inverse rendering. Inline-only `text` (no block-level syntax inside).
- Set / Get / List: helpers that read existing top-level blocks via the blocks package, diff against the supplied markdown, and emit per-record create/update/delete ops on the editor_blocks dataset. PUT /v1/.../markdown is one HTTP wrapper around Set; other clients can call it directly.
All splitting, parsing, and diffing is caller-side — the SDK's per-record CRDT handler (in internal/editor) is what enforces the block-shape on apply.
Index ¶
- Constants
- func Get(ctx context.Context, sp space.Space, objectId, collection string) (string, error)
- func Join(blocks []string) string
- func RenderBlock(b ParsedBlock) string
- func Split(md string) []string
- type AmbiguousMatchError
- type DiffResult
- type Edit
- type NewOp
- type NoMatchError
- type OpKind
- type OverlapError
- type ParsedBlock
- type SetResult
- func Append(ctx context.Context, sp space.Space, objectId, collection, content string) (SetResult, error)
- func EditContent(ctx context.Context, sp space.Space, objectId, collection string, edits []Edit) (SetResult, error)
- func Set(ctx context.Context, sp space.Space, objectId, collection, content string) (SetResult, error)
Constants ¶
const DefaultSimilarityThreshold = 0.5
DefaultSimilarityThreshold is the cutoff for "edit vs delete + insert" inside a gap. Below this score the pair is treated as unrelated. Tuned to be permissive — small typo corrections should be matched as edits — without being so loose that wholly different blocks merge.
Variables ¶
This section is empty.
Functions ¶
func Get ¶
Get returns the full markdown content of objectId by rendering each top-level block and joining with "\n\n". Round-trip is canonical: blank lines between two content blocks are exactly one plus one per empty paragraph between them, and block-type canonicalisation (setext → ATX, `+` → `-`) applies the same way as on Set.
func Join ¶
Join is the inverse of Split: it renders block entries back to one canonical markdown document.
Content blocks are separated by a single blank line. An empty entry is an empty paragraph and adds one blank line beyond that separator; at either edge of the document, where there is no separator to build on, it adds one blank line of its own — and a trailing run needs one extra newline to survive splitLines eating the terminator.
Round-trip is idempotent — Split(Join(blocks)) yields back the same blocks (whitespace inside a block is preserved).
func RenderBlock ¶
func RenderBlock(b ParsedBlock) string
RenderBlock produces the canonical markdown bytes for a ParsedBlock. Inverse of ParseBlock for the canonical-output case — input `"+ foo"` parses as `{list_item}` and renders as `"- foo"`; setext headings parse as `{heading, level}` and render as ATX. Round-trip equivalence holds for the common cases used by the e2e suite (paragraphs and ATX headings).
func Split ¶
Split breaks raw markdown into block-records.
A "block" is one of:
- ATX heading (`# foo` … `###### foo`) — single line.
- Setext heading (text line followed by `===` or `---` underline).
- Fenced code block (``` … ``` or ~~~ … ~~~), opening fence to matching closing fence inclusive. Content kept verbatim.
- Blockquote: any run of consecutive `>`-prefixed lines, including blank `>` lines that only have the marker. The whole run is one block.
- List item (`-`, `*`, `+`, `\d+.`, `\d+)`): each item is its own block. Continuation/lazy lines indented under the marker are part of the same item.
- HTML block (line starting with `<`): consecutive non-blank lines.
- Table (line starting with `|`): consecutive non-blank lines.
- Paragraph: anything else, until a blank line or a different block opens.
- Empty paragraph: a blank line not acting as a block separator, returned as an empty entry.
One blank line between two content blocks is the plain separator; every blank line beyond it is one empty paragraph. A run at either edge of the document has no separator duty, so all of its lines are empty paragraphs — including a document that is blank throughout. splitLines still treats a single trailing newline as a terminator, so content that merely ends in "\n" gains nothing.
The returned strings carry no trailing newline. Join reconstructs a canonical markdown document; round-trip is idempotent — Split(Join(Split(x))) == Split(x).
Types ¶
type AmbiguousMatchError ¶
AmbiguousMatchError: an edit's OldText occurs more than once without ReplaceAll.
func (AmbiguousMatchError) Error ¶
func (e AmbiguousMatchError) Error() string
type DiffResult ¶
DiffResult is the full plan returned by Diff. NewSeq aligns 1:1 with the new block slice. Deletes is the set of old indices that were neither kept nor updated, in ascending order.
func Diff ¶
func Diff(oldBlocks, newBlocks []string) DiffResult
Diff plans the per-block ops needed to transform old into new.
Two passes:
Pass 1 anchors: Myers-LCS-on-xxh3-hashes finds blocks that are byte-identical between old and new. Those become Keep ops at their new positions.
Pass 2 gap-fill: in each gap between anchors (and at the edges), pair unmatched old blocks with unmatched new blocks by similarity score (sergi/go-diff Levenshtein-like ratio). Pairs above the threshold become Update ops; leftover olds go to Deletes; leftover news go to Insert ops at their new positions.
func DiffWithThreshold ¶
func DiffWithThreshold(oldBlocks, newBlocks []string, threshold float64) DiffResult
DiffWithThreshold is Diff with a caller-supplied similarity cutoff for the gap-fill phase. Useful for tests.
type Edit ¶
type Edit struct {
// OldText must be non-empty and, unless ReplaceAll is set, occur
// exactly once in the rendered document.
OldText string
// NewText replaces OldText. Empty deletes the matched text.
NewText string
ReplaceAll bool
}
Edit is one targeted replacement against the rendered markdown — the caller quotes text, never lines or block ids.
type NewOp ¶
NewOp is one entry in DiffResult.NewSeq, walked in new-document order. OldIdx is the index into the old block slice for Keep and Update, and -1 for Insert.
type NoMatchError ¶
type NoMatchError struct {
Index int
}
NoMatchError: an edit's OldText was not found, even fuzzily.
func (NoMatchError) Error ¶
func (e NoMatchError) Error() string
type OpKind ¶
type OpKind uint8
OpKind classifies the per-position outcome of a diff. The new document is described by a NewSeq of length len(newBlocks); each entry says how that position's text is sourced.
type OverlapError ¶
OverlapError: two edits matched intersecting regions.
func (OverlapError) Error ¶
func (e OverlapError) Error() string
type ParsedBlock ¶
type ParsedBlock struct {
Type string
Style map[string]any // nil when no style metadata applies
Text string // inline markdown only (no block-level syntax)
}
ParsedBlock is the typed view of one raw markdown block produced by Split. It mirrors the on-the-wire shape of a editor_blocks record (minus the SDK-managed id and _ver) so the markdown.Set path can diff and emit ops directly.
func ParseBlock ¶
func ParseBlock(raw string) ParsedBlock
ParseBlock classifies a raw block from Split into a typed ParsedBlock. The classification mirrors the splitter's block recognisers — same checks, just lifted into structured form.
type SetResult ¶
type SetResult struct {
// Inserted is the id of every newly created block, in document
// order.
Inserted []string
// Updated is the id of every block whose fields were replaced.
Updated []string
// Deleted is the id of every block that was tombstoned.
Deleted []string
// Unchanged is the count of blocks that were kept verbatim — no
// DB write touched them.
Unchanged int
}
SetResult bundles the per-call summary returned by Set, so callers that want to log / surface what changed can inspect it without re-running the diff.
func Append ¶
func Append(ctx context.Context, sp space.Space, objectId, collection, content string) (SetResult, error)
Append parses content into blocks and appends them all after the object's current last top-level block, in a single ModifyBatch. It never reads the existing block bodies and never diffs — only the tail position is looked up (one indexed `-nav.pos` query via editor.MaxPos) — so the cost is O(appended content), independent of how large the document already is.
This is the append-only fast path for whole-document round-trips: callers like the agent debug-log collector grow a page turn-by-turn and would otherwise pay Set's O(document) GET+diff on every append, making a full run O(N²) in page size. Append makes each call O(chunk) and the run O(N).
Semantics differ from Set in two ways the caller must accept:
- No diffing. Append is purely additive: every parsed block becomes a new record. It cannot update or delete existing blocks, and it will happily create a block identical to an existing one.
- No separator control. A fragment is positioned by the append itself: its blocks simply follow the current last one, and blank lines wrapping the fragment are framing, not content, so Split's leading/trailing empty paragraphs are dropped here. Empty paragraphs BETWEEN blocks of the fragment are kept, as they are for Set.
Empty (or blank-only) content is a no-op that returns a zero result.
func EditContent ¶
func EditContent(ctx context.Context, sp space.Space, objectId, collection string, edits []Edit) (SetResult, error)
EditContent applies targeted replacements to the rendered markdown of objectId — the surgical alternative to Set for callers that know the text they want changed but not the block ids. Edits resolve against the current rendering and the result goes through Set's diff pipeline, so a one-block change lands as one record op and a stale quote fails (NoMatchError) instead of clobbering concurrent edits. All-or-nothing: any unresolvable edit aborts before any write; byte-identical output is a no-op.
func Set ¶
func Set(ctx context.Context, sp space.Space, objectId, collection, content string) (SetResult, error)
Set replaces the markdown content of objectId with content, operating over the editor_blocks dataset.
Pipeline:
- Split content into raw markdown blocks, then ParseBlock each into a typed ParsedBlock {type, style, text}.
- List existing top-level editor_blocks records in pos order.
- Diff old-vs-new by their canonical-rendered string. Matched pairs become Update (only fields that changed land as $set); leftover new entries become Insert; leftover old ones become Delete.
- Allocate nav.pos lexids for new inserts so they slot between their kept neighbours in document order.
- Submit one ModifyBatch (creates + updates) followed by one DeleteBatch (tombstones). Same two-write split the old md_blocks path used.
The two writes are NOT a single atomic any-sync change. Because the CRDT resolves concurrent update-vs-delete as delete-wins, the observable end state is correct regardless of partial-failure retry, but a reader that catches the gap between (1) and (2) sees a transient state with both old-deleted and new-created blocks.
content is stored as INLINE markdown inside each block's `text` field — no full-block markdown bytes survive on disk. Empty content tombstones every existing top-level block. Blank lines beyond the one separating two blocks become empty paragraph records, so a document's vertical spacing survives the round trip (see Split).
Source Files