Help us shape the future of Go! Take the 2026 Go Developer Survey

markdown

package module
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2026 License: BSD-3-Clause Imports: 11 Imported by: 0

README

go-richdoc/markdown

Convert between Markdown and the format-agnostic richdoc document model.

  • Parse turns CommonMark (with the GFM table and strikethrough extensions, PHP-Markdown-Extra footnotes and explicit heading anchors) into a *richdoc.Document. Parsing delegates to the reference Go Markdown library, goldmark; this package never hand-rolls a CommonMark parser. It walks goldmark's AST and builds the richdoc tree.
  • Write renders a *richdoc.Document back to clean CommonMark.
Enabled goldmark extensions and options

Parse configures goldmark with:

  • extension.Table and extension.Strikethrough — GFM tables and ~~strike~~.
  • extension.Footnote — PHP Markdown Extra footnotes (text[^id] references with [^id]: … definitions). This is goldmark's own footnote extension; the package does not hand-roll footnote parsing.
  • parser.WithHeadingAttribute() — explicit heading anchors written as ## Title {#id}. WithAutoHeadingID() is deliberately not enabled: auto ids would fabricate an anchor on every heading, and Write would then stamp a {#id} the source never had. Only ids an author wrote are captured, which keeps the round-trip faithful.

The package is pure Go and builds with CGO_ENABLED=0.

import "github.com/go-richdoc/markdown"

doc, err := markdown.Parse([]byte("# Title\n\nHello **world**.\n"))
// ... inspect or edit doc ...
out, err := markdown.Write(doc)

⛔ The nesting bound, and why the cost of nesting is not the cost of bytes

Parse takes the default Limits; ParseWithLimits states them (v0.11.0+). One field, MaxBlockNesting, default 256, zero meaning the default and a negative value meaning no limit — the convention go-richdoc/rst's LineLengthLimit and go-richdoc/latex's MaxDepth already use.

It exists because block nesting is quadratic, and cheap to write. BenchmarkQuoteNestingDeep (2026-10-31, Apple M4 Max, Go 1.27.2):

block quotes source parse
10 000 deep 20 KB 54.7 ms
50 000 deep 100 KB 1.35 s
100 000 deep 200 KB 5.97 s
200 000 deep 400 KB 21.9 s

Doubling the depth multiplies the time by about 3.7. A block quote costs the author two bytes a level, so 400 KB of source occupies a core for 22 seconds and a megabyte for minutes.

Every benchmark here carries an equal-size flat control, because without one the numbers only say "a big document is slow", which is not a finding:

nested the same bytes, flat
list, 100 levels, 10.3 KB 1.40 ms 288 µs → 4.9×
list, 300 levels, 90.9 KB 30.9 ms 1.05 ms → 29.5×

The ratio itself grows with depth.

The guard runs before goldmark, because the cost is goldmark's. BenchmarkGoldmarkOnly times the engine alone on the same documents: 22% of the total at depth 100, 75% at 1000, 92% at 5000. A counter in this package's own recursion would never reach it, so the check is a line scan of its own — the placement go-richdoc/fb2 uses for the same reason.

goldmark has no nesting bound of its own (a search of its whole source for one finds nothing), and CommonMark is content for an implementation to have one: "Implementations may impose limits on parentheses nesting to avoid performance issues, but at least three levels of nesting should be supported."

The default is measured, not chosen. mddepthprobe, beside the Markdown corpus, read the 14 651 real .md files on the machine this was written on: the deepest reaches five leading > markers, and the widest indentation is 73 columns — about 36 levels. 256 clears the widest real document seven times over, and the controls in limits_test.go are those real shapes, which must still parse.

⛔ Fifty-five bytes were a denial of service

a footnote[^1]

[^1]: the note[^1]

A footnote's body is inlined at its reference, so resolving one is recursive. A note whose body cites itself recursed forever and Go answered fatal error: stack overflow — which is not a panic. No recover() catches it, so a service converting a document it did not write could be killed by fifty-five bytes and could not defend itself. A mutual pair and a ring of any length do it too.

The guard is the set of notes being inlined on the current path, so the same note referenced twice still carries its body twice — that is the control, and a guard that remembered instead of tracking would quietly empty the second reference.

It is the same shape go-richdoc/rst was fixed for two rounds earlier, and the audit that followed missed this one. That audit searched the family for a map[string] of definitions; this map is keyed by goldmark's int index. Searching for the shape of the code already seen, instead of for the behaviour — a definition inlined by reference — is how the second instance survived the sweep that was looking for it. FuzzParse found it in two minutes, and the input is committed under testdata/fuzz/FuzzParse.

⛔ What the fuzzer found that no unit test would have

The first version of the scan read a line's prefix for indentation and > markers and stopped at the first other character. FuzzParse produced this in six seconds:

*   *   *   *   *   *   ... x

In CommonMark a * and three spaces open an item whose content begins four columns in, so that single line is nested lists at four bytes a level — and the scan read it as depth zero. Measured (BenchmarkOneLineListNesting), 20 KB of it costs 74.6 ms, quadratically: worse per byte than the block quotes the guard was written for.

It was caught by the one property that does not trust the guard — the depth of the tree that came back — and the failing input is committed under testdata/fuzz/FuzzParse. The scan now walks the whole block prefix: quote markers, bullet and ordered list markers, and indentation, in any order.

Fenced code is tracked, and that is not a refinement: inside a fence a leading > is a transcript and a deep indent is a code sample, so a README quoting a shell session would otherwise be refused for its own text. The fence rules are the spec's, quoted in the tests — a closer "may be preceded by up to three spaces of indentation, and may be followed only by spaces or tabs", "must use the same character as the opening", and "must be at least as long".

Round-trip

The pair aims for a stable round-trip: Write(Parse(src)) is semantically equivalent to src, and Parse(Write(Parse(src))) reproduces the same richdoc tree, up to harmless normalisation:

  • emphasis markers normalised to * (_emph_ → *emph*);
  • list markers normalised to - / 1.;
  • setext headings rewritten as ATX (#);
  • CommonMark autolinks (<http://x>) rewritten as inline links;
  • character references and backslash escapes decoded into literal text on parse (as goldmark's own HTML renderer would resolve them, in the same single left-to-right scan — a backslash-escaped & never lets the entity reference that follows it resolve), and re-escaped on write, including a literal & immediately before what would otherwise look like a genuine entity or numeric reference;
  • a soft line break inside a block flattened to a single space on write — it survives Parse as a literal newline in Text.Value, but writing it through unchanged would corrupt the block it lands in: an ATX heading is a single physical line, and a paragraph line ending right before a ---/=== line re-parses as an accidental Setext heading. Flattening it matches the soft break's own HTML rendering (whitespace);
  • a link/image destination containing a space, a parenthesis, or a backslash anywhere in it written in the <...> form rather than the bare (url) form, which can't represent any of those safely (a bare destination's own scanner treats any \ + punctuation pair as an escape too, not just one right before the closing paren);
  • a code span whose content starts and ends with a space padded with one extra space on each side — the content already reflects CommonMark's own single-space padding strip, so writing it back with a bare fence would have it stripped a second time on re-parse;
  • nested single emphasis falls back to _ at whichever boundary sharing * with its own nested content would merge into ** (strong) — but not when the adjacent run is a different length already forming its own valid construct (* next to a leading ** from a nested Strong makes ***, CommonMark's own strong-wrapping-emphasis idiom, and is left alone);
  • two adjacent List blocks of the same kind (both ordered or both unordered) alternate their marker glyph — unordered -/+, ordered delimiter ./) — instead of both using the writer's default. List carries no record of which literal marker the source used, only Ordered/Start/Tight; two such lists only sit side by side in the model because Parse's source already split them (a bullet-character or delimiter change, per CommonMark's own list grammar — a single loose list never appears as two adjacent List nodes). Writing both with the same glyph would have goldmark re-merge them into one list on the next Parse and skew tightness in the process; alternating keeps them visibly separate, mirroring how nested emphasis falls back to _ above. The tracking is scoped to each block sequence (list items and block quotes recurse with their own), so nested adjacent sub-lists get the same treatment;
  • a paragraph whose rendered text would otherwise begin with block-level syntax has its first character neutralised: a leading # (heading), > (blockquote), - / + (list marker, or a bare - in a run of 3+, a thematic break) followed by a space or end-of-line, or an ordered-list marker (1. / 1)) get the responsible character backslash-escaped (1\. not a list, the same idiom the CommonMark spec itself uses); 4 or more columns of leading space/tab (which would otherwise read back as an indented code block) has its first whitespace byte written as a numeric character reference (&#32; / &#9;) instead, since backslash can't escape whitespace. Block parsing runs on raw bytes at column 0 of a line, before any inline backslash escape is resolved, so this is checked independently of — and in addition to — escapeText's ordinary character-class escaping, which is position-blind;
  • a heading whose text ends in a run of # preceded by whitespace (or made up entirely of #) has that run's leading # backslash-escaped, or goldmark's ATX "closing sequence" rule strips it on re-parse — in the all-# case, silently emptying the heading's text entirely.

Node mapping

Parse (goldmark AST → richdoc)
goldmark node richdoc node
Heading (ATX and setext) Heading{Level, ID, Inlines} (ID from a {#id} attribute)
Paragraph, TextBlock Paragraph{Inlines}
FencedCodeBlock CodeBlock{Language, Text}
CodeBlock (indented) CodeBlock{Text} (no language)
Blockquote BlockQuote{Blocks}
List List{Ordered, Start, Tight, Items}
ListItem ListItem{Blocks}
ThematicBreak ThematicBreak{}
extension/ast.Table Table{Align, Header, Rows}
extension/ast.TableCell Cell{Inlines}
HTMLBlock RawBlock{Format:"html", Text}
LinkReferenceDefinition dropped (resolved into the referring link)
Text Text{Value} (+ LineBreak on hard break)
String Text{Value}
CodeSpan Code{Value}
Emphasis level 1 Emph{Inlines}
Emphasis level 2 Strong{Inlines}
extension/ast.Strikethrough Strikethrough{Inlines}
Link Link{URL, Title, Inlines}
Image Image{URL, Alt, Title}
AutoLink Link{URL, Inlines:[Text(label)]}
RawHTML (inline) RawInline{Format:"html", Text}
Link with a clean #fragment dest CrossRef{Target, Kind:RefLabel, Inlines}
extension/ast.FootnoteLink Footnote{Blocks} (body from its definition)
extension/ast.FootnoteList dropped (bodies inlined at each reference)
extension/ast.FootnoteBacklink dropped (goldmark render artifact)

Adjacent text runs are coalesced so goldmark's tokenisation does not leak into the model.

Write (richdoc → CommonMark)
richdoc node Markdown output
Heading #…###### ATX heading (level clamped to 1–6), {#id} suffix when ID set
Paragraph inline text
CodeBlock fenced ``` block with language (fence widened past inner backticks)
BlockQuote >-prefixed lines
List - / N. items, tight or loose
Table GFM pipe table with an alignment delimiter row
Table.Caption an emphasised paragraph AFTER the table (see below)
ThematicBreak ---
MathBlock $$ … $$
RawBlock verbatim text (html and any other format)
Text escaped literal text
Emph *emph*
Strong **strong**
Strikethrough ~~strike~~
Code `code` (backticks widened / padded as needed)
Link [text](url "title")
Image ![alt](url "title")
Math $…$
LineBreak backslash + newline hard break
RawInline verbatim text
Footnote [^n] reference, body emitted as a [^n]: … definition (numbered in reference order, continuation lines indented 4 spaces)
CrossRef (RefLabel) [text](#target) internal link (target stands in when the reference has no text)
CrossRef (RefCite) pandoc [@key] citation (CommonMark has no native citation)
Anchor its inline content only — CommonMark has no anchor syntax, so the ID is dropped (write-only; Parse never produces an Anchor)
Constructs routed through raw passthrough

goldmark core has no math extension, so $…$ / $$…$$ are not recognised on parse: they remain literal Text. Writing a MathBlock / Math node (produced by another source) emits $$…$$ / $…$.

Any HTML goldmark exposes — block or inline — is preserved as RawBlock/RawInline with Format:"html". No other CommonMark or GFM construct in scope currently needs a raw fallback: every one maps onto a first-class richdoc node.

A Markdown link whose destination is a clean #fragment — no title, a non-empty fragment with no whitespace and none of ()<>#\" — maps to a CrossRef{Kind:RefLabel}. Anything else stays a plain Link: an external URL, a bare #, a titled link, or a fragment that could not be written back and re-parsed identically (for example [x](<#a b>), whose fragment contains a space). The mapping is intentionally conservative so it never turns an ordinary link into a cross-reference by accident, and it round-trips: a mapped CrossRef writes back as [text](#target) and re-parses to the same node.

What a pipe table cannot hold (richdoc v0.5.0)

Two fields arrived in the model that GFM has no syntax for, and this converter's answer to each is written down rather than left to be discovered.

Table.Caption is written as an emphasised paragraph after the table. GFM has no caption, so the choice was between dropping the author's words and putting them where a reader sees them. It does not come back as a caption on the next Parse — there is nothing in GFM to mark one with — so it is prose from then on. No table this package parses ever has a caption, so the round trip is unaffected: a caption can only arrive from another converter, where emitting it is a gain over losing it.

Cell.Blocks has nowhere to go: the GFM table extension parses a cell's content as inlines. richdoc's contract is that a producer which fills Blocks also fills Inlines with the flattened view, and that is what this writer reads, so a cell holding a list in reST or LaTeX keeps its words here and loses its structure. That is the format's limit, not a loss this package introduces.

Toolchain

Requires Go 1.27.1, which is the version the CI workflow pins. The two were out of step — the module asked for 1.26.4 while CI already ran 1.27.1 — and that difference decides two things a reader of this repository should not have to guess: 1.27 counts statements more finely, so a coverage figure from an older toolchain is an upper bound rather than a measurement, and its gofmt reindents a composite literal inside a multi-value return, which an older local gofmt reports as clean. Both were checked with 1.27.1 here: coverage is unchanged at 100%, and the tree needs no reformatting.

License

BSD-3-Clause. See LICENSE.

Documentation

Overview

Package markdown converts between Markdown and the format-agnostic github.com/go-richdoc/richdoc document model.

Parse turns CommonMark source (with the GFM table and strikethrough extensions, PHP-Markdown-Extra footnotes and explicit heading anchors enabled) into a richdoc.Document. Parsing delegates to the reference Go Markdown library, goldmark, and walks its AST to build the richdoc tree, so the package never hand-rolls a CommonMark parser. Footnotes come from goldmark's own extension.Footnote; heading anchors from parser.WithHeadingAttribute (explicit "{#id}"), not auto-generated ids.

Write renders a richdoc.Document back to clean CommonMark. The pair aims for a stable round-trip: Write(Parse(src)) is semantically equivalent to src, and Parse(Write(Parse(src))) reproduces the same richdoc tree, up to harmless normalisation (emphasis markers normalised to '*', list markers to '-'/'1.', and so on).

The package is pure Go and builds with CGO disabled.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Parse

func Parse(src []byte) (*richdoc.Document, error)

Parse converts CommonMark source (with GFM tables and strikethrough, PHP Markdown Extra footnotes and explicit heading anchors) into a richdoc.Document, with the default Limits.

goldmark's parser never reports an error of its own. The error this returns is therefore the limits': since v0.11.0 a document whose blocks nest past Limits.MaxBlockNesting is refused BEFORE goldmark is handed it, because the cost of that nesting is quadratic and 92% of it is the engine's at the depth where it starts to matter. See Limits.

func ParseWithLimits added in v0.11.0

func ParseWithLimits(src []byte, lim Limits) (*richdoc.Document, error)

ParseWithLimits is Parse with the bounds stated. See Limits.

func Write

func Write(d *richdoc.Document) ([]byte, error)

Write renders a richdoc.Document to clean CommonMark (with GFM tables and strikethrough, PHP Markdown Extra footnotes and explicit heading anchors). The error return is always nil; it is kept for symmetry with Parse. A nil document renders to empty output.

Footnotes are collected while the body renders and their definitions are emitted, numbered in reference order, after every block.

Types

type Limits added in v0.11.0

type Limits struct {
	MaxBlockNesting int
}

Limits bounds what this reader will do with a document nobody wrote.

⛔ MaxBlockNesting is how deeply block constructs may nest, and it exists because the cost of that nesting is not the cost of bytes. Measured by BenchmarkListNesting, which carries its own equal-size FLAT control:

nested list, 100 levels, 10.3 KB    1.40 ms   |  the same 10.3 KB flat   288 µs
nested list, 300 levels, 90.9 KB   30.86 ms   |  the same 90.9 KB flat  1.05 ms

Same bytes, same construct count: 4.9x at 100 levels and 29.5x at 300, so the ratio itself grows with depth.

Block quotes are the shape that makes it a defect rather than a slow case, because they cost the author TWO BYTES a level. Measured by BenchmarkQuoteNestingDeep (2026-10-31, Apple M4 Max, Go 1.27.2):

 10 000 deep    20 KB    54.7 ms
 50 000 deep   100 KB     1.35 s
100 000 deep   200 KB     5.97 s
200 000 deep   400 KB    21.87 s

Doubling the depth multiplies the time by about 3.7, so 400 KB of source occupies a core for 22 seconds and a megabyte for minutes.

⛔ The cost belongs to goldmark rather than to this package's walk, which is what decides where the guard goes. BenchmarkGoldmarkOnly times the engine alone on the same documents: at depth 100 it is 22% of the whole conversion, at 1000 it is 75%, and at 5000 it is 92% -- the share grows with depth, so at the depth that matters almost all of the time is spent before this package's code runs at all. A counter in our own recursion would never reach it.

So this is checked in a PASS OF ITS OWN before goldmark is handed the document -- the same placement go-richdoc/fb2 uses for the same reason, and the lesson go-odf/odf learned from a 115 MiB allocation inside a decoder while its own counter stood at two.

goldmark has no nesting bound of its own: a search of its whole source for a maximum-depth or nesting limit finds nothing. The CommonMark spec is content for one to exist -- "Implementations may impose limits on parentheses nesting to avoid performance issues, but at least three levels of nesting should be supported" (spec §6.3, about link destinations) -- so a bound is the implementation's to choose, and this one is chosen from a measurement.

THE DEFAULT IS 256, AND IT IS MEASURED. mddepthprobe (beside the Markdown corpus) read the 14 651 real `.md` files on the machine this was written on and counted both halves of the metric below: the deepest document reaches FIVE leading ">" markers, and the widest indentation is 73 columns -- about 36 levels. 256 clears the widest real document seven times over.

Zero means the default. A NEGATIVE value means no limit, for a caller who would rather spend the time than lose the document, which is the convention go-richdoc/rst's LineLengthLimit and go-richdoc/latex's MaxDepth already use.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL