Documentation
¶
Overview ¶
Package xmlx bounds the work an untrusted XML document can cost before and during an encoding/xml decode.
A byte cap on the response body is not a bound on decoding. One token can be as large as the body, millions of tiny elements expand into a much larger object graph, and every open element is a live entry on the decoder's stack, with Decoder.Skip walking unmodeled children at any depth.
The package has two halves, matching where the cost is paid:
- Preflight is a lexical gate over the raw bytes, run before the decoder sees them. One allocation-free scan rejects a body whose tokens, element count or nesting are already outside the caller's Limits.
- Budget is the decode-time text accounting a schema decoder charges each retained value against, with a per-value cap and a document-wide cap.
Preflight bounds what the decoder must materialize, and Budget bounds what the program keeps. Schema decoding stays with the caller.
if err := xmlx.Preflight(body, xmlx.DefaultLimits()); err != nil {
return err
}
var doc feed
if err := xml.Unmarshal(body, &doc); err != nil { // UnmarshalXML charges a Budget
return err
}
A non-positive limit is reported as ErrInvalidLimits, never read as unbounded. Start from DefaultLimits and DefaultBudget and size them to the document contract.
Preflight refuses every XML directive other than a comment or CDATA section, such as <!DOCTYPE. The bounds model the default decoder settings. docs/design.md explains the cost model, and docs/non-goals.md covers directives, XXE, round-trip stability and decoder settings.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrLimit reports that a document exceeded one of the caller's bounds. // Every rejection from Preflight and Budget wraps it, so a consumer can // classify "the document was too big to be worth decoding" with one // errors.Is and read *LimitError only when it wants the specific bound. ErrLimit = errors.New("xmlx: limit exceeded") // ErrInvalidLimits reports a non-positive bound: a configuration mistake in // the CALLER, not a property of the document. It is deliberately not // treated as an unbounded setting, see the package doc. ErrInvalidLimits = errors.New("xmlx: invalid limits") )
Sentinel errors, matched with errors.Is through the wrapped errors this package returns.
Functions ¶
func Preflight ¶
Preflight walks the raw bytes of an XML document and rejects one that is already outside lim, BEFORE any decoder allocates a token from it. It is a single allocation-free pass: one sequential scan, no buffering, no copies.
It rejects an overlong raw text or CDATA run, an overlong markup token, a start tag carrying more than lim.MaxTagAttrs XML attributes, more than lim.MaxElements elements, element nesting whose PEAK exceeds lim.MaxDepth (a self-closing element counts as one level), and any XML directive (see the package doc). Each rejection is a *LimitError naming the bound and the byte offset where it was detected, and every one wraps ErrLimit.
Preflight is a bounds check, not a validator. It reads the document's surface structure only, enough to know where one token ends and the next begins, so a malformed body that stays within the bounds passes through for the decoder to reject with its own parse error. The converse does not hold: malformed input can also trip a bound (a run of `=` bytes reads as attributes; a truncated `<!` opener is refused with the directive class), so a rejection does not prove the document was oversized, only that it was outside the contract.
Decoder configuration ¶
The bounds model the DEFAULT decoder: Strict enabled, AutoClose empty, no caller-supplied Entity map, no CharsetReader. Each of those knobs changes what the bytes mean. With Strict disabled a bare attribute name becomes an attribute this scan does not count; an Entity map can expand a short reference into an arbitrarily long value after the raw bound has passed; a CharsetReader transforms the very bytes that were scanned. A caller that sets any of them keeps the token and depth bounds but should treat the attribute and text bounds as advisory.
body is not modified or retained.
Example ¶
The gate runs over the raw bytes before the decoder sees them, so a document outside the caller's contract costs one scan instead of a decode.
package main
import (
"fmt"
"github.com/cplieger/xmlx"
)
func main() {
body := []byte(`<rss><channel><title>Example</title></channel></rss>`)
if err := xmlx.Preflight(body, xmlx.DefaultLimits()); err != nil {
fmt.Println("rejected:", err)
return
}
fmt.Println("accepted")
}
Output: accepted
Example (Directive) ¶
An XML directive is refused as a class: bounding one truthfully means reproducing encoding/xml's own directive tokenizer, so the package declines instead of reporting a bound it cannot honor.
package main
import (
"fmt"
"github.com/cplieger/xmlx"
)
func main() {
err := xmlx.Preflight([]byte(`<!DOCTYPE rss><rss/>`), xmlx.DefaultLimits())
fmt.Println(err)
}
Output: xmlx: XML directives are not allowed
Example (LimitError) ¶
A rejection names which bound fired, so a consumer can log the specific contract the upstream broke instead of "too big".
package main
import (
"errors"
"fmt"
"strings"
"github.com/cplieger/xmlx"
)
func main() {
// A body nested far past a feed's real shape: each 3 bytes of wire would
// cost the decoder one live open-element stack entry.
body := []byte(strings.Repeat("<a>", 5000))
err := xmlx.Preflight(body, xmlx.DefaultLimits())
if le, ok := errors.AsType[*xmlx.LimitError](err); ok {
fmt.Println(le.Kind, "limit:", le.Limit)
}
fmt.Println("is a limit error:", errors.Is(err, xmlx.ErrLimit))
}
Output: element nesting depth limit: 64 is a limit error: true
Types ¶
type Budget ¶
type Budget struct {
// contains filtered or unexported fields
}
Budget is the decode-time text accounting for ONE document: a per-field cap on any single decoded value, and a cumulative cap on everything the document retains. It is the half of this package that Preflight cannot cover, because raw bytes and decoded text are different quantities: entity references, CDATA seams, and repeated elements all break the correspondence, so a document can pass every lexical bound and still hand a schema decoder more text than the caller means to hold.
What it bounds is what the caller BUILDS and KEEPS. The per-token allocation happens inside encoding/xml before any check here can run, which is precisely why Preflight exists: only a gate over the raw bytes bounds what the decoder must materialize. The two are complements, not alternatives.
Use one Budget per document, threaded through the decoders that charge it. Create it with NewBudget or DefaultBudget: the caps are immutable afterwards, so a document cannot be part-way through its allowance when the allowance changes. Copying a Budget would fork its running total, leaving two copies each believing they hold the whole allowance, so a copy is a vet error. It is not safe for concurrent use; a document decodes on one goroutine.
A Budget does NOT bound element count or nesting. A caller using Budget alone still lets encoding/xml skip unknown children through Decoder.Skip, which is iterative and unbounded, so the decoder's element stack still grows to the document's true depth. Only Preflight bounds that.
Example ¶
A schema decoder charges each retained value against one Budget, so the document is bounded by what it makes the program HOLD, not only by its wire size.
const doc = `<item><title>Show & Tale</title><guid>x1</guid></item>`
budget := mustBudget(64, 4096)
var title, guid string
d := xml.NewDecoder(strings.NewReader(doc))
for {
tok, err := d.Token()
if err != nil {
break
}
start, ok := tok.(xml.StartElement)
if !ok {
continue
}
switch start.Name.Local {
case "title":
title, err = budget.DecodeText(d)
case "guid":
guid, err = budget.DecodeText(d)
}
if err != nil {
fmt.Println("rejected:", err)
return
}
}
fmt.Printf("%s / %s / charged %d bytes\n", title, guid, budget.Total())
Output: Show & Tale / x1 / charged 13 bytes
func DefaultBudget ¶
func DefaultBudget() *Budget
DefaultBudget returns the decode-time counterpart of DefaultLimits: 4 KiB per decoded value, 4 MiB of decoded text per document. The gap between this and DefaultLimits.MaxTextRunBytes is entity-expansion headroom (see that field).
func NewBudget ¶
NewBudget returns a Budget bounding one document to maxFieldBytes per decoded value and maxTotalBytes of decoded text overall. Both must be positive: a non-positive cap is a configuration mistake (ErrInvalidLimits), never read as unbounded.
Size maxTotalBytes from what the caller is willing to HOLD, which is usually far below the transport cap. A consumer parsing a catalogue-scale document sets it at that document's real ceiling; the DefaultBudget values are for a small structured document and will reject a multi-megabyte payload.
func (*Budget) Charge ¶
Charge accounts one already-decoded value against both caps and reports whether it may be retained. It is what a decoder calls for a value encoding/xml has handed it whole, such as an attribute value off a StartElement.Attr, where the allocation has happened but retention has not.
Call it BEFORE storing the value: a decoder that stores first and charges after has already kept the document it was about to reject.
Every occurrence is charged, including repeats of the same element name. That is deliberate: a document that sends <title> ten thousand times overwrites one destination field but costs ten thousand decodes, so charging only the value that survives would leave that repetition unaccounted. Note the bound is on BYTES, so repetition of EMPTY values costs nothing here; bounding the number of elements is Limits.MaxElements's job, at the lexical gate.
Charge each value exactly once. A value returned by DecodeText has already been charged.
func (*Budget) DecodeText ¶
DecodeText reads the text content of the element the decoder has just entered, bounded and charged, and returns it. It replaces d.DecodeElement(&s, &start) for a plain text field.
The difference that matters is WHEN the cap applies. DecodeElement concatenates every CharData token in the element and hands back the finished string, so a value split across CDATA seams, each chunk individually small enough to pass any lexical bound, is only measurable after it exists. DecodeText accumulates under the per-field cap AND under the document's remaining allowance, stopping at the token that would cross either, so the string it builds never exceeds what the caller would accept.
Nested markup is skipped whole; comments and processing instructions are ignored, as DecodeElement ignores them.
Where the two stop agreeing, measured ¶
The swap is byte-identical to DecodeElement for every value inside encoding/xml's own acceptance set, and the fuzz oracle pins that. It is NOT identical outside it, in ONE respect, and the difference is a real acceptance-set divergence rather than a wording caveat.
encoding/xml guards its unmarshal recursion by sampling the decoder's live open-element count on each entry into that recursion, against a fixed internal ceiling: 10000, and 5000 when GOARCH is wasm. DecodeElement enters that recursion, so it inherits the ceiling. Measured on go1.27.0 by entering an element at a known open depth: the two agree to open depth 9999, and from open depth 10000 DecodeElement refuses with an unexported, unwrapped errors.New("exceeded max depth") while DecodeText returns the value. So DecodeText is the LOOSER of the two above that depth.
That is deliberate rather than an omission. DecodeText is iterative (Token plus Skip), so it carries none of the recursion the ceiling exists to bound, and nesting is Preflight's bound in this package's split, not a Budget's: a Budget bounds bytes retained, never document shape. A caller that wants the decoder's ceiling enforced sets Limits.MaxDepth at or below it and runs Preflight, which rejects first and reports KindDepth wrapping ErrLimit -- a classifiable error, which the stdlib's is not.
On success the element's end tag is consumed, so the caller's token loop continues at the next sibling. On ANY error the document is over, and the caller must abandon it: a rejected value can leave the decoder part-way through the element, and a decoder error leaves it wherever encoding/xml stopped. There is no defined position to resume from.
Example (SplitValue) ¶
The per-field cap applies while the value accumulates, so a value split across CDATA seams, every chunk individually small, is refused at the token that would cross it rather than after the whole string exists.
doc := "<title>" + strings.Repeat("<![CDATA[xxxxxxxx]]>", 20) + "</title>"
budget := mustBudget(32, 4096)
d := xml.NewDecoder(strings.NewReader(doc))
if _, err := d.Token(); err != nil {
return
}
_, err := budget.DecodeText(d)
fmt.Println(err)
fmt.Println("charged:", budget.Total())
Output: xmlx: decoded field longer than 32 bytes charged: 0
func (*Budget) MaxFieldBytes ¶
MaxFieldBytes reports the per-value cap this Budget was built with.
func (*Budget) MaxTotalBytes ¶
MaxTotalBytes reports the document-wide cap this Budget was built with.
type ConfigError ¶
type ConfigError struct {
// Field is the struct field name of the offending bound.
Field string
// Value is what the caller set it to.
Value int
}
ConfigError reports a non-positive bound, naming the field that is wrong. It wraps ErrInvalidLimits.
type Kind ¶
type Kind uint8
Kind names which bound a *LimitError reports. It exists so a consumer can branch or log on the specific bound without parsing an error string: a depth rejection and an oversized-field rejection say different things about the upstream that sent the document.
const ( KindUnknown Kind = iota // KindTextRun: one contiguous run of raw character data exceeded // Limits.MaxTextRunBytes. KindTextRun // KindCDATA: one CDATA section's content exceeded Limits.MaxTextRunBytes. // It shares the text bound because a CDATA section IS character data, just // spelled so that markup bytes inside it are literal. KindCDATA // KindComment: one comment exceeded Limits.MaxTokenBytes. KindComment // KindProcInst: one processing instruction exceeded Limits.MaxTokenBytes. KindProcInst // KindToken: one tag exceeded Limits.MaxTokenBytes. KindToken // KindTagAttrs: one start tag carried more than Limits.MaxTagAttrs XML // attributes. KindTagAttrs // KindDepth: element nesting exceeded Limits.MaxDepth. KindDepth // KindElements: the document carried more elements than // Limits.MaxElements. KindElements // KindDirective: the document carried an XML directive (a DOCTYPE, ENTITY, // ATTLIST or NOTATION declaration), or a truncated `<!` opener that could // only become one. Preflight rejects the whole class; see the package doc // for why. Its LimitError carries no numeric bound. KindDirective // KindField: one decoded value exceeded Budget.MaxFieldBytes. KindField // KindTotalText: the cumulative decoded text charged to one Budget // exceeded Budget.MaxTotalBytes. KindTotalText )
The bounds this package enforces. KindUnknown is the zero value and names no bound; it never appears in a returned error.
type LimitError ¶
type LimitError struct {
// Kind names the bound that fired.
Kind Kind
// Limit is the configured bound, or 0 for a bound that is not numeric
// (KindDirective).
Limit int
// Offset is the byte offset in the document where the rejection was
// detected. Preflight sets it; Budget leaves it 0, because a decoded value
// has no single offset and the caller's decoder knows which field it was
// reading (wrap the error with that name).
Offset int
}
LimitError reports which bound a document exceeded. Match it with errors.AsType; match the class with errors.Is against ErrLimit.
It carries no excerpt of the document. That is deliberate: the offending bytes are untrusted input, and an error built from them would be an unbounded, unsanitized string on its way to a log line, which is the amplification this package exists to stop reintroduced through its own diagnostics. Offset is the safe half of that diagnostic need: one bounded integer, no attacker-chosen content, and enough to find the offending token in a saved payload.
func (*LimitError) Is ¶
func (e *LimitError) Is(target error) bool
Is reports ErrLimit for every LimitError, so a consumer that only cares about the class does not have to enumerate kinds.
type Limits ¶
type Limits struct {
// MaxTextRunBytes caps CHARACTER DATA: one contiguous run of raw text, or
// one CDATA section's content. Both are measured as content, which is what
// the decoder hands back as a CharData token.
//
// This bounds RAW bytes, while a decoded-value cap (Budget) bounds resolved
// text. The two are not interchangeable and the raw one must be the looser:
// a reference expands to fewer bytes than it occupies, and the widest
// predefined one is 6 to 1 (`"` and `'` are six raw bytes for one
// decoded), so a run that would decode within a 4 KiB value cap can
// legitimately arrive as tens of KiB of raw bytes. Setting this at or below
// the decoded cap would reject valid documents.
MaxTextRunBytes int
// MaxTokenBytes caps one MARKUP TOKEN whole, delimiters included: a tag
// (`<` through `>`), a comment (`<!--` through `-->`), or a processing
// instruction (`<?` through `?>`). A start tag carrying MaxTagAttrs
// attributes of the largest value the caller accepts should still fit with
// margin.
MaxTokenBytes int
// MaxTagAttrs caps the XML attributes on ONE start tag.
//
// This is a LEXICAL bound on attributes written inside a tag
// (`<enclosure url=".." length=".."/>` is two), which is a different
// question from how many child ELEMENTS the caller's schema permits. The
// latter is schema cardinality and belongs at the caller's decode site.
//
// Attributes are counted as `=` bytes outside a quoted value, which is
// exact for well-formed XML (the grammar is Name Eq AttValue) and for
// everything the DEFAULT decoder accepts. It is not exact for a decoder
// with Strict disabled, which accepts a bare name as an attribute whose
// value is its own name; such a tag carries attributes this bound does not
// see. See Preflight's decoder-configuration note.
MaxTagAttrs int
// MaxDepth caps element nesting.
//
// This is the bound with the widest gap between wire size and cost: the
// decoder pushes one heap-allocated entry per open element, and its
// Decoder.Skip path (which every child the caller's schema does not model
// goes through) has no depth bound at all, so a body of three-byte start
// tags converts each 3 bytes of wire into a live stack entry for the whole
// decode. Set it to the document's real shape (a syndication feed is about
// 4 deep), not to a round number.
//
// It bounds the PEAK number of simultaneously open ELEMENTS, which is what
// the decoder's stack holds. A self-closing element counts: encoding/xml
// pushes `<e/>` and pops it on the synthesized end tag, so `<a><b/></a>`
// reaches depth 2, not 1. The decoder additionally pushes one entry per
// namespace declaration it meets, so the stack LENGTH is bounded by
// MaxDepth * (1 + MaxTagAttrs) rather than by MaxDepth alone. Those
// namespace entries do NOT feed the decoder's own depth guard below, which
// counts start elements only.
//
// # The decoder's own ceiling, and where it overlaps this bound
//
// encoding/xml carries a fixed internal ceiling of its own on the same
// quantity: 10000 open elements, and 5000 when GOARCH is wasm. It is not an
// alternative to this bound, because the decoder SAMPLES it only on each
// entry into its unmarshal recursion rather than tracking the document's
// peak. Measured on go1.27.0: a document 349,525 elements deep under a
// schema that models only the root is accepted, and Decoder.Skip walks it
// unbounded, so a caller relying on the decoder's ceiling has no depth
// bound at all for the part its schema ignores. That is the case this
// bound exists for.
//
// Where the two do overlap is a schema whose own nesting follows the
// document's, which is what makes the region of MaxDepth above the
// decoder's ceiling unreachable for such a schema: measured at MaxDepth
// 10001, a 10001-deep document passes Preflight and is then refused by
// xml.Unmarshal with an unexported, unwrapped errors.New("exceeded max
// depth") that no errors.Is can classify. So keep MaxDepth at or below
// 10000 (5000 for a wasm build) whenever the schema is nested as deeply as
// the document, and this package reports the rejection instead, as
// KindDepth wrapping ErrLimit.
MaxDepth int
// MaxElements caps the total number of elements in the document.
//
// It is the bound for amplification by COUNT rather than by size: a body of
// millions of three-byte elements passes every per-token bound above (each
// token is tiny, the nesting is flat, there is no text at all) and still
// expands into a decoded object graph many times the wire size. Depth
// cannot catch it and neither can a text budget, because empty elements
// carry no text to charge.
//
// This is a lexical count of start tags, not schema cardinality: it needs
// no vocabulary from the caller, and unlike an "at most N <item>" rule it
// protects a plain xml.Unmarshal consumer that has no custom decode site to
// put such a rule in.
MaxElements int
}
Limits are Preflight's lexical bounds over the raw document bytes. Every field must be positive; a non-positive value is a configuration mistake (ErrInvalidLimits), never an unbounded setting.
Size these from the document CONTRACT, not from the transport cap. The question each bound answers is "what would a legitimate document from this endpoint never exceed?", and the useful answer is usually orders of magnitude below the byte cap: that gap is the amplification headroom the preflight removes.
func DefaultLimits ¶
func DefaultLimits() Limits
DefaultLimits returns bounds sized for a small structured document, such as a syndication feed or an XML API response, where fields are short, elements carry a handful of attributes, and nesting is shallow.
They are a starting point, not a recommendation. A caller that knows its document contract should tighten them, and one parsing genuinely large documents must raise MaxTextRunBytes, MaxDepth and MaxElements deliberately rather than discovering the rejection in production: a catalogue-scale dump of tens of thousands of records will exceed MaxElements, which is the point of the bound but not a surprise worth having at 3am.