include

Overview ¶

Command include expands Edge Side Include markers into fetched content, streaming, without buffering either the page or what it pulls in.

<esi:include src="/header"/>

Three things make this more than a substitution.

The first is that an esi: element is not a void element to an HTML parser. It has a colon in its name and it is conventionally written unclosed, so its content runs to the next matching end tag - which there never is - and replacing it takes the rest of the enclosing element with it. lolhtml.WithESITags is what says otherwise, and it is not optional here: the package documentation has the malformed output it produces without it. The selector needs the colon escaped too, as esi\:include, or the error blames a pseudo-class.

The second is where the fetch goes. The obvious place is inside the lolhtml.StreamFunc - that is where the content is wanted - and it is the wrong place, because a sink write reaches the destination as it is made. A fetch that fails after the first write has already committed a page with half an include in it, and no error path leads anywhere useful: the response has left. So the fetch happens in the handler, where returning an error still costs nothing, and the sink is given only a body that is already open. That buys the ESI spec's own error handling: onerror="continue" drops a failed include, an alt="..." tries a second source, and a caller who asked for neither gets an error before anything was sent.

The commitment is only moved, not removed: a body that fails after its first byte still truncates the page, and this program reports that as Result.Truncated rather than pretending otherwise. A stream cannot promise atomicity. What it can do is not spend the promise before it has to.

The third is that includes nest. A fetched fragment can contain includes of its own, and expanding them by buffering the fragment would give back everything the streaming was for. Instead the fragment is run through its own rewriter whose destination is the sink:

fragment -> rewriter(depth+1) -> sink -> destination

so nothing is held anywhere. Each nested rewriter gets its own option set, built with its own depth and its own path, because the function inside an Option is shared with every Writer it is given to - see lolhtml.Option. The path is also the cycle check: a fragment that includes an ancestor of itself is refused rather than followed until the depth limit stops it.

Two other pieces of ESI, both small and both worth having for the shape of them:

<esi:remove> holds content for clients that do not process ESI, so a processor removes it. That is lolhtml.Element.Remove, and it is the one operation on an unclosed container that is not made worse by WithESITags being absent - though it is enabled anyway.

<!--esi ... --> is the reverse: content hidden from those same clients inside a comment, which a processor unwraps and processes. Unwrapping means writing the comment's text out as markup, so the check is whether the document really spelled it as a comment: the delimiter arithmetic on lolhtml.Comment.SourceLocation says whether this is <!--esi ...--> or a processing instruction that happens to start with the same three letters.