Documentation
¶
Overview ¶
Command abtest picks a variant for each experiment, keeps the markup for the chosen one, removes the rest, and marks the document with what it chose.
<div data-experiment="hero" data-variant="a">…</div> <div data-experiment="hero" data-variant="b">…</div>
The bucket is decided before the document starts - it is a function of a key from the request and the experiment's name - which is what makes this a single-pass rewrite at all. Nothing about the choice depends on the page, so every decision can be made at a start tag.
Three things in it are worth more than the substitution.
The bucketing has to be stable, and stable across processes: a visitor who gets variant b on one request and a on the next has been shown an experiment rather than been in one. So it is a hash of the key and the experiment name rather than anything stateful - FNV-1a, which is in the standard library and is not a security decision - taken modulo 10000 and compared against the cumulative weights. Including the experiment name in the hash is what stops every experiment bucketing the same visitor the same way, which would make two 50/50 experiments into one.
Marking the document is where the ordering constraint bites. The mark belongs on <html>, which is the first element, so there is exactly one chance to write it and no way back if the document does not have one. This program has five answers, in order: the <html> element, a <meta> prepended to <head>, the <body> element, a <meta> before the first element of any kind, and - for a document with no elements at all - lolhtml.OnDocumentEnd. Each is measured, because "every document has an <html> tag" is true of documents from a browser and not of documents from a template.
A mark this program wrote on an earlier pass is updated rather than added to, which is what makes running twice a no-op: a rewrite in front of a cache may well see its own output.
Removing the losing variant is the part that can go wrong quietly, and it is a consequence of B122: removing an element removes everything up to the token that closed it, and where the document left that element's end tag out, the token belongs to something else.
<ul><li data-experiment=x data-variant=b>lose<li data-variant=a>keep</ul>
Removing the first item removes the second as well - both variants - and nothing reports it. The removal is decided at the start tag, before the end tag is known, so this cannot be avoided; what it can be is noticed. Every removal registers an end-tag handler first, and a callback whose name is not the element's own means the removal reached further than the element. That is counted as Result.Overreach, and -strict turns it into an error, because a page that has silently lost half its content is worse than a failed request.
Turning it into an error is not enough on its own. The rewrite is streaming, so by the time the end tag reveals the overreach the destination already holds everything up to it, and returning an error there sends a truncated page with a failure reported after it - which is the outcome -strict exists to avoid. So -strict buffers the document and releases it only when the rewrite finishes cleanly. Refusing a document means holding it; a rewrite that cannot hold it cannot refuse it either.
Source Files