references

Overview ¶

Command references decodes the character references a document did not need.

<p>Caf&eacute; &amp; bar &#8212; open</p>
  ->  <p>Cafe&#769; &amp; bar - open</p>   (with the real characters, not these)

An escaped character says the same thing as the character, so a document full of &eacute; and &#8212; is bigger and harder to read for no benefit - and the ones that do carry meaning are few: & and < in text, & and the quote in an attribute value. This program decodes the rest and leaves those alone.

Which references a browser decodes is not one rule but two ¶

In text, a named reference is decoded whether or not it ends in a semicolon, and the match is the longest one - so "&notit;" is the not sign followed by "it;". In an attribute value the same reference without its semicolon is not a reference at all when the character after it is "=" or ASCII alphanumeric, which is the rule that keeps "?a=1&copy=2" a URL with a parameter called copy rather than one with a copyright sign in it.

So the standard library's html.UnescapeString is the parser's decoder for text and not for attribute values: measured, it turns "?a=1&notit=2" into something a browser never has. This program implements the attribute rule itself, which is the difference between normalising a URL and changing it. See differential/attrrefs_test.go.

Writing the result back ¶

The decoded text goes back with lolhtml.HTML rather than lolhtml.Text, because Text escapes the three markup characters and would put back the references just removed - and it escapes ">", which needs no escaping in text at all. So this program decides for itself what stays encoded: "&" and "<" in text, "&" and the double quote in an attribute, and nothing else. That is the rule for moving a value into a context, applied to a value that is staying where it is.