Documentation
¶
Overview ¶
Package utf8stream decodes UTF-8 chunks the way Node does: TextDecoder with { stream: true } and StringDecoder.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Decoder ¶
type Decoder struct {
StripBOM bool
// contains filtered or unexported fields
}
Decoder mirrors the WHATWG UTF-8 decoder behind TextDecoder with { stream: true }: an incomplete trailing sequence waits for the next chunk, and each maximal invalid subpart becomes one U+FFFD.
With StripBOM set it also mirrors TextDecoder's default ignoreBOM: false, dropping a U+FEFF that is the stream's first code point, even when its bytes arrive split across chunks. Buffer.toString (file reads) keeps the BOM, so those callers leave StripBOM unset.
func (*Decoder) AppendDecode ¶
AppendDecode appends the decoding of data to dst. Between characters, the well-formed prefix of a chunk that ends at a character boundary is appended as it is; the decoder walks only what is left (an incomplete last character, or a chunk with an invalid sequence).