Documentation
¶
Overview ¶
Package marc8 decodes MARC-8 byte sequences to UTF-8 for the common Western subset: Basic Latin (ASCII, the default G0 set) and ANSEL Extended Latin (the default G1 set), including combining diacritics. It is shared by the MARC serialization codecs that may carry MARC-8 data (e.g. iso2709, mrk).
In MARC-8 a combining diacritic is stored BEFORE its base character, the reverse of Unicode, where the combining mark follows the base. This decoder buffers pending marks and emits them after the base character, composing the base and the first mark to a precomposed (NFC) code point when one exists.
Out of scope: EACC/CJK, Cyrillic, Greek, Hebrew, Arabic, and the subscript/superscript/Greek-symbol sets. Their escape designations are recognized only well enough to skip the escape bytes and pass subsequent bytes through best-effort (as Latin-1) without crashing.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Decode ¶
Decode decodes a MARC-8 byte sequence to UTF-8 using a fresh Decoder. It is a convenience for one-shot decoding where designation state need not persist.
func Encode ¶
Encode encodes a UTF-8 string to MARC-8 for the supported Western subset: ASCII (G0) and ANSEL Extended Latin (G1) including combining diacritics. It is the inverse of Decode: a precomposed Latin character is decomposed to its base and combining mark, and combining marks are emitted BEFORE their base character (as MARC-8 requires, the reverse of Unicode order). It returns an error on the first code point outside the supported subset (e.g. Greek, Cyrillic, CJK), so callers learn the value is not representable rather than producing mojibake.
Types ¶
type Decoder ¶
type Decoder struct {
// contains filtered or unexported fields
}
Decoder decodes MARC-8 with persistent G1 designation state. MARC-8 reinstates the default working sets at the start of each field, not each subfield, so a field is decoded with a single Decoder (create a new one per field) and its subfields share the designation state.
func NewDecoder ¶
func NewDecoder() *Decoder
NewDecoder returns a Decoder initialized to the MARC-8 default working sets (Basic Latin in G0, ANSEL Extended Latin in G1).
func (*Decoder) Decode ¶
Decode decodes a MARC-8 byte sequence to a UTF-8 string for the supported Western subset, passing unsupported sets through best-effort and carrying the G1 designation state forward for the next call.
func (*Decoder) Lossy ¶
Lossy reports whether any Decode call on this Decoder fell back to best-effort passthrough of an out-of-scope MARC-8 character set, meaning the decoded text may contain mojibake. Callers re-serializing MARC-8 as UTF-8 can use this to avoid silently emitting corrupted data labeled as clean Unicode.
Source Files