Documentation
¶
Overview ¶
Package marc8 transcodes between MARC-8 byte sequences and UTF-8, supporting every MARC-8 graphic character set:
- Basic Latin (ASCII) and Extended Latin (ANSEL), with combining diacritics
- Basic and Extended Cyrillic, Basic and Extended Arabic, Basic Hebrew, Basic Greek, Greek Symbols, Subscripts and Superscripts
- the multibyte East Asian (CJK) set, EACC
It is shared by the MARC serialization codecs that may carry MARC-8 data (e.g. iso2709, mrk).
MARC-8 follows ISO 2022: a primary set occupies G0 (invoked by bytes 0x21-0x7E) and an extension set occupies G1 (invoked by bytes 0xA1-0xFE); escape sequences re-designate either working set. The defaults are Basic Latin in G0 and Extended Latin (ANSEL) in G1. In the ANSEL set a combining diacritic is stored BEFORE its base character, the reverse of Unicode; the decoder reorders such marks after the base and composes the pair to a precomposed (NFC) code point when one exists.
Bytes under an unrecognized designation are passed through best-effort (as Latin-1) without crashing, and mark the decode lossy.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Decode ¶
Decode decodes a MARC-8 byte sequence to UTF-8 using a fresh Decoder. It is a convenience for one-shot decoding where designation state need not persist.
func Encode ¶
Encode encodes a UTF-8 string to MARC-8 across all supported character sets, emitting ISO 2022 escape sequences to switch sets as needed and returning to the default sets at the end. It is the inverse of Decode: a precomposed Latin character is decomposed to its base and combining mark, and combining marks are emitted BEFORE their base character (as MARC-8 requires, the reverse of Unicode order). It returns an error on the first code point no MARC-8 set can represent, so callers learn the value is not representable rather than producing mojibake.
Types ¶
type Decoder ¶
type Decoder struct {
// contains filtered or unexported fields
}
Decoder decodes MARC-8 with persistent G0/G1 designation state. MARC-8 reinstates the default working sets at the start of each field, not each subfield, so a field is decoded with a single Decoder (create a new one per field) and its subfields share the designation state.
func NewDecoder ¶
func NewDecoder() *Decoder
NewDecoder returns a Decoder initialized to the MARC-8 default working sets (Basic Latin in G0, ANSEL Extended Latin in G1).
func (*Decoder) Decode ¶
Decode decodes a MARC-8 byte sequence to a UTF-8 string for the supported Western subset, passing unsupported sets through best-effort and carrying the G1 designation state forward for the next call.
func (*Decoder) Lossy ¶
Lossy reports whether any Decode call on this Decoder fell back to best-effort passthrough of an out-of-scope MARC-8 character set, meaning the decoded text may contain mojibake. Callers re-serializing MARC-8 as UTF-8 can use this to avoid silently emitting corrupted data labeled as clean Unicode.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Command gen produces tables_gen.go from the Library of Congress MARC-8 code tables (codetables.xml), the authoritative mapping between MARC-8 and Unicode.
|
Command gen produces tables_gen.go from the Library of Congress MARC-8 code tables (codetables.xml), the authoritative mapping between MARC-8 and Unicode. |