transcode

package module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 10, 2026 License: MIT Imports: 25 Imported by: 0

README

transcode

Go library that converts data between character sets, including the single-byte EBCDIC code pages used on mainframes.

Status: version 0.3.0; see CHANGELOG.

go get code.waarp.fr/lib/transcode@v0.3.0

Built-in EBCDIC code pages

The single-byte EBCDIC code pages are generated from the tables that IBM publishes through the ICU project, at a fixed commit of its data repository:

go generate ./...

Each page is checked byte by byte against a witness produced by the iconv of the GNU C library (testdata/reference/witness.sh).

License

MIT, see LICENSE. The code page data come from the ICU project, under the terms given in NOTICE.

Documentation

Overview

Package transcode converts data between character sets, including the single-byte EBCDIC code pages used on mainframes.

Translation tables

A Table translates data byte by byte, without going through Unicode, so it reproduces exactly an existing single-byte translation. It is read from and written to its binary form (256 bytes) or its text form (512 hexadecimal digits), can be inverted when it is bijective, and composed with another table.

Code pages

A CodePage is a single-byte character set: each byte stands for at most one Unicode character. It implements golang.org/x/text/encoding.Encoding, so it converts to and from UTF-8, and through UTF-8 to any other encoding. A page is built from the character of each byte (NewCodePage), read from a mapping file in the format of the Unicode consortium (ParseCodePage), or taken from golang.org/x/text/encoding/charmap (FromCharmap).

TableBetween computes once the Table that converts data between two code pages. A byte without equivalent in the target page makes it fail, the error naming every such byte, unless a substitution byte is given.

Registry

A Registry holds the character sets and translation tables available by name. It comes with 121 built-in character sets: the Unicode encodings, the single-byte code pages and the multi-byte encodings of golang.org/x/text/encoding, and 60 single-byte EBCDIC code pages, 57 of which are generated from the tables that IBM publishes through the ICU project. Code pages and tables can be added at run time; the registry lists them all, for instance for an administration screen.

Records

Converting a whole file also converts its record separators: the line feed of ASCII becomes 0x25 in EBCDIC, so that a program which cuts the file into lines at 0x0A no longer finds them. A RecordTransformer applies a conversion to the content of each record instead, and lays the records out as asked. Each side has its RecordFormat:

  • Stream: the data is a single record;
  • Separated: records ended by a separator, such as LF, CRLF or the EBCDIC new line NL;
  • SameSeparators, on output: the separators read on input, unchanged;
  • Fixed: records of a fixed length, padded on output and optionally trimmed on input;
  • RDW: variable-length records preceded by a record descriptor word, as z/OS writes them;
  • Lengths, on input: records whose lengths are known apart from the data, such as the articles of a PeSIT transfer.

RecordFormat.Reporting gives the length of each record written, so that the lengths of the result can be kept as those of the source were.

Combining them adds, removes or changes separators: Separated input and Fixed output turn lines into fixed-length records, Fixed input and Separated output the reverse.

Conversions

Registry.NewConverter builds the conversion that a Conversion describes by names: from a source to a target character set, UTF-8 to EBCDIC for instance, or with a translation table, as a whole or record by record. Separators and padding are then given as characters, which the converter writes in each character set: LF becomes 0x25 in EBCDIC, NEL the EBCDIC new line 0x15, and padding defaults to the space of the target. Records too long are cut between two characters, and byte order marks are read and written once. Without records, a conversion behaves exactly as the decoder and encoder of golang.org/x/text.

With the same character set as source and target, a conversion only lays the records out again: their content is copied byte for byte, never decoded, so that a byte which stands for no character goes through unchanged. Separators and padding are still written in that character set, and a record too long is still cut between two characters.

Streams

Every conversion implements golang.org/x/text/transform.Transformer, so it applies to a stream of any size through golang.org/x/text/transform.NewReader or golang.org/x/text/transform.NewWriter.

Index

Examples

Constants

View Source
const (
	// LF is the line feed, which ends lines on Unix and Linux systems.
	LF = "\n"
	// CRLF is the carriage return and line feed, which end lines on Windows.
	CRLF = "\r\n"
	// NL is the EBCDIC new line, which ends lines in the z/OS UNIX file system.
	NL = "\x15"
	// EBCDICLF is the EBCDIC line feed, into which EBCDIC code pages turn LF.
	EBCDICLF = "\x25"
)

Record separators commonly found in data.

View Source
const NEL = "\u0085"

NEL is the Unicode next line character. EBCDIC code pages turn it into their new line, 0x15, and LF into their line feed, 0x25: in the records of a Conversion, NEL stands for the EBCDIC new line.

View Source
const NoChar rune = -1

NoChar marks, among the characters given to NewCodePage, a byte that stands for no character.

View Source
const TableSize = 256

TableSize is the number of entries of a Table, one per byte value.

Variables

View Source
var (
	// ErrInvalidCodePage is returned when a code page definition is invalid.
	ErrInvalidCodePage = errors.New("invalid code page")

	// ErrUndefinedByte reports a byte that stands for no character in its
	// code page.
	ErrUndefinedByte = errors.New("byte stands for no character")

	// ErrUnsupportedChar reports a character that a code page cannot
	// represent.
	ErrUnsupportedChar = errors.New("character not supported by the code page")
)
View Source
var (
	// ErrRecordFormat is returned for an invalid record format, or one that
	// does not apply to the direction it is used in.
	ErrRecordFormat = errors.New("invalid record format")

	// ErrInvalidRecord is returned for a record that cannot be read or
	// written as its format requires.
	ErrInvalidRecord = errors.New("invalid record")
)
View Source
var (
	// ErrUnknownName is returned for a name that the registry does not hold.
	ErrUnknownName = errors.New("unknown character set or table")

	// ErrWrongKind is returned when an entry is not of the kind asked for.
	ErrWrongKind = errors.New("entry of another kind")

	// ErrInvalidName is returned for an empty name, or a name that starts or
	// ends with white space.
	ErrInvalidName = errors.New("invalid name")

	// ErrNameTaken is returned when a name, or a name that the registry
	// takes for the same, is used by another entry.
	ErrNameTaken = errors.New("name already used by another entry")

	// ErrBuiltinEntry is returned on an attempt to change or remove a
	// built-in entry.
	ErrBuiltinEntry = errors.New("built-in entry cannot be changed")
)
View Source
var (
	// ErrInvalidTable is returned when data cannot be read as a [Table].
	ErrInvalidTable = errors.New("invalid translation table")

	// ErrNotInvertible is returned by [Table.Invert] when the table
	// translates two bytes into the same value.
	ErrNotInvertible = errors.New("translation table is not invertible")
)
View Source
var ErrConversion = errors.New("invalid conversion")

ErrConversion is returned for a conversion that cannot be made as described.

Functions

This section is empty.

Types

type CodePage

type CodePage struct {
	// contains filtered or unexported fields
}

CodePage is a single-byte character set: each byte stands for at most one Unicode character.

*CodePage implements encoding.Encoding. Its decoder, which converts the page to UTF-8, turns a byte that stands for no character into U+FFFD. Its encoder, which converts UTF-8 to the page, stops with an *UnsupportedCharError on a character that the page cannot represent; encoding.ReplaceUnsupported turns that error into the page's substitution byte. Both follow the conventions of golang.org/x/text/encoding/charmap.

A CodePage is immutable and safe for concurrent use.

func FromCharmap

func FromCharmap(source *charmap.Charmap) *CodePage

FromCharmap returns the code page that stands for the same characters as a code page of golang.org/x/text/encoding/charmap, under the same name. Bytes that the charmap decodes into U+FFFD stand for no character.

func NewCodePage

func NewCodePage(name string, chars *[TableSize]rune, options ...PageOption) (*CodePage, error)

NewCodePage returns the code page called name in which byte b stands for chars[b], or for no character when chars[b] is NoChar.

When several bytes stand for the same character, the encoder uses the lowest one that is not marked DecodeOnly. U+FFFD cannot be assigned to a byte: the decoder reserves it for bytes that stand for no character.

func ParseCodePage

func ParseCodePage(name string, data []byte) (*CodePage, error)

ParseCodePage reads the code page called name from a mapping file, the format of the Unicode consortium mapping tables. Each line maps a byte to a character, both in hexadecimal with a 0x prefix, separated by spaces or tabs:

0xC1	0x0041	# LATIN CAPITAL LETTER A

A line with a byte and no character, and a byte without a line, stand for no character. Text after # is a comment; blank lines are ignored; lines may end with LF or CRLF. A byte may appear only once, and a byte cannot stand for a sequence of characters.

The comment "decode only" on a mapping line marks its byte DecodeOnly:

0x51	0x0E48	# decode only

func (*CodePage) Byte

func (p *CodePage) Byte(char rune) (byte, bool)

Byte returns the byte that stands for char, and false when the page cannot represent char.

func (*CodePage) Char

func (p *CodePage) Char(b byte) (rune, bool)

Char returns the character that b stands for, and false when b stands for no character.

func (*CodePage) Chars

func (p *CodePage) Chars() [TableSize]rune

Chars returns the character of every byte, NoChar marking the bytes that stand for no character.

func (*CodePage) IsDecodeOnly

func (p *CodePage) IsDecodeOnly(b byte) bool

IsDecodeOnly reports whether b is marked DecodeOnly: the decoder reads it, the encoder never writes it.

func (*CodePage) MarshalText

func (p *CodePage) MarshalText() ([]byte, error)

MarshalText returns the mapping file of the page, in the format read by ParseCodePage: one line per byte that stands for a character.

func (*CodePage) Name

func (p *CodePage) Name() string

Name returns the name of the code page.

func (*CodePage) NewDecoder

func (p *CodePage) NewDecoder() *encoding.Decoder

NewDecoder returns a decoder that converts the page to UTF-8.

func (*CodePage) NewEncoder

func (p *CodePage) NewEncoder() *encoding.Encoder

NewEncoder returns an encoder that converts UTF-8 to the page.

func (*CodePage) Replacement

func (p *CodePage) Replacement() byte

Replacement returns the byte that encoding.ReplaceUnsupported writes in place of a character that the page cannot represent: the page's SUB control character if it has one, its question mark otherwise.

func (*CodePage) String

func (p *CodePage) String() string

String returns the name of the code page.

type Conversion

type Conversion struct {
	// Source and Target are the names of the character sets of the data and
	// of the result. When they name the same character set, the content is
	// copied as it is, byte for byte, without being decoded: only the records
	// are laid out again, their separators and padding written in that
	// character set.
	Source, Target string

	// Table is the name of a translation table that converts the data byte by
	// byte, in place of Source and Target. These are then single-byte code
	// pages that only serve to write separators and padding, and may be left
	// empty: separators and padding are then taken as bytes, and padding has
	// no default.
	Table string

	// Input and Output are the record formats of the data and of the result;
	// their zero value converts the data as a whole. Their separators and
	// padding are characters, which the converter writes in Source and
	// Target, with code units of the right size; padding defaults to a space.
	// On output, a record too long is cut between two characters.
	Input, Output RecordFormat

	// Substitute writes the substitution character of Target in place of a
	// character that Target cannot represent, instead of failing.
	Substitute bool
}

Conversion describes a conversion made by Registry.NewConverter.

type Converter

type Converter struct {
	transform.Transformer
}

Converter converts data as a Conversion describes it. It implements transform.Transformer.

type Entry

type Entry struct {
	// Name is the name of the entry.
	Name string
	// Kind tells what the entry is.
	Kind Kind
	// EBCDIC reports a single-byte code page of the EBCDIC family.
	EBCDIC bool
	// Builtin reports an entry of the library, which cannot be changed or
	// removed.
	Builtin bool
	// Deprecated reports a built-in entry kept so that existing settings
	// still work, which a list offered for new settings should leave out.
	Deprecated bool
	// Source tells where the definition of a built-in entry comes from.
	Source string
}

Entry describes a character set or a translation table of a Registry.

type Kind

type Kind int

Kind tells what a registry entry is.

const (
	// KindUnicode is a Unicode encoding: UTF-8, UTF-16 or UTF-32.
	KindUnicode Kind = iota + 1
	// KindSingleByte is a single-byte code page, also available as a
	// [CodePage].
	KindSingleByte
	// KindMultiByte is a multi-byte encoding, such as Shift JIS or GB18030.
	KindMultiByte
	// KindTable is a byte-to-byte translation [Table].
	KindTable
)

Kinds of registry entries.

func (Kind) String

func (k Kind) String() string

String returns the name of the kind.

type PageOption

type PageOption func(*CodePage)

PageOption adjusts a code page built by NewCodePage.

func DecodeOnly

func DecodeOnly(octets ...byte) PageOption

DecodeOnly marks bytes that the decoder reads but the encoder never writes. Some pages place a character at two positions: both decode into the character, which encodes into the other position only.

type RecordFormat

type RecordFormat struct {
	// contains filtered or unexported fields
}

RecordFormat describes how data is cut into records, on input, and laid out, on output. The zero value is Stream: the data is a single record.

A RecordFormat is built by Stream, Separated, SameSeparators, Fixed, RDW or Lengths, then adjusted with its methods. Separators and padding are bytes of the data, in its encoding: the EBCDIC space is "\x40", the UTF-16LE line feed "\n\x00".

func Fixed

func Fixed(length int) RecordFormat

Fixed is the format of records of length bytes, with nothing between them. On input, the data must end with a whole record. On output, a shorter record is completed with the padding given by RecordFormat.WithPad, and a longer one makes the conversion fail, unless RecordFormat.Truncating.

func Lengths added in v0.3.0

func Lengths(lengths iter.Seq[int]) RecordFormat

Lengths is the input format of records whose lengths are known apart from the data, such as the articles of a PeSIT transfer, whose lengths the receiver keeps: the records follow each other with nothing between them, each as long as the next value of lengths, in bytes. The data must hold exactly the records that lengths describes; a record may be empty. lengths is read again from its start when the transformer is reset. The lengths count the bytes that reach the records: a byte order mark that a Converter reads apart is not part of them.

func RDW

func RDW() RecordFormat

RDW is the format of variable-length records preceded by a record descriptor word, as z/OS writes them: 4 bytes, the length of the record header included as a 2-byte big-endian number, then 2 zero bytes. Spanned records and block descriptor words are not supported. On output, a record holds at most 32,756 bytes of content.

func SameSeparators

func SameSeparators() RecordFormat

SameSeparators is an output format only, for Separated input: each record is followed by the separator that ended it on input, unchanged. The separators thus stay out of the conversion, which applies to the content of the records only.

func Separated

func Separated(separators ...string) RecordFormat

Separated is the format of records ended by a separator. On input, a record ends with any of the separators, the first that occurs, and the longest at the same place, so that Separated(CRLF, LF) reads both kinds of lines; the last record may lack its separator. On output, each record is followed by the first separator, except a last record that had none on input.

func Stream

func Stream() RecordFormat

Stream is the format of data that is not cut into records: on input, the data is a single record; on output, records are written one after the other, with nothing between them.

func (RecordFormat) Reporting added in v0.3.0

func (f RecordFormat) Reporting(report func(length int)) RecordFormat

Reporting makes an output format call report with the length in bytes of each record written: its content with its padding, without its separator nor its record descriptor word. It serves to keep the lengths of the records, such as those of the articles of a PeSIT transfer.

func (RecordFormat) TrimmingPad

func (f RecordFormat) TrimmingPad() RecordFormat

TrimmingPad makes the input of fixed-length, RDW or Lengths records remove the padding repeated at the end of each record.

func (RecordFormat) Truncating

func (f RecordFormat) Truncating() RecordFormat

Truncating makes the output of fixed-length or RDW records cut a record that is too long, instead of failing. The cut falls at a multiple of the code unit; a Converter makes it fall between two characters.

func (RecordFormat) WithPad

func (f RecordFormat) WithPad(pad string) RecordFormat

WithPad sets the padding of fixed-length records, one code unit long: on output, it completes short records; on input, RecordFormat.TrimmingPad removes it from the end of records.

func (RecordFormat) WithUnit

func (f RecordFormat) WithUnit(size int) RecordFormat

WithUnit sets the size of the code units of the data: 1 (the default), 2 for UTF-16 or 4 for UTF-32. Separators are only recognized at the start of a code unit, so that a byte sequence that straddles two characters is never taken for one; lengths are multiples of the unit.

type RecordTransformer

type RecordTransformer struct {
	// contains filtered or unexported fields
}

RecordTransformer converts data record by record; it implements transform.Transformer. It streams the content of separated and stream records, and holds a whole record only when its format bounds its length: fixed-length and RDW input, fixed-length and RDW output.

func NewRecordTransformer

func NewRecordTransformer(
	input RecordFormat, convert transform.Transformer, output RecordFormat,
) (*RecordTransformer, error)

NewRecordTransformer returns a transformer that reads records laid out as input, converts the content of each record with convert, and writes the records laid out as output. A nil convert leaves the content unchanged.

The content of each record is converted on its own: convert is reset at the start of each record and receives its end as the end of the data. With an encoding that writes a byte order mark, the mark is thus written once per record; a Converter writes it once per file instead.

Errors on a record name its number, counted from 1.

func (*RecordTransformer) Reset

func (t *RecordTransformer) Reset()

Reset implements transform.Transformer.

func (*RecordTransformer) Transform

func (t *RecordTransformer) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error)

Transform implements transform.Transformer.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry holds the character sets and translation tables available by name: the built-in ones, and those registered at run time with Registry.SetCodePage and Registry.SetTable. It keeps everything in memory and stores nothing; an application that keeps its own pages and tables registers them when it starts.

A name is looked up as written, then regardless of case, spaces, hyphens and underscores ("utf8" finds "UTF-8", "iso-8859-15" finds "ISO 8859-15"), and IBM code pages also by number ("IBM1147", "CP037", "IBM-01147"). A registered name may not be taken for the name of another entry in these ways.

A Registry is safe for concurrent use.

Example
package main

import (
	"fmt"

	"code.waarp.fr/lib/transcode"
)

func main() {
	registry := transcode.NewRegistry()

	// Names are found regardless of case and punctuation, and IBM code
	// pages by number.
	entry, _ := registry.Lookup("ibm-1147")
	fmt.Println(entry.Name, "-", entry.Kind)

	page, err := registry.CodePage(entry.Name)
	if err != nil {
		fmt.Println(err)

		return
	}

	encoded, _ := page.NewEncoder().String("été")
	fmt.Printf("% X\n", encoded)
}
Output:
IBM Code Page 1147 - single-byte code page
C0 A3 C0

func NewRegistry

func NewRegistry() *Registry

NewRegistry returns a registry that holds the built-in character sets.

func (*Registry) CodePage

func (r *Registry) CodePage(name string) (*CodePage, error)

CodePage returns the single-byte code page called name.

func (*Registry) Encoding

func (r *Registry) Encoding(name string) (encoding.Encoding, error)

Encoding returns the character set called name. For the single-byte code pages and the encodings that come from golang.org/x/text, it returns the object of golang.org/x/text itself, so that they behave exactly as with that package.

func (*Registry) List

func (r *Registry) List() []Entry

List returns every entry: the built-in ones, Unicode encodings first, then single-byte code pages, EBCDIC pages by number, and multi-byte encodings; then the registered ones, by name.

func (*Registry) Lookup

func (r *Registry) Lookup(name string) (Entry, bool)

Lookup returns the entry called name.

func (*Registry) NewConverter

func (r *Registry) NewConverter(conversion *Conversion) (*Converter, error)

NewConverter returns the converter that makes conversion.

Without records, a conversion between two character sets is the chain of the decoder of Source and the encoder of Target, as with golang.org/x/text. With records, the content of each record is converted on its own, and a byte order mark is read or written once, at the start of the data: on input, a mark of the other byte order fails.

Example
package main

import (
	"fmt"

	"code.waarp.fr/lib/transcode"
	"golang.org/x/text/transform"
)

func main() {
	registry := transcode.NewRegistry()

	// UTF-8 lines to the French EBCDIC page 1147, keeping the line feeds for
	// a reader that cuts the result into lines at 0x0A.
	converter, err := registry.NewConverter(&transcode.Conversion{
		Source: "UTF-8",
		Target: "IBM1147",
		Input:  transcode.Separated(transcode.LF),
		Output: transcode.SameSeparators(),
	})
	if err != nil {
		fmt.Println(err)

		return
	}

	converted, _, _ := transform.String(converter, "été\nhiver\n")
	fmt.Printf("% X\n", converted)
}
Output:
C0 A3 C0 0A 88 89 A5 85 99 0A

func (*Registry) Remove

func (r *Registry) Remove(name string) error

Remove removes the registered entry called name.

func (*Registry) SetCodePage

func (r *Registry) SetCodePage(page *CodePage) error

SetCodePage registers page under its name, or replaces the page registered under that name.

func (*Registry) SetTable

func (r *Registry) SetTable(name string, table *Table) error

SetTable registers a copy of table under name, or replaces the table registered under that name.

func (*Registry) Table

func (r *Registry) Table(name string) (Table, error)

Table returns a copy of the translation table called name.

type Table

type Table [TableSize]byte

Table is a single-byte translation table: byte b is translated into t[b].

A table converts data byte by byte, without going through Unicode. It can therefore reproduce exactly any existing single-byte translation, including the handling of control characters and of bytes that a character set leaves undefined.

A table has two external forms, both read by ParseTable: the binary form, 256 bytes where the byte at offset b is the translation of b, and the text form, 512 hexadecimal digits, two per byte in the same order.

The zero value translates every byte into 0x00; IdentityTable returns a table that leaves data unchanged. *Table implements transform.Transformer; a table must not be modified while a transformation uses it.

func IdentityTable

func IdentityTable() Table

IdentityTable returns the table that translates every byte into itself.

func ParseTable

func ParseTable(data []byte) (Table, error)

ParseTable reads a table from its binary form, which is exactly 256 bytes long, or from its text form, described in Table.UnmarshalText.

Example
package main

import (
	"fmt"

	"code.waarp.fr/lib/transcode"
	"golang.org/x/text/transform"
)

func main() {
	// A table in text form that turns lower-case ASCII letters into
	// upper-case ones and leaves every other byte unchanged.
	upper := transcode.IdentityTable()
	for b := byte('a'); b <= 'z'; b++ {
		upper[b] = b - 'a' + 'A'
	}

	text, _ := upper.MarshalText()

	table, err := transcode.ParseTable(text)
	if err != nil {
		fmt.Println(err)

		return
	}

	converted, _, _ := transform.String(&table, "Hello, world")
	fmt.Println(converted)
}
Output:
HELLO, WORLD

func TableBetween

func TableBetween(source, target *CodePage, options ...TableOption) (Table, error)

TableBetween returns the table that converts data from the source code page to the target code page: each byte is translated into the byte that stands for the same character in the target page. The table is computed once, then converts data byte by byte.

A byte has no equivalent when it stands for no character in the source page (ErrUndefinedByte), or when the target page cannot represent its character (ErrUnsupportedChar). TableBetween then fails, and the error names every such byte, unless WithSubstitute is given.

func (*Table) Invert

func (t *Table) Invert() (Table, error)

Invert returns the table that undoes t: applying t, then the result, leaves data unchanged. It fails with ErrNotInvertible when t translates two bytes into the same value, since the reverse translation would then be ambiguous; the error names the conflicting bytes.

func (*Table) MarshalBinary

func (t *Table) MarshalBinary() ([]byte, error)

MarshalBinary returns the binary form of t: 256 bytes, the byte at offset b being the translation of b.

func (*Table) MarshalText

func (t *Table) MarshalText() ([]byte, error)

MarshalText returns the text form of t: 512 upper-case hexadecimal digits, two per byte in byte order, on a single line.

func (*Table) Reset

func (*Table) Reset()

Reset implements transform.Transformer. A table keeps no state.

func (*Table) Then

func (t *Table) Then(next *Table) Table

Then returns the table that applies t, then next: byte b is translated into next[t[b]].

func (*Table) Transform

func (t *Table) Transform(dst, src []byte, _ bool) (nDst, nSrc int, err error)

Transform implements transform.Transformer. It translates as many bytes of src as dst can hold, and returns transform.ErrShortDst when dst is shorter than src.

func (*Table) Translate

func (t *Table) Translate(p []byte)

Translate translates p in place.

func (*Table) UnmarshalBinary

func (t *Table) UnmarshalBinary(data []byte) error

UnmarshalBinary reads the binary form of a table, which must be exactly 256 bytes long. On error, t is left unchanged.

func (*Table) UnmarshalText

func (t *Table) UnmarshalText(text []byte) error

UnmarshalText reads the text form of a table: 512 hexadecimal digits, two per byte in byte order. Digits may be upper or lower case. Spaces, tabs and line breaks between digits are ignored, so the digits may be split over several lines. A hyphen that ends a line, which configuration exports use to continue a long value on the next line, is ignored too:

00010203372D2E2F1605250B0C0D0E0F -
101112133C3D322618193F271C1D1E1F -

On error, t is left unchanged.

type TableOption

type TableOption func(*tableSettings)

TableOption adjusts how TableBetween builds a table.

func WithSubstitute

func WithSubstitute(substitute byte) TableOption

WithSubstitute makes TableBetween translate into substitute every byte that has no equivalent in the target page, instead of failing.

type UnsupportedCharError

type UnsupportedCharError struct {
	// Page is the name of the code page.
	Page string
	// Char is the character that the page cannot represent.
	Char rune
	// contains filtered or unexported fields
}

UnsupportedCharError reports a character that a code page cannot represent. Invalid UTF-8 is reported as U+FFFD. The error matches ErrUnsupportedChar with errors.Is, and encoding.ReplaceUnsupported replaces the character with the page's substitution byte.

func (*UnsupportedCharError) Error

func (e *UnsupportedCharError) Error() string

func (*UnsupportedCharError) Replacement

func (e *UnsupportedCharError) Replacement() byte

Replacement returns the page's substitution byte, which encoding.ReplaceUnsupported writes in place of the character.

func (*UnsupportedCharError) Unwrap

func (*UnsupportedCharError) Unwrap() error

Unwrap returns ErrUnsupportedChar.

Source Files

  • between.go
  • builtin.go
  • codepage.go
  • converter.go
  • doc.go
  • ebcdic_tables.go
  • mapfile.go
  • records.go
  • registry.go
  • table.go

Directories

Path Synopsis
internal
gen command
Command gen generates ebcdic_tables.go, the EBCDIC code pages built into the transcode package, from the mapping files of the ICU project.
Command gen generates ebcdic_tables.go, the EBCDIC code pages built into the transcode package, from the mapping files of the ICU project.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL