Documentation
¶
Overview ¶
Package text provides functionalities to manipulate texts.
Index ¶
- Constants
- type BlockReader
- type Decoder
- type DecoderOption
- type DefaultDecoder
- type Index
- type Lines
- type LinesInput
- type MultiLineValue
- func NewMultiLineValue[T MultiLineValueInput](v T, decoder Decoder) MultiLineValue
- func NewMultiLineValueFromIndex(i Index, decoder Decoder) MultiLineValue
- func NewMultiLineValueFromIndices(indices []Index, decoder Decoder) MultiLineValue
- func NewMultiLineValueFromString(s string, decoder Decoder) MultiLineValue
- func (v MultiLineValue) Bytes(source []byte) []byte
- func (v MultiLineValue) Index() Index
- func (v MultiLineValue) Indices() []Index
- func (v MultiLineValue) IsEmpty() bool
- func (v MultiLineValue) IsOwned() bool
- func (v MultiLineValue) Str(source []byte) string
- func (v MultiLineValue) Value(source []byte) string
- func (v MultiLineValue) WriteTo(w io.Writer, source []byte) (int, error)
- type MultiLineValueInput
- type Reader
- type Segment
- func (t Segment) Between(other Segment) Segment
- func (t Segment) Bytes(source []byte) []byte
- func (t Segment) IsEmpty() bool
- func (t Segment) Len() int
- func (t Segment) Str(source []byte) string
- func (t Segment) TrimLeftSpace(source []byte) Segment
- func (t Segment) TrimLeftSpaceWidth(width int, source []byte) Segment
- func (t Segment) TrimRightSpace(source []byte) Segment
- func (t Segment) WithStart(v int) Segment
- func (t Segment) WithStop(v int) Segment
- type SingleLineValue
- func NewSingleLineValue[T SingleLineValueInput](v T, decoder Decoder) SingleLineValue
- func NewSingleLineValueFromIndex(i Index, decoder Decoder) SingleLineValue
- func NewSingleLineValueFromSegment(seg Segment, decoder Decoder) SingleLineValue
- func NewSingleLineValueFromString(s string, decoder Decoder) SingleLineValue
- func (v SingleLineValue) Bytes(source []byte) []byte
- func (v SingleLineValue) Index() Index
- func (v SingleLineValue) Indices() []Index
- func (v SingleLineValue) IsEmpty() bool
- func (v SingleLineValue) IsOwned() bool
- func (v SingleLineValue) Str(source []byte) string
- func (v SingleLineValue) Value(source []byte) string
- func (v SingleLineValue) WithStop(stop int) SingleLineValue
- func (v SingleLineValue) WriteTo(w io.Writer, source []byte) (int, error)
- type SingleLineValueInput
- type Value
- type ValueBuilder
- func (b *ValueBuilder) AddIndex(i Index) *ValueBuilder
- func (b *ValueBuilder) AddSegment(seg Segment) *ValueBuilder
- func (b *ValueBuilder) Build() Value
- func (b *ValueBuilder) BuildMultiLine() MultiLineValue
- func (b *ValueBuilder) BuildSingleLine() SingleLineValue
- func (b *ValueBuilder) Decoder(d Decoder) *ValueBuilder
- func (b *ValueBuilder) OwnedBytes(bts []byte) *ValueBuilder
- func (b *ValueBuilder) OwnedString(s string) *ValueBuilder
Constants ¶
const EOF = byte(0xff)
EOF indicates the end of file.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type BlockReader ¶
type BlockReader interface {
Reader
// Reset resets current state and sets new segments to the reader.
Reset(segs []Segment)
}
A BlockReader interface is a reader that is optimized for Blocks.
func NewBlockReader ¶
func NewBlockReader(source []byte, segs []Segment, decoder Decoder) BlockReader
NewBlockReader returns a new BlockReader, decoding constructed Values using the given Decoder.
type Decoder ¶
type Decoder interface {
// Decode decodes the given byte slice and returns the decoded bytes.
Decode(b []byte) []byte
// DecodeTo decodes the given byte slice and writes the decoded bytes to w.
DecodeTo(w io.Writer, b []byte) (int, error)
}
A Decoder decodes a byte slice, e.g. resolving backslash escapes and character references, and returns or writes out the decoded bytes.
Note that this interface is not intended to 'normalize' a byte slice, but to decode it. For example, CommonMark requires that code spans normalize leading spaces, trailing spaces and newlines, but this interface does not do that.
var IdentityDecoder Decoder = &identityDecoder{}
IdentityDecoder is a decoder that does not perform any decoding and returns the bytes as is.
type DecoderOption ¶
type DecoderOption func(*decoderConfig)
DecoderOption is a function that configures a decoder.
func WithEscapedSpace ¶
func WithEscapedSpace() DecoderOption
WithEscapedSpace configures the decoder to treat escaped spaces as an empty character.
type DefaultDecoder ¶
type DefaultDecoder struct {
// contains filtered or unexported fields
}
DefaultDecoder is the default implementation of the Decoder interface.
CommonMark defines texts except some inline elements (e.g., code span, auto link, etc) can contain character references and backslash escapes. This decoder decodes them.
- decodes entity references (e.g., `& -> &`) - decodes numeric character references (e.g., `& -> &`) - decodes `\` escaped punctuations (e.g., `\* -> *`)
- If WithEscapedSpace is set, it will also decode `\ ` to an empty character.
func NewDecoder ¶
func NewDecoder(opts ...DecoderOption) *DefaultDecoder
NewDecoder creates a new Decoder with the given options.
type Index ¶
An Index represents a position range in a source byte slice. Unlike Segment, Index does not carry parsing metadata such as Padding or ForceNewline.
func NewIndexFromSegment ¶
NewIndexFromSegment returns a new Index derived from the given Segment, dropping any padding and ignoring ForceNewline.
type Lines ¶
type Lines struct {
// contains filtered or unexported fields
}
A Lines value holds the raw content of a block node. It is either an owned string or a sequence of Segment values copied from the parser's source-segment list.
Lines is used for block nodes whose rendered content is taken directly from the source (e.g. CodeBlock, FencedCodeBlock, HTMLBlock).
Lines is always 'raw'; it does not perform any decoding of the source content.
func NewLines ¶
func NewLines[T LinesInput](v T) Lines
NewLines returns a Lines from the given input, which may be a string, byte slice, or slice of Segment.
func NewLinesFromSegments ¶
NewLinesFromSegments returns a Lines backed by source segments.
func NewLinesFromString ¶
NewLinesFromString returns a Lines backed by an owned string.
func (*Lines) AppendSegment ¶
AppendSegment appends a Segment to the internal segment list.
func (Lines) Bytes ¶
Bytes returns the concatenated byte content of all segments, or the owned string. The returned byte slice is read-only and should not be modified.
func (Lines) IsOwned ¶
IsOwned returns true if this value is an owned string not derived from the source byte slice.
func (Lines) Segments ¶
Segments returns the internal segment slice, allowing callers to read and otherwise inspect the segment list.
type LinesInput ¶
LinesInput is a type constraint for types that can be converted to a Lines.
type MultiLineValue ¶
type MultiLineValue struct {
// contains filtered or unexported fields
}
A MultiLineValue holds a potentially multiline inline value that is either an owned string or a set of source position ranges. Use MultiLineValue for data that may span multiple lines per the CommonMark spec (e.g. link labels).
When constructing the byte value from source positions, the ranges are simply concatenated verbatim; no newline folding or other transformation is applied.
func NewMultiLineValue ¶
func NewMultiLineValue[T MultiLineValueInput](v T, decoder Decoder) MultiLineValue
NewMultiLineValue returns a MultiLineValue from the given input, which may be a string, byte slice, Index, or slice of Index, bound to the given Decoder.
func NewMultiLineValueFromIndex ¶
func NewMultiLineValueFromIndex(i Index, decoder Decoder) MultiLineValue
NewMultiLineValueFromIndex returns a MultiLineValue backed by a single source position, bound to the given Decoder.
func NewMultiLineValueFromIndices ¶
func NewMultiLineValueFromIndices(indices []Index, decoder Decoder) MultiLineValue
NewMultiLineValueFromIndices returns a MultiLineValue backed by source positions, bound to the given Decoder.
func NewMultiLineValueFromString ¶
func NewMultiLineValueFromString(s string, decoder Decoder) MultiLineValue
NewMultiLineValueFromString returns a MultiLineValue backed by an owned string, bound to the given Decoder.
func (MultiLineValue) Bytes ¶
func (v MultiLineValue) Bytes(source []byte) []byte
Bytes implements Value.Bytes .
func (MultiLineValue) Indices ¶
func (v MultiLineValue) Indices() []Index
Indices implements Value.Indices .
func (MultiLineValue) IsEmpty ¶
func (v MultiLineValue) IsEmpty() bool
IsEmpty implements Value.IsEmpty .
func (MultiLineValue) IsOwned ¶
func (v MultiLineValue) IsOwned() bool
IsOwned implements Value.IsOwned .
func (MultiLineValue) Str ¶
func (v MultiLineValue) Str(source []byte) string
Str implements Value.Str .
type MultiLineValueInput ¶
MultiLineValueInput is a type constraint for types that can be converted to a MultiLineValue.
type Reader ¶
type Reader interface {
io.RuneReader
// Source returns a source of the reader.
Source() []byte
// ResetPosition resets positions.
ResetPosition()
// Peek returns a byte at current position without advancing the internal pointer.
Peek() byte
// PeekLine returns the current line without advancing the internal pointer.
PeekLine() ([]byte, Segment)
// PrecedingCharacter returns a character just before current internal pointer.
PrecedingCharacter() rune
// ValueBetween returns a MultiLineValue covering the given [start, stop)
// byte range within the source.
//
// Value will be decoded using the Decoder bound to this reader.
ValueBetween(start, stop int) MultiLineValue
// Decoder returns the Decoder bound to this reader. Values constructed
// from positions within this reader's source should be bound to this
// Decoder.
Decoder() Decoder
// LineOffset returns a distance from the line head to current position.
LineOffset() int
// Position returns current line number and position.
Position() (int, Segment)
// SetPosition sets current line number and position.
SetPosition(int, Segment)
// SetPadding sets padding to the reader.
SetPadding(int)
// Advance advances the internal pointer.
Advance(int)
// AdvanceAndSetPadding advances the internal pointer and add padding to the
// reader.
AdvanceAndSetPadding(int, int)
// AdvanceToEOL advances the internal pointer to the end of line.
// If the line ends with a newline, it will be included in the segment.
// If the line ends with EOF, it will not be included in the segment.
AdvanceToEOL()
// AdvanceLine advances the internal pointer to the next line head.
AdvanceLine()
// SkipSpaces skips space characters and returns a non-blank line.
// If it reaches EOF, returns false.
SkipSpaces() (Segment, int, bool)
// SkipBlankLines skips blank lines and returns a non-blank line.
// If it reaches EOF, returns false.
SkipBlankLines() (Segment, int, bool)
// Match performs regular expression matching to current line.
Match(reg *regexp.Regexp) bool
// Match performs regular expression searching to current line.
FindSubMatch(reg *regexp.Regexp) [][]byte
}
A Reader interface provides abstracted method for reading text.
type Segment ¶
type Segment struct {
// Start is a start position of the segment.
Start int
// Stop is a stop position of the segment.
// This value should be excluded.
Stop int
// Padding is a padding length of the segment.
Padding int
// ForceNewline is true if the segment should be ended with a newline.
// Some elements(i.e. CodeBlock, FencedCodeBlock) does not trim trailing
// newlines. Spec defines that EOF is treated as a newline, so we need to
// add a newline to the end of the segment if it is not empty.
//
// i.e.:
//
// “`go
// const test = "test"
//
// This code does not close the code block and ends with EOF. In this case,
// we need to add a newline to the end of the last line like `const test = "test"\n`.
ForceNewline bool
}
A Segment struct holds information about source positions.
func NewSegmentPadding ¶
NewSegmentPadding returns a new Segment with the given padding.
func (Segment) TrimLeftSpace ¶
TrimLeftSpace returns a new segment by slicing off all leading space characters including padding.
func (Segment) TrimLeftSpaceWidth ¶
TrimLeftSpaceWidth returns a new segment by slicing off leading space characters until the given width.
func (Segment) TrimRightSpace ¶
TrimRightSpace returns a new segment by slicing off all trailing space characters.
type SingleLineValue ¶
type SingleLineValue struct {
// contains filtered or unexported fields
}
A SingleLineValue holds a single-line inline value that is either an owned string or a position range within a source byte slice. Use SingleLineValue for data that must not contain newlines per the CommonMark spec (e.g. link destinations, fenced code block info strings).
func NewSingleLineValue ¶
func NewSingleLineValue[T SingleLineValueInput](v T, decoder Decoder) SingleLineValue
NewSingleLineValue returns a Value from the given input, bound to the given Decoder.
func NewSingleLineValueFromIndex ¶
func NewSingleLineValueFromIndex(i Index, decoder Decoder) SingleLineValue
NewSingleLineValueFromIndex returns a Value backed by a source position, bound to the given Decoder.
func NewSingleLineValueFromSegment ¶
func NewSingleLineValueFromSegment(seg Segment, decoder Decoder) SingleLineValue
NewSingleLineValueFromSegment returns a Value backed by a source position derived from the given Segment, bound to the given Decoder.
func NewSingleLineValueFromString ¶
func NewSingleLineValueFromString(s string, decoder Decoder) SingleLineValue
NewSingleLineValueFromString returns a Value backed by an owned string, bound to the given Decoder. This function does not check whether the string contains newlines; it is the caller's responsibility to ensure that the string is single-line.
ValueBuilder.Build will automatically choose between SingleLineValue and MultiLineValue based on the presence of newlines in the string.
func (SingleLineValue) Bytes ¶
func (v SingleLineValue) Bytes(source []byte) []byte
Bytes implements Value.Bytes .
func (SingleLineValue) Index ¶
func (v SingleLineValue) Index() Index
Index implements Value.Index .
func (SingleLineValue) Indices ¶
func (v SingleLineValue) Indices() []Index
Indices implements Value.Indices .
func (SingleLineValue) IsEmpty ¶
func (v SingleLineValue) IsEmpty() bool
IsEmpty implements Value.IsEmpty .
func (SingleLineValue) IsOwned ¶
func (v SingleLineValue) IsOwned() bool
IsOwned implements Value.IsOwned .
func (SingleLineValue) Str ¶
func (v SingleLineValue) Str(source []byte) string
Str implements Value.Str .
func (SingleLineValue) Value ¶
func (v SingleLineValue) Value(source []byte) string
Value implements Value.Value .
func (SingleLineValue) WithStop ¶
func (v SingleLineValue) WithStop(stop int) SingleLineValue
WithStop returns a new SingleLineValue with the same start index but a different stop index. This method panics if the value is owned.
type SingleLineValueInput ¶
SingleLineValueInput is a type constraint for types that can be converted to a Value.
type Value ¶
type Value interface {
// Value returns the decoded string representation of this value.
Value(source []byte) string
// Bytes returns the source byte slice corresponding to this value.
Bytes(source []byte) []byte
// Str returns the string representation of this value.
Str(source []byte) string
// IsOwned returns true if this value is an owned string not derived from the source byte slice.
IsOwned() bool
// IsEmpty returns true if this value is empty, otherwise false.
IsEmpty() bool
// Index returns the source position of this value.
//
// The result is meaningful only when IsOwned() returns false.
// If Value is backed by multiple source positions, Index returns the first position.
Index() Index
// Indices returns the slice of Index values in this value.
//
// The result is meaningful only when IsOwned() returns false.
Indices() []Index
// WriteTo writes the value to the given buffer, using the [Value].Value.
WriteTo(w io.Writer, source []byte) (int, error)
}
A Value represents an inline value.
type ValueBuilder ¶
type ValueBuilder struct {
// contains filtered or unexported fields
}
ValueBuilder is a helper for building a Value.
func (*ValueBuilder) AddIndex ¶
func (b *ValueBuilder) AddIndex(i Index) *ValueBuilder
AddIndex adds an Index to the builder.
func (*ValueBuilder) AddSegment ¶
func (b *ValueBuilder) AddSegment(seg Segment) *ValueBuilder
AddSegment adds an Index derived from the given Segment to the builder.
func (*ValueBuilder) Build ¶
func (b *ValueBuilder) Build() Value
Build returns a Value from the accumulated state. If OwnedString or OwnedBytes was called, the result is backed by that string. Otherwise, the result is backed by the accumulated indices.
func (*ValueBuilder) BuildMultiLine ¶
func (b *ValueBuilder) BuildMultiLine() MultiLineValue
BuildMultiLine returns a MultiLineValue from the accumulated state. If OwnedString or OwnedBytes was called, the result is backed by that string. Otherwise, the result is backed by the accumulated indices.
func (*ValueBuilder) BuildSingleLine ¶
func (b *ValueBuilder) BuildSingleLine() SingleLineValue
BuildSingleLine returns a SingleLineValue from the accumulated state. If OwnedString or OwnedBytes was called, the result is backed by that string. Otherwise, the result is backed by the first accumulated index.
func (*ValueBuilder) Decoder ¶
func (b *ValueBuilder) Decoder(d Decoder) *ValueBuilder
Decoder sets the Decoder to be bound to the Value produced by Build, BuildSingleLine, or BuildMultiLine. If not set, the produced Value is bound to IdentityDecoder.
func (*ValueBuilder) OwnedBytes ¶
func (b *ValueBuilder) OwnedBytes(bts []byte) *ValueBuilder
OwnedBytes sets an owned string value from the given byte slice. When Build is called, the resulting Value will be backed by this string rather than any accumulated indices.
func (*ValueBuilder) OwnedString ¶
func (b *ValueBuilder) OwnedString(s string) *ValueBuilder
OwnedString sets an owned string value. When Build is called, the resulting Value will be backed by this string rather than any accumulated indices.