token

package
v0.1.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 29, 2025 License: Apache-2.0 Imports: 7 Imported by: 0

Documentation

Index

Constants

View Source
const (
	Null               = 0x0 // the type of JSON null
	Boolean            = 0x1 // a JSON boolean
	Number             = 0x2 // a JSON number
	String  ScalarType = 0x3 // a JSON string
)
View Source
const (
	TypeMask      = 0b00011
	KeyMask       = 0b00100
	AlnumMask     = 0b01000
	UnescapedMask = 0b10000
)

Variables

View Source
var (
	TrueScalar  = NewScalar(Boolean, trueBytes)
	FalseScalar = NewScalar(Boolean, falseBytes)
	NullScalar  = NewScalar(Null, nullBytes)
)

Functions

func CloneReadStream

func CloneReadStream(stream ReadStream) (*Cursor, *Cursor)

func ConsumeStream

func ConsumeStream(in <-chan Token, sink StreamSink) error

func StartStream

func StartStream(source StreamSource, handleError func(error)) <-chan Token

StartStream uses the source to start producing items and returns a new json stream where these items are produced. This is always fast because the source is computed in a goroutine.

As a source can produce errors, a handleError function can be provided.

func TransformStream

func TransformStream(in <-chan Token, transformer StreamTransformer) <-chan Token

TransformStream applies the transformer to the incoming json stream, returning a new json stream. This is always fast because the transformer is computed in a goroutine.

Types

type AccumulatorStream

type AccumulatorStream struct {
	// contains filtered or unexported fields
}

func NewAccumulatorStream

func NewAccumulatorStream() *AccumulatorStream

func (*AccumulatorStream) GetTokens

func (w *AccumulatorStream) GetTokens() []Token

func (*AccumulatorStream) Put

func (w *AccumulatorStream) Put(tok Token)

type ChannelReadStream

type ChannelReadStream <-chan Token

func (ChannelReadStream) Next

func (r ChannelReadStream) Next() Token

type ChannelWriteStream

type ChannelWriteStream chan<- Token

func (ChannelWriteStream) Put

func (w ChannelWriteStream) Put(tok Token)

type Cursor

type Cursor struct {
	// contains filtered or unexported fields
}

func NewCursorFromData

func NewCursorFromData(data []Token) *Cursor

func (*Cursor) Clone

func (c *Cursor) Clone() *Cursor

func (*Cursor) Detach

func (c *Cursor) Detach()

func (*Cursor) Next

func (c *Cursor) Next() Token

type CursorPool

type CursorPool struct {
	// contains filtered or unexported fields
}

func NewCursorPool

func NewCursorPool(stream ReadStream) *CursorPool

func (*CursorPool) AdvanceCursor

func (p *CursorPool) AdvanceCursor(c *Cursor) Token

func (*CursorPool) CloneCursor

func (p *CursorPool) CloneCursor(c *Cursor) *Cursor

func (*CursorPool) DetachCursor

func (p *CursorPool) DetachCursor(c *Cursor)

func (*CursorPool) NewCursor

func (p *CursorPool) NewCursor() *Cursor

type Elision

type Elision struct{}

Elision is not part of the JSON syntax but is used to remove contents from an array or an object but signal to the user that the content has been 'elided'.

func (*Elision) String

func (e *Elision) String() string

type EndArray

type EndArray struct{}

EndArray represents the end of a JSON array (introduced by '}') Values of type *EndArray implement the StreamItem interface.

func (*EndArray) String

func (e *EndArray) String() string

type EndObject

type EndObject struct{}

EndObject represents the end of a JSON object (introduced by '}') Values of type *EndObject implement the StreamItem interface.

func (*EndObject) String

func (e *EndObject) String() string

type ReadStream

type ReadStream interface {
	Next() Token
}

type Reader

type Reader interface {
	Read([]Token) (int, error)
}

type Scalar

type Scalar struct {

	// Literal representation of the value, e.g.
	// - the string "foo" is represented as []byte("\"foo\"")
	// - the number 123.5 is represented as []byte("132.5")
	// - the boolean true is represented as []byte("true")
	Bytes []byte

	// Type of the value
	TypeAndFlags uint8
}

Scalar is the type used to represent all scalar JSON values, i.e. - strings - numbers - booleans (to values) - null (a single value)

The type is encoded in the Type field, while the Bytes fields contains the literal representation of the value as found in the input.

func BoolScalar

func BoolScalar(b bool) *Scalar

func Float64Scalar

func Float64Scalar(x float64) *Scalar

func Int64Scalar

func Int64Scalar(n int64) *Scalar

func NewKey

func NewKey(tp ScalarType, bytes []byte) *Scalar

func NewScalar

func NewScalar(tp ScalarType, bytes []byte) *Scalar

func StringScalar

func StringScalar(s string) *Scalar

func ToScalar

func ToScalar(value any) (*Scalar, error)

func (*Scalar) Equal

func (s *Scalar) Equal(t *Scalar) bool

func (*Scalar) EqualsString

func (s *Scalar) EqualsString(str string) bool

EqualsString is a convenience method to check if a Scalar represents the passed string.

func (*Scalar) IsAlnum

func (s *Scalar) IsAlnum() bool

func (*Scalar) IsKey

func (s *Scalar) IsKey() bool

func (*Scalar) IsUnescaped

func (s *Scalar) IsUnescaped() bool

func (*Scalar) String

func (s *Scalar) String() string

func (*Scalar) ToGo

func (s *Scalar) ToGo() any

func (*Scalar) ToString

func (s *Scalar) ToString() string

panics if not a string

func (*Scalar) Type

func (s *Scalar) Type() ScalarType

type ScalarType

type ScalarType uint8

ScalarType encodes the four possible JSON scalar types.

type SliceReadStream

type SliceReadStream struct {
	// contains filtered or unexported fields
}

func NewSliceReadStream

func NewSliceReadStream(toks []Token) *SliceReadStream

func (*SliceReadStream) Next

func (r *SliceReadStream) Next() (tok Token)

type StartArray

type StartArray struct{}

StartArray represents the start of a JSON array (introduced by '['). Values of type *StartArray implement the StreamItem interface.

func (*StartArray) String

func (s *StartArray) String() string

type StartObject

type StartObject struct{}

StartObject represents the start of a JSON object (introduced by '{'). Values of type *StartObject implement the StreamItem interface.

func (*StartObject) String

func (s *StartObject) String() string

type StreamSink

type StreamSink interface {
	Consume(<-chan Token) error
}

type StreamSource

type StreamSource interface {
	Produce(chan<- Token) error
}

type StreamTransformer

type StreamTransformer interface {
	Transform(in <-chan Token, out WriteStream)
}

A StreamTransformer can transform a json stream into another. Use the TransformStream function to apply it.

type Token

type Token interface {
	fmt.Stringer
}

A Token is an item in a stream that encodes a JSON value For example, the JSON value

{"id": 123, "tags": ["important", "new"]}

would be represented by the stream of Token (in pseudocode for clarity):

{            -> StartObject
"id":        -> Scalar("id", String)
123,         -> Scalar(123, Number)
"tags":      -> Scalar("tags", String)
[            -> StartArray
"important", -> Scalar("important", String)
"new"        -> Scalar("new", String)
]            -> EndArray
}            -> EndObject

encoded of a JSON input into a stream of Token values, processing of this stream and outputting the outcome can be done concurrently using channels of Token values.

type WriteStream

type WriteStream interface {
	Put(Token)
}

type Writer

type Writer interface {
	Write([]Token)
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL