jsontext

package

v0.0.0-...-c27c302 Latest Latest Go to latest Published: Aug 20, 2026 License: BSD-3-Clause Imports: 2 Imported by: 254

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/go-json-experiment/json

Links

Open Source Insights

Documentation ¶

Overview ¶

Package jsontext implements syntactic processing of JSON as specified in RFC 4627, RFC 7159, RFC 7493, RFC 8259, and RFC 8785. JSON is a simple data interchange format that can represent primitive data types such as booleans, strings, and numbers, in addition to structured data types such as objects and arrays.

The Encoder and Decoder types are used to encode or decode a stream of JSON tokens or values.

Tokens and Values ¶

A JSON token refers to the basic structural elements of JSON:

a JSON literal (i.e., null, true, or false)
a JSON string (e.g., "hello, world!")
a JSON number (e.g., 123.456)
a begin or end delimiter for a JSON object (i.e., '{' or '}')
a begin or end delimiter for a JSON array (i.e., '[' or ']')

A JSON token is represented by the Token type in Go. Technically, there are two additional structural characters (i.e., ':' and ','), but there is no Token representation for them since their presence can be inferred by the structure of the JSON grammar itself. For example, there must always be an implicit colon between the name and value of a JSON object member.

A JSON value refers to a complete unit of JSON data:

a JSON literal, string, or number
a JSON object (e.g., `{"name":"value"}`)
a JSON array (e.g., `[1,2,3]`)

A JSON value is represented by the Value type in Go and is a []byte containing the raw textual representation of the value. There is some overlap between tokens and values as both contain literals, strings, and numbers. However, only a value can represent the entirety of a JSON object or array.

The Encoder and Decoder types contain methods to read or write the next Token or Value in a sequence. They maintain a state machine to validate whether the sequence of JSON tokens and/or values produces valid JSON. Options may be passed to the NewEncoder or NewDecoder constructors to configure the behavior of encoding and decoding.

Terminology ¶

The terms "encode" and "decode" are used for syntactic functionality that is concerned with processing JSON based on its grammar, and the terms "marshal" and "unmarshal" are used for semantic functionality that determines the meaning of JSON values as Go values and vice versa. This package deals with JSON syntax, while encoding/json/v2 deals with JSON semantics. The goal is to provide a clear distinction between functionality that is purely concerned with encoding versus that of marshaling. For example, one can directly encode a stream of JSON tokens without needing to marshal a concrete Go value representing them. Similarly, one can decode a stream of JSON tokens without needing to unmarshal them into a concrete Go value.

This package uses JSON terminology when discussing JSON, which may differ from related concepts in Go or elsewhere in computing literature.

a JSON "object" refers to an unordered collection of name/value members.
a JSON "array" refers to an ordered sequence of elements.
a JSON "value" refers to either a literal (i.e., null, false, or true), string, number, object, or array.

See RFC 8259 for more information.

Specifications ¶

Relevant specifications include RFC 4627, RFC 7159, RFC 7493, RFC 8259, and RFC 8785. Each RFC is generally a stricter subset of another RFC. In increasing order of strictness:

RFC 4627 and RFC 7159 do not require (but recommend) the use of UTF-8 and also do not require (but recommend) that object names be unique.
RFC 8259 requires the use of UTF-8, but does not require (but recommends) that object names be unique.
RFC 7493 requires the use of UTF-8 and also requires that object names be unique.
RFC 8785 defines a canonical representation. It requires the use of UTF-8 and also requires that object names be unique and in a specific ordering. It specifies exactly how strings and numbers must be formatted.

The primary difference between RFC 4627 and RFC 7159 is that the former restricted top-level values to only JSON objects and arrays, while RFC 7159 and subsequent RFCs permit top-level values to additionally be JSON nulls, booleans, strings, or numbers.

By default, this package operates on RFC 7493, but can be configured to operate according to the other RFC specifications. RFC 7493 is a stricter subset of RFC 8259 and fully compliant with it. In particular, it makes specific choices about behavior that RFC 8259 leaves as undefined in order to ensure greater interoperability.

Security Considerations ¶

See the "Security Considerations" section in encoding/json/v2.

Index ¶

Constants
Variables
func AppendFormat[Bytes ~[]byte | ~string](dst []byte, src Bytes, opts ...Options) ([]byte, error)
func AppendQuote[Bytes ~[]byte | ~string](dst []byte, src Bytes) ([]byte, error)
func AppendUnquote[Bytes ~[]byte | ~string](dst []byte, src Bytes) ([]byte, error)
type Decoder
- func NewDecoder(r io.Reader, opts ...Options) *Decoder
type Encoder
- func NewEncoder(w io.Writer, opts ...Options) *Encoder
type Kind
type Options
type Pointer
type SyntacticError
type Token
type Value

Constants ¶

View Source

const (
	KindInvalid     = jsontext.KindInvalid
	KindNull        = jsontext.KindNull
	KindFalse       = jsontext.KindFalse
	KindTrue        = jsontext.KindTrue
	KindString      = jsontext.KindString
	KindNumber      = jsontext.KindNumber
	KindBeginObject = jsontext.KindBeginObject
	KindEndObject   = jsontext.KindEndObject
	KindBeginArray  = jsontext.KindBeginArray
	KindEndArray    = jsontext.KindEndArray
)

Variables ¶

View Source

var (
	Null        = jsontext.Null
	False       = jsontext.False
	True        = jsontext.True
	BeginObject = jsontext.BeginObject
	EndObject   = jsontext.EndObject
	BeginArray  = jsontext.BeginArray
	EndArray    = jsontext.EndArray
)

View Source

var ErrDuplicateName = jsontext.ErrDuplicateName

ErrDuplicateName indicates that a JSON token could not be encoded or decoded because it results in a duplicate JSON object name. This error is directly wrapped within a SyntacticError when produced.

The name of a duplicate JSON object member can be extracted as:

err := ...
serr, ok := errors.AsType[*jsontext.SyntacticError](err)
if ok && serr.Err == jsontext.ErrDuplicateName {
	ptr := serr.JSONPointer // JSON pointer to duplicate name
	name := ptr.LastToken() // duplicate name itself
	...
}

This error is only returned if AllowDuplicateNames is false.

View Source

var ErrNonStringName = jsontext.ErrNonStringName

ErrNonStringName indicates that a JSON token could not be encoded or decoded because it is not a string, as required for JSON object names according to RFC 8259, section 4. This error is directly wrapped within a SyntacticError when produced.

Functions ¶

func AppendFormat ¶

func AppendFormat[Bytes ~[]byte | ~string](dst []byte, src Bytes, opts ...Options) ([]byte, error)

AppendFormat formats the JSON value in src and appends it to dst according to the specified options. See [Value.Format] for more details about the formatting behavior.

The dst and src may overlap. If an error is reported, then the entirety of src is appended to dst.

func AppendQuote ¶

func AppendQuote[Bytes ~[]byte | ~string](dst []byte, src Bytes) ([]byte, error)

AppendQuote appends a double-quoted JSON string literal representing src to dst and returns the extended buffer. It uses the minimal string representation per RFC 8785, section 3.2.2.2. Invalid UTF-8 bytes are replaced with the Unicode replacement character and an error is returned at the end indicating the presence of invalid UTF-8. The dst must not overlap with the src.

func AppendUnquote ¶

func AppendUnquote[Bytes ~[]byte | ~string](dst []byte, src Bytes) ([]byte, error)

AppendUnquote appends the decoded interpretation of src as a double-quoted JSON string literal to dst and returns the extended buffer. The input src must be a JSON string without any surrounding whitespace. Invalid UTF-8 bytes are replaced with the Unicode replacement character and an error is returned at the end indicating the presence of invalid UTF-8. Any trailing bytes after the JSON string literal results in an error. The dst must not overlap with the src.

Types ¶

type Decoder ¶

type Decoder = jsontext.Decoder

Decoder is a streaming decoder for raw JSON tokens and values. It is used to read a stream of top-level JSON values, each separated by optional whitespace characters.

[Decoder.ReadToken] and [Decoder.ReadValue] calls may be interleaved. For example, the following JSON value:

{"name":"value","array":[null,false,true,3.14159],"object":{"k":"v"}}

can be parsed with the following calls (ignoring errors for brevity):

d.ReadToken() // {
d.ReadToken() // "name"
d.ReadToken() // "value"
d.ReadValue() // "array"
d.ReadToken() // [
d.ReadToken() // null
d.ReadToken() // false
d.ReadValue() // true
d.ReadToken() // 3.14159
d.ReadToken() // ]
d.ReadValue() // "object"
d.ReadValue() // {"k":"v"}
d.ReadToken() // }

The above is one of many possible sequences of calls and may not represent the most sensible method to call for any given token/value. For example, it is probably more common to call [Decoder.ReadToken] to obtain a string token for object names.

func NewDecoder ¶

func NewDecoder(r io.Reader, opts ...Options) *Decoder

NewDecoder constructs a new streaming decoder reading from r.

If r is a bytes.Buffer, then the decoder parses directly from the buffer without first copying the contents to an intermediate buffer. Additional writes to the buffer must not occur while the decoder is in use.

type Encoder ¶

type Encoder = jsontext.Encoder

Encoder is a streaming encoder to raw JSON tokens and values. It is used to write a stream of top-level JSON values, each terminated with a newline character.

[Encoder.WriteToken] and [Encoder.WriteValue] calls may be interleaved. For example, the following JSON value:

{"name":"value","array":[null,false,true,3.14159],"object":{"k":"v"}}

can be composed with the following calls (ignoring errors for brevity):

e.WriteToken(BeginObject)        // {
e.WriteToken(String("name"))     // "name"
e.WriteToken(String("value"))    // "value"
e.WriteValue(Value(`"array"`))   // "array"
e.WriteToken(BeginArray)         // [
e.WriteToken(Null)               // null
e.WriteToken(False)              // false
e.WriteValue(Value("true"))      // true
e.WriteToken(Float(3.14159))     // 3.14159
e.WriteToken(EndArray)           // ]
e.WriteValue(Value(`"object"`))  // "object"
e.WriteValue(Value(`{"k":"v"}`)) // {"k":"v"}
e.WriteToken(EndObject)          // }

The above is one of many possible sequences of calls and may not represent the most sensible method to call for any given token/value. For example, it is probably more common to call [Encoder.WriteToken] with a string for object names.

func NewEncoder ¶

func NewEncoder(w io.Writer, opts ...Options) *Encoder

NewEncoder constructs a new streaming encoder writing to w configured with the provided options. It flushes the internal buffer when the buffer is sufficiently full or when a top-level value has been written.

If w is a bytes.Buffer, then the encoder appends directly into the buffer without copying the contents from an intermediate buffer.

type Kind ¶

type Kind = jsontext.Kind

A Kind represents the kind of a JSON token.

A Kind is a single byte, which is conveniently the first byte of that kind's symbol in the grammar (except for numbers, which are always represented with '0').

type Options ¶

type Options = jsontext.Options

Options configures NewEncoder, [Encoder.Reset], NewDecoder, and [Decoder.Reset] with specific features. Each function takes in a variadic list of options, where properties set in later options override the value of previously set properties.

There is a single Options type, which is used with both encoding and decoding. Some options affect both operations, while others only affect one operation:

AllowDuplicateNames affects encoding and decoding
AllowInvalidUTF8 affects encoding and decoding
EscapeForHTML affects encoding only
EscapeForJS affects encoding only
PreserveRawStrings affects encoding only
CanonicalizeRawInts affects encoding only
CanonicalizeRawFloats affects encoding only
ReorderRawObjects affects encoding only
SpaceAfterColon affects encoding only
SpaceAfterComma affects encoding only
Multiline affects encoding only
WithIndent affects encoding only
WithIndentPrefix affects encoding only

Options that do not affect a particular operation are ignored.

The Options type is identical to encoding/json.Options and encoding/json/v2.Options. Options from the other packages may be passed to functionality in this package, but are ignored. Options from this package may be used with the other packages.

func AllowDuplicateNames ¶

func AllowDuplicateNames(v bool) Options

AllowDuplicateNames specifies that JSON objects may contain duplicate member names. Disabling the duplicate name check may provide performance benefits, but breaks compliance with RFC 7493, section 2.3. The input or output will still be compliant with RFC 8259, which leaves the handling of duplicate names as unspecified behavior.

This affects either encoding or decoding.

func AllowInvalidUTF8 ¶

func AllowInvalidUTF8(v bool) Options

AllowInvalidUTF8 specifies that JSON strings may contain invalid UTF-8, which will be mangled as the Unicode replacement character, U+FFFD. This causes the encoder or decoder to break compliance with RFC 7493, section 2.1, and RFC 8259, section 8.1.

This affects either encoding or decoding.

func CanonicalizeRawFloats ¶

func CanonicalizeRawFloats(v bool) Options

CanonicalizeRawFloats specifies that when encoding a raw JSON floating-point number (i.e., a number with a fraction or exponent) in a Token or Value, the number is canonicalized according to RFC 8785, section 3.2.2.3. As a special case, the number -0 is canonicalized as 0.

JSON numbers are treated as IEEE 754 double precision numbers. It is safe to canonicalize a serialized single precision number and parse it back as a single precision number and expect the same value. If a number exceeds ±1.7976931348623157e+308, which is the maximum finite number, then it is saturated at that value and formatted as such.