state

package
v0.0.0-...-eb263e1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 21, 2026 License: Apache-2.0, BSD-3-Clause, MIT Imports: 21 Imported by: 33

README

State Encoding and Decoding

The state package implements the encoding and decoding of data structures for go_stateify. This package is designed for use cases other than the standard encoding packages, e.g. gob and json. Principally:

  • This package operates on complex object graphs and accurately serializes and restores all relationships. That is, you can have things like: intrusive pointers, cycles, pointers into the middle of objects, slices sharing backing arrays, and pointer chains of arbitrary depths.

  • This package allows installing order-dependent load callbacks and then resolves that graph at load time, with cycle detection.

  • This package handles the resolution of interfaces, based on a registered type name.

  • Streams can only be loaded by the binary that saved them (more precisely, by a binary with the same registered types; this is checked with a schema fingerprint). This allows a compact format without field names or type tags, decoded without reflection.

Engines

The package provides two serialization engines:

  • The graph engine (enabled by default, or via EnableGraphEngine(true) / --state-engine=graph), documented below, which encodes and decodes object graphs without runtime reflection.
  • The legacy wire engine (EnableGraphEngine(false) / --state-engine=wire), which encodes tagged pkg/state/wire objects via reflect.

Load and pkg/state/pretty inspect the stream header and automatically decode either format.

Types

Each Go type is described by a typeInfo, built once on first use and cached. Values are saved and loaded by switching on their kind and using unsafe pointers. Generated StateSave and StateLoad methods call typed Sink and Source methods (SaveInt64, LoadInt64, etc.) for predeclared scalar, string, and byte-slice fields, LoadValueTo and AfterLoadTo to avoid closure allocations, and Save/Load (passing field pointers as any, whose interface type word identifies the cached typeInfo) for other fields. Pure scalar structs and identtype wrappers without hooks are registered via RegisterNoHooks / RegisterIdent and copied directly as raw bytes.

StateSave must save fields in the same order as StateLoad loads them.

Encoding

Encoding begins with graphEncodeState.save.

The graph is divided into objects: regions of memory saved as a unit, which pointers may point into. An object is either a single value (the target of a pointer), an array (the backing array of slices), or a map. Pointers are saved as indices of targets: a location (object and byte offset) within an object.

Objects are encoded as they are found, starting from the root, in a breadth first order. When a pointer is saved, its target is looked up by address. If it is not known, the object index (which maps blocks of memory to the objects intersecting them) is searched for an object containing it. If there is none, a new object is created, which absorbs any objects that it contains: their targets are relocated into the new object, and their encodings (if done) are reused when the new object is encoded, so that StateSave is called once per value.

Of arrays, only the elements within slices, or pointed to, are saved.

The stream is:

  1. A magic number and the schema fingerprint.
  2. The type table: the types of objects, and of values held in interfaces.
  3. The object table: the kind and type of each object.
  4. The target table: the object and offset of each target.
  5. The data of each object, in order.

Decoding

Decoding begins with graphDecodeState.load. It allocates all objects (except maps, which are allocated when they are first referred to, since their exact type is only known then), resolves all targets to pointers, and then decodes the data of each object in a single pass. Then, the load callbacks of objects are run once the objects they wait for are complete. Cycles of waiting objects are detected and reported as errors.

Documentation

Overview

Package state provides functionality related to saving and loading object graphs. For most types, it provides a set of default saving / loading logic that will be invoked automatically if custom logic is not defined.

Kind             Support
----             -------
Bool             default
Int              default
Int8             default
Int16            default
Int32            default
Int64            default
Uint             default
Uint8            default
Uint16           default
Uint32           default
Uint64           default
Float32          default
Float64          default
Complex64        default
Complex128       default
Array            default
Chan             custom
Func             custom
Interface        default
Map              default
Ptr              default
Slice            default
String           default
Struct           custom (*) Unless zero-sized.
UnsafePointer    custom

See README.md for an overview of how encoding and decoding works.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CollectStats

func CollectStats(enable bool)

CollectStats enables or disables the collection of per-type timing statistics by the graph engine's Save and Load, which is costly.

func EnableGraphEngine

func EnableGraphEngine(enable bool)

EnableGraphEngine enables or disables the reflection-free graph engine for Save. Load automatically detects which engine produced a stream from its header.

EnableGraphEngine mutates process-global state and must not be called concurrently with Save.

func Failf

func Failf(fmtStr string, v ...any)

Failf is a wrapper around panic that should be used to generate errors that can be caught during saving and loading.

func GraphEngineEnabled

func GraphEngineEnabled() bool

GraphEngineEnabled reports whether Save uses the reflection-free graph engine.

func IsZeroValue

func IsZeroValue(val any) bool

IsZeroValue checks if the given value is the zero value.

This function is used by the stateify tool.

func PrintGraph

func PrintGraph(w io.Writer, data []byte, graph uint64, asHTML bool) error

PrintGraph prints a single graph-engine object graph from data, as text or HTML.

Since streams don't describe the types of fields, they can only be printed by the binary that saved them. Objects are decoded without running their load callbacks.

func ReadHeader

func ReadHeader(r *wire.Reader) (length uint64, object bool, err error)

ReadHeader reads an object header.

Each object written to the statefile is prefixed with a header. See WriteHeader for more information; these functions are exported to allow non-state writes to the file to play nice with debugging tools.

func Register

func Register(t Type)

Register registers a type.

This must be called on init and only done once. Registration lets Load reconstruct concrete types held in interfaces without a preceding Save.

func RegisterIdent

func RegisterIdent(t, base StructType)

RegisterIdent registers a struct type defined over base (+stateify identtype), inheriting base's RegisterNoHooks property regardless of init order.

This must be called on init and only done once.

func RegisterNoHooks

func RegisterNoHooks(t StructType)

RegisterNoHooks registers a struct type whose StateSave and StateLoad methods save and load its fields, and do nothing else, like RegisterStruct. The state package may save values of the type without calling these methods.

This must be called on init and only done once.

func RegisterStruct

func RegisterStruct(t StructType)

RegisterStruct registers a struct type, like Register. Passing the type as a StructType makes the interface table of StructType for the type static, so that the state package does not build it at run time.

This must be called on init and only done once.

func Release

func Release()

Release releases references to global type databases. Must only be called in contexts where they will definitely never be used, in order to save memory.

func WriteHeader

func WriteHeader(w *wire.Writer, length uint64, object bool) error

WriteHeader writes a header.

Each object written to the statefile should be prefixed with a header. In order to generate statefiles that play nicely with debugging tools, raw writes should be prefixed with a header with object set to false and the appropriate length. This will allow tools to skip these regions.

Types

type AfterLoader

type AfterLoader interface {
	// StateAfterLoad calls the afterLoad method.
	StateAfterLoad(ctx context.Context)
}

AfterLoader is implemented by the generated code of structs with afterLoad methods (see Source.AfterLoadTo).

type ErrState

type ErrState struct {
	// contains filtered or unexported fields
}

ErrState is returned when an error is encountered during encode/decode.

func (*ErrState) Error

func (e *ErrState) Error() string

Error returns a sensible description of the state error.

func (*ErrState) Unwrap

func (e *ErrState) Unwrap() error

Unwrap implements standard unwrapping.

type Header struct {
	// Length is the payload length: for legacy object graphs, the number of
	// wire objects; for graph-engine object graphs and non-object regions,
	// the length in bytes.
	Length uint64

	// Object reports whether this region is an encoded object graph rather
	// than raw non-state data.
	Object bool

	// Graph reports whether an object graph uses the graph engine format.
	Graph bool
}

Header describes a framed region in a statefile stream.

func ReadHeaderMode

func ReadHeaderMode(r *wire.Reader) (Header, error)

ReadHeaderMode reads an object header and returns its metadata.

type SaverLoader

type SaverLoader interface {
	// StateSave saves the state of the object to the given Map.
	StateSave(Sink)

	// StateLoad loads the state of the object.
	StateLoad(context.Context, Source)
}

SaverLoader defines a struct's state methods.

type Sink

type Sink struct {
	// contains filtered or unexported fields
}

Sink is used for Type.StateSave.

func (Sink) Context

func (s Sink) Context() context.Context

Context returns the context object provided at save time.

func (Sink) Save

func (s Sink) Save(slot int, objPtr any)

Save adds the given object to the map.

You should pass always pointers to the object you are saving. For example:

type X struct {
	A int
	B *int
}

func (x *X) StateTypeInfo(m Sink) state.TypeInfo {
	return state.TypeInfo{
		Name:   "pkg.X",
		Fields: []string{
			"A",
			"B",
		},
	}
}

func (x *X) StateSave(m Sink) {
	m.Save(0, &x.A) // Field is A.
	m.Save(1, &x.B) // Field is B.
}

func (x *X) StateLoad(m Source) {
	m.Load(0, &x.A) // Field is A.
	m.Load(1, &x.B) // Field is B.
}

func (Sink) SaveBool

func (s Sink) SaveBool(slot int, v *bool)

SaveBool saves a bool field. See Save.

func (Sink) SaveFloat32

func (s Sink) SaveFloat32(slot int, v *float32)

SaveFloat32 saves a float32 field. See Save.

func (Sink) SaveFloat64

func (s Sink) SaveFloat64(slot int, v *float64)

SaveFloat64 saves a float64 field. See Save.

func (Sink) SaveInt

func (s Sink) SaveInt(slot int, v *int)

SaveInt saves a int field. See Save.

func (Sink) SaveInt8

func (s Sink) SaveInt8(slot int, v *int8)

SaveInt8 saves a int8 field. See Save.

func (Sink) SaveInt16

func (s Sink) SaveInt16(slot int, v *int16)

SaveInt16 saves a int16 field. See Save.

func (Sink) SaveInt32

func (s Sink) SaveInt32(slot int, v *int32)

SaveInt32 saves a int32 field. See Save.

func (Sink) SaveInt64

func (s Sink) SaveInt64(slot int, v *int64)

SaveInt64 saves a int64 field. See Save.

func (Sink) SaveString

func (s Sink) SaveString(slot int, v *string)

SaveString saves a string field. See Save.

func (Sink) SaveUint

func (s Sink) SaveUint(slot int, v *uint)

SaveUint saves a uint field. See Save.

func (Sink) SaveUint8

func (s Sink) SaveUint8(slot int, v *uint8)

SaveUint8 saves a uint8 field. See Save.

func (Sink) SaveUint16

func (s Sink) SaveUint16(slot int, v *uint16)

SaveUint16 saves a uint16 field. See Save.

func (Sink) SaveUint32

func (s Sink) SaveUint32(slot int, v *uint32)

SaveUint32 saves a uint32 field. See Save.

func (Sink) SaveUint64

func (s Sink) SaveUint64(slot int, v *uint64)

SaveUint64 saves a uint64 field. See Save.

func (Sink) SaveUintptr

func (s Sink) SaveUintptr(slot int, v *uintptr)

SaveUintptr saves a uintptr field. See Save.

func (Sink) SaveValue

func (s Sink) SaveValue(slot int, obj any)

SaveValue adds the given object value to the map.

This should be used for values where pointers are not available, or casts are required during Save/Load.

For example, if we want to cast external package type P.Foo to int64:

func (x *X) StateSave(m Sink) {
	m.SaveValue(0, "A", int64(x.A))
}

func (x *X) StateLoad(m Source) {
	m.LoadValue(0, new(int64), func(x any) {
		x.A = P.Foo(x.(int64))
	})
}

type Source

type Source struct {
	// contains filtered or unexported fields
}

Source is used for Type.StateLoad.

func (Source) AfterLoad

func (s Source) AfterLoad(fn func())

AfterLoad schedules a function execution when all objects have been allocated and their automated loading and customized load logic have been executed. fn will not be executed until all of current object's dependencies' AfterLoad() logic, if exist, have been executed.

func (Source) AfterLoadTo

func (s Source) AfterLoadTo(l AfterLoader)

AfterLoadTo is like AfterLoad, but calls l.StateAfterLoad.

This is used by go_stateify, and avoids the allocation of a function.

func (Source) Context

func (s Source) Context() context.Context

Context returns the context object provided at load time.

func (Source) Load

func (s Source) Load(slot int, objPtr any)

Load loads the given object passed as a pointer..

See Sink.Save for an example.

func (Source) LoadBool

func (s Source) LoadBool(slot int, v *bool)

LoadBool loads a bool field. See Load.

func (Source) LoadFloat32

func (s Source) LoadFloat32(slot int, v *float32)

LoadFloat32 loads a float32 field. See Load.

func (Source) LoadFloat64

func (s Source) LoadFloat64(slot int, v *float64)

LoadFloat64 loads a float64 field. See Load.

func (Source) LoadInt

func (s Source) LoadInt(slot int, v *int)

LoadInt loads a int field. See Load.

func (Source) LoadInt8

func (s Source) LoadInt8(slot int, v *int8)

LoadInt8 loads a int8 field. See Load.

func (Source) LoadInt16

func (s Source) LoadInt16(slot int, v *int16)

LoadInt16 loads a int16 field. See Load.

func (Source) LoadInt32

func (s Source) LoadInt32(slot int, v *int32)

LoadInt32 loads a int32 field. See Load.

func (Source) LoadInt64

func (s Source) LoadInt64(slot int, v *int64)

LoadInt64 loads a int64 field. See Load.

func (Source) LoadString

func (s Source) LoadString(slot int, v *string)

LoadString loads a string field. See Load.

func (Source) LoadUint

func (s Source) LoadUint(slot int, v *uint)

LoadUint loads a uint field. See Load.

func (Source) LoadUint8

func (s Source) LoadUint8(slot int, v *uint8)

LoadUint8 loads a uint8 field. See Load.

func (Source) LoadUint16

func (s Source) LoadUint16(slot int, v *uint16)

LoadUint16 loads a uint16 field. See Load.

func (Source) LoadUint32

func (s Source) LoadUint32(slot int, v *uint32)

LoadUint32 loads a uint32 field. See Load.

func (Source) LoadUint64

func (s Source) LoadUint64(slot int, v *uint64)

LoadUint64 loads a uint64 field. See Load.

func (Source) LoadUintptr

func (s Source) LoadUintptr(slot int, v *uintptr)

LoadUintptr loads a uintptr field. See Load.

func (Source) LoadValue

func (s Source) LoadValue(slot int, objPtr any, fn func(any))

LoadValue loads the given object value from the map.

See Sink.SaveValue for an example.

func (Source) LoadValueTo

func (s Source) LoadValueTo(slot int, typ any, l ValueLoader)

LoadValueTo is like LoadValue, but calls l.StateLoadValue(ctx, slot, value) instead of a function. typ is a nil pointer to the type of the value. StateLoadValue must not retain the address of the value, which it must copy.

This is used by go_stateify, and avoids the allocations of LoadValue.

func (Source) LoadWait

func (s Source) LoadWait(slot int, objPtr any)

LoadWait loads the given objects from the map, and marks it as requiring all AfterLoad executions to complete prior to running this object's AfterLoad.

See Sink.Save for an example.

type Stats

type Stats struct {
	// contains filtered or unexported fields
}

Stats tracks encode / decode timing.

This currently provides a meaningful String function and no other way to extract stats about individual types.

All exported receivers accept nil.

func Load

func Load(ctx context.Context, r io.Reader, rootPtr any) (Stats, error)

Load loads a checkpoint.

func Save

func Save(ctx context.Context, w io.Writer, rootPtr any) (Stats, error)

Save saves the given object state.

func (*Stats) String

func (s *Stats) String() string

String returns a table representation of the stats.

type StructType

type StructType interface {
	Type
	SaverLoader
}

StructType is implemented by pointers to struct types with state methods.

type Type

type Type interface {
	// StateTypeName returns the type's name.
	//
	// This is used for matching type information during encoding and
	// decoding, as well as dynamic interface dispatch. This should be
	// globally unique.
	StateTypeName() string

	// StateFields returns information about the type.
	//
	// Fields is the set of fields for the object. Calls to Sink.Save and
	// Source.Load must be made in-order with respect to these fields.
	//
	// The field list must be stable for the type.
	StateFields() []string
}

Type describes a type's saved representation while minimizing runtime reflection. The package separately registers binary codecs for supported foreign structs.

All these methods can be automatically generated by the go_stateify tool.

type ValueLoader

type ValueLoader interface {
	// StateLoadValue loads the value of the field in the given slot.
	StateLoadValue(ctx context.Context, slot int, value any)
}

ValueLoader is implemented by the generated code of structs with fields saved through values (see Source.LoadValueTo).

Directories

Path Synopsis
Package pretty is a pretty-printer for state streams.
Package pretty is a pretty-printer for state streams.
Package statefile defines the state file data stream.
Package statefile defines the state file data stream.
Package tests tests the state packages.
Package tests tests the state packages.
Package wire contains a few basic types that can be composed to serialize graph information for the state package.
Package wire contains a few basic types that can be composed to serialize graph information for the state package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL