zap_codec

package
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 6, 2026 License: BSD-3-Clause Imports: 4 Imported by: 0

README

proto/zap_codec — ZAP-native wire codec for proto/{p,x,block,warp}

This package is the single canonical construction site for the wire codecs consumed by proto/p, proto/x, proto/p/warp, proto/p/warp/payload, proto/p/warp/message, proto/p/block, and proto/x/block.

It exists so SDK consumers (sdk/wallet/chain/{p,x}) can wire up the parser dependencies of the proto packages without importing github.com/luxfi/codec at the wallet layer. The codec backend choice is bound EXACTLY ONCE — here.

Wire-format verdict — BREAK from legacy linearcodec

This codec emits little-endian wire bytes via github.com/luxfi/codec/zapcodec. The previous linearcodec-backed construction emitted big-endian wire bytes.

For every multi-byte field — uint16/uint32/uint64, length prefixes, interface type-id discriminators, codec version bytes, string length prefixes — the byte order is inverted vs the pre-Wave-2G wire layout.

This is a forward-only wire-format change aligned with LP-023 ZAP-native activation (proto/zap_native/codec_select.go: ZAPActivationUnix=0). There is no dual-mode emission, no fallback, no rollback path. Network validators expecting LE wire bytes will accept this codec's output; validators still on BE linearcodec will not.

Canonical hex dump comparison

Test struct:

type dummyTx struct {
    NetworkID uint32 `serialize:"true"`  // 0x01020304
    Nonce     uint64 `serialize:"true"`  // 0x0102030405060708
    Memo      []byte `serialize:"true"`  // "hi"
}

Marshalled at codec version 0:

ZAP-native (this codec):
  00 00                                          ← codec version 0 (LE uint16)
  04 03 02 01                                    ← NetworkID 0x01020304 (LE uint32)
  08 07 06 05 04 03 02 01                        ← Nonce 0x0102030405060708 (LE uint64)
  02 00 00 00                                    ← Memo length 2 (LE uint32)
  68 69                                          ← Memo "hi"

linearcodec (legacy, gone after Wave 2G):
  00 00                                          ← codec version 0 (BE uint16, same byte pattern at v=0)
  01 02 03 04                                    ← NetworkID 0x01020304 (BE uint32)
  01 02 03 04 05 06 07 08                        ← Nonce 0x0102030405060708 (BE uint64)
  00 00 00 02                                    ← Memo length 2 (BE uint32)
  68 69                                          ← Memo "hi" (byte body identical)

Both encodings carry the same fields in the same order — the delta is byte order alone.

The above ZAP-native dump is captured live by TestManager_WireFormat_ZAPNative_HexDump. To reproduce:

$ go test -v ./zap_codec/... -run TestManager_WireFormat_ZAPNative_HexDump
ZAP-native canonical dummyTx hex dump:
  0000040302010807060504030201020000006869

The legacy linearcodec bytes shown above are anchored against the linearcodec spec (luxfi/codec/linearcodec); they are historical and no longer produced by any in-tree code path.

Architecture

Following Rich Hickey decomplection:

  • Value, not place — the wire codec choice is a value qualified by namespace (zap_codec.NewVersionedManager). It is bound in ONE place, not braided into every consumer's package init.
  • Composition over inheritanceManager wraps zapcodec.Codec with a thin version-prefix outer layer. The outer carries the codec version (uint16 LE, matching the inner byte order). The two layers are independently complete primitives.
  • Orthogonal — this package neither imports proto/{p,x} nor knows about block/tx/warp types. Per-chain type registration happens at the caller site (sdk/wallet/chain/{p/pcodecs.go,x/constants.go, x/builder/constants.go}) which RegisterTypes onto the registries returned here.
  • One canonical way — there is exactly ONE construction expression per chain primitive (NewPVMRuntime, NewPVMGenesis, NewXVMParser, NewWarp, NewPayload, NewMessage). Wallet, builder, signer, genesis builder all reach for the same constructor.

Surface

Constructor Returns Budget
NewVersionedManager(v, maxSize) *Manager caller-supplied
NewPVMRuntime(v) *Manager MaxSize (1 MiB)
NewPVMGenesis(v) *Manager math.MaxInt32
NewXVMParser(v) (*Manager runtime, *Manager genesis) MaxSize / math.MaxInt32
NewWarp(v) *Manager math.MaxInt
NewPayload(v, payloadMax) *Manager caller-supplied
NewMessage(v) *Manager math.MaxInt

Each *Manager satisfies the local Codec interface from every proto/{p,x,block,warp,payload,message} package by shape (Marshal(version, source), Unmarshal(bytes, dest), Size(version, value)) and also acts as a Registry (RegisterType, SkipRegistrations).

Documentation

Overview

Package zap_codec is the single canonical construction site for the ZAP-native wire codecs consumed by proto/p, proto/x, proto/p/warp, proto/p/warp/payload, proto/p/warp/message, proto/p/block, and proto/x/block. It exists so the SDK wallet (sdk/wallet/chain/{p,x}) can construct its codec dependencies without importing github.com/luxfi/codec at the wallet layer — the codec choice is bound EXACTLY ONCE in this package.

Wire-format relationship to linearcodec:

  • This package's codec backend is luxfi/codec/zapcodec, which is the little-endian sibling of linearcodec. Reflection layout (struct-tag order, slice/map length prefixes, interface type-id mechanism) is identical to linearcodec; the only delta is byte order:

  • linearcodec writes BIG-endian for uint16/uint32/uint64 fields, length prefixes, interface type-ids, codec version bytes, string length prefixes (uint16 BE).

  • zapcodec writes LITTLE-endian for the same set of fields. x86_64 and arm64 are LE-native, so LE writes map to single MOV instructions where BE writes need BSWAP.

    This is a WIRE-FORMAT BREAK from the legacy linearcodec wire bytes that previously fed proto/{p,x} consumers. The break is intentional and aligned with LP-023 ZAP-native activation (proto/zap_native/codec_select.go: ZAPActivationUnix=0 means "ZAP is mandatory from genesis"). Forward-only — there is no dual-mode emission and no rollback path. Network validators expecting LE wire bytes will accept this codec's output; validators still on BE linearcodec will not. See README.md in this package for a hex dump of a canonical signed BaseTx in both modes.

Architecture (Hickey decomplection):

  • VALUE: the wire codec choice. Today: zapcodec LE. (Was: linearcodec BE.) The value lives here, qualified by namespace, not by braided prefix in every consumer.

  • COMPOSITION: NewVersionedManager wraps a zapcodec.Codec with a version-prefix outer layer. The outer layer carries the codec version (uint16 LE, matching the inner codec's byte order) and dispatches to the registered inner codec on Unmarshal. Outer + inner are independently complete primitives.

  • ORTHOGONAL: this package neither imports proto/{p,x} nor knows about block/tx/warp types. Per-chain type registration happens at the caller site (sdk/wallet/chain/{p/pcodecs.go,x/constants.go, x/builder/constants.go}) which RegisterTypes onto the registries returned by NewPVMRuntime / NewPVMGenesis / NewXVMParser / NewWarp / NewPayload / NewMessage.

Index

Constants

View Source
const (
	ByteLen  = wrappers.ByteLen
	ShortLen = wrappers.ShortLen
	IntLen   = wrappers.IntLen
	LongLen  = wrappers.LongLen
	BoolLen  = wrappers.BoolLen
)

Byte-length constants for the wire-size accounting code in node/vms. Identical values to wrappers.{Byte,Short,Int,Long,Bool}Len — re- exported here so consumers don't reach across into luxfi/codec.

View Source
const DefaultMaxSize = 1024 * 1024

DefaultMaxSize is the default wire-payload budget when the caller doesn't size their Manager explicitly. Matches the legacy codec.DefaultMaxSize (1 MiB).

View Source
const MaxSize = 1024 * 1024

MaxSize is the default maximum wire-payload size for runtime txs. Mirrors codec.DefaultMaxSize (1 MiB) so per-tx envelope is unchanged vs linearcodec-backed configurations.

View Source
const VersionSize = 2

VersionSize is the on-wire length of the codec-version prefix the outer manager prepends. Two bytes, uint16 little-endian.

Variables

View Source
var (
	ErrCantPackVersion           = errors.New("zap_codec: couldn't pack codec version")
	ErrCantUnpackVersion         = errors.New("zap_codec: couldn't unpack codec version")
	ErrUnknownVersion            = errors.New("zap_codec: unknown codec version")
	ErrExtraSpace                = errors.New("zap_codec: trailing buffer space")
	ErrMaxSizeExceeded           = errors.New("zap_codec: wire payload exceeds max size")
	ErrMaxSliceLenExceeded       = errors.New("zap_codec: max slice length exceeded")
	ErrMarshalNil                = errors.New("zap_codec: can't marshal nil pointer")
	ErrUnmarshalNil              = errors.New("zap_codec: can't unmarshal into nil")
	ErrDoesNotImplementInterface = errors.New("zap_codec: does not implement interface")
)

Sentinel errors. Mirror the shape of luxfi/codec errors so callers that previously asserted on codec.ErrUnknownVersion / .ErrCantUnpackVersion continue to work via errors.Is once they switch to errors.Is on these sentinels.

View Source
var ErrInsufficientLength = wrappers.ErrInsufficientLength

ErrInsufficientLength is re-exported from wrappers so callers can assert on the codec's underflow sentinel without importing luxfi/codec/wrappers themselves.

Functions

This section is empty.

Types

type Codec

type Codec interface {
	MarshalInto(interface{}, *wrappers.Packer) error
	UnmarshalFrom(*wrappers.Packer, interface{}) error
	Size(value interface{}) (int, error)
}

Codec is the local wire-codec interface — Marshal/Unmarshal/Size on a *wrappers.Packer. Structurally identical to the legacy github.com/luxfi/codec.Codec interface; every zapcodec.Codec instance satisfies it by embedding the same three methods. Consumers of proto/zap_codec hold this local type rather than importing the upstream codec package.

type Errs

type Errs = wrappers.Errs

Errs re-exports wrappers.Errs — a multi-error accumulator used by per-VM codec.go files that fan in many RegisterCodec / RegisterType calls under a single error tap.

type GeneralCodec

type GeneralCodec interface {
	Codec
	Registry
}

GeneralCodec is the Codec + Registry union — every concrete codec instance the platform uses (currently zapcodec) satisfies it.

type LinearCodec

type LinearCodec = zapcodec.Codec

LinearCodec is the GeneralCodec + SkipRegistrations surface every VM consumer holds. The name "Linear" refers to LINEAR TYPE-ID ASSIGNMENT (sequential ids assigned in registration order), NOT to the historical linearcodec big-endian wire format. The underlying implementation is luxfi/codec/zapcodec — little-endian on the wire.

func NewLinearCodec

func NewLinearCodec() LinearCodec

NewLinearCodec returns a fresh zapcodec-backed Codec instance with the default ("serialize") struct tag. The name "Linear" refers to LINEAR TYPE-ID ASSIGNMENT (sequential ids), NOT to the historical linearcodec big-endian wire layout — this codec emits LE wire bytes.

func NewLinearCodecWithTags

func NewLinearCodecWithTags(tags ...string) LinearCodec

NewLinearCodecWithTags returns a fresh zapcodec-backed Codec instance that honours the supplied struct-tag names. Used by the metadata codec wiring where v0:"true" / v1:"true" tags select per-version field sets.

type LinearRegistry

type LinearRegistry interface {
	Registry
	SkipRegistrations(int)
}

LinearRegistry extends Registry with the slot-skipping operation used by historical PVM/XVM type layouts. zapcodec.Codec satisfies this (its SkipRegistrations bumps the next-type-id counter).

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager is the version-prefix wire codec implementation. It owns the codec.Manager-shaped surface (Marshal/Unmarshal/Size) that proto/{p,x}'s local Codec interfaces structurally match, but contains zero reference to github.com/luxfi/codec.Manager — Marshal/Unmarshal/Size are implemented directly on top of a zapcodec.Codec.

Manager also satisfies zapcodec.Codec's Registry surface by delegating RegisterType / SkipRegistrations to the inner codec. This is the same contract every Wave 2-era consumer expects: hand back ONE value that is BOTH the wire codec AND the type registry.

Thread-safety: the inner zapcodec.Codec is concurrency-safe (its RWMutex protects the type registry). Manager itself is stateless past construction — no per-call mutations on the *Manager pointer — so callers may share a *Manager across goroutines without external sync.

func NewMessage

func NewMessage(version uint16) *Manager

NewMessage returns a Manager wired for proto/p/warp/message types. Budget is math.MaxInt — warp messages embed AddressedCall payloads whose size is bounded by the payload codec, not this one.

Caller is responsible for message.RegisterTypes(m) before any Marshal/Unmarshal.

func NewPVMGenesis

func NewPVMGenesis(version uint16) *Manager

NewPVMGenesis returns a Manager wired for proto/p PVM genesis blobs. Same wire layout as NewPVMRuntime but with math.MaxInt32 size budget (genesis can be very large — full validator set + every chain's initial state).

func NewPVMRuntime

func NewPVMRuntime(version uint16) *Manager

NewPVMRuntime returns a Manager wired for proto/p PVM runtime transactions. Caller is responsible for registering block/tx types via proto/p/block.RegisterTypes(m) before any Marshal/Unmarshal.

Budget: MaxSize (1 MiB). For genesis blobs use NewPVMGenesis.

func NewPayload

func NewPayload(version uint16, payloadMaxSize uint64) *Manager

NewPayload returns a Manager wired for proto/p/warp/payload types (Hash, AddressedCall). Budget is payloadMaxSize.

Caller is responsible for payload.RegisterTypes(m) before any Marshal/Unmarshal.

func NewVersionedManager

func NewVersionedManager(version uint16, maxSize uint64) *Manager

NewVersionedManager constructs a Manager that emits/accepts wire bytes prefixed by a uint16 little-endian codec version. The inner zapcodec instance is allocated fresh; SkipRegistrations and RegisterType go directly to it.

`version` is the codec version the manager will write on Marshal and require on Unmarshal. `maxSize` is the post-version-prefix wire budget (Marshal refuses to emit, Unmarshal refuses to read, anything past this size).

func NewWarp

func NewWarp(version uint16) *Manager

NewWarp returns a Manager wired for proto/p/warp signature + teleport payload types. Budget is math.MaxInt because warp signature aggregates may grow with validator set size.

Caller is responsible for warp.RegisterTypes(m) before any Marshal/Unmarshal.

func NewXVMParser

func NewXVMParser(version uint16) (runtime, genesis *Manager)

NewXVMParser returns the four codec/registry values proto/x's txs.NewParser / block.NewParser require: runtime Codec, genesis Codec, runtime Registry, genesis Registry.

Each returned Manager is independent (Marshal/Unmarshal on the runtime manager does NOT touch the genesis manager's type-id table). Caller is responsible for routing fx-owned and block-level type registrations onto BOTH registries — see proto/x/txs/parser.go and proto/x/block/parser.go for the canonical wiring.

runtimeMaxSize / genesisMaxSize budget the two managers separately. Pass MaxSize for the runtime manager and math.MaxInt32 for the genesis manager to match the historical linearcodec wiring.

func (*Manager) Marshal

func (m *Manager) Marshal(version uint16, source interface{}) ([]byte, error)

Marshal serializes source into a fresh byte slice with the manager's codec version (uint16 LE) prepended. The caller-supplied version MUST match the manager's bound version — any other value returns ErrUnknownVersion.

Satisfies the proto/{p,x,block,warp}.Codec interface.

func (*Manager) RegisterType

func (m *Manager) RegisterType(val interface{}) error

RegisterType registers val with the inner reflection codec so it can be unmarshalled into an interface field. Satisfies the proto/{p,x,block,warp}.Registry interface.

func (*Manager) Size

func (m *Manager) Size(version uint16, value interface{}) (int, error)

Size returns the on-wire size of value INCLUDING the manager's version-prefix bytes.

Satisfies the proto/{p,x,block,warp}.Codec interface.

func (*Manager) SkipRegistrations

func (m *Manager) SkipRegistrations(n int)

SkipRegistrations bumps the inner codec's next-type-id by n. Lets callers preserve legacy type-id slot layouts even across the linear→ zap codec backend swap.

func (*Manager) Unmarshal

func (m *Manager) Unmarshal(bytes []byte, dest interface{}) (uint16, error)

Unmarshal deserializes bytes into dest. Returns the codec version read from the wire prefix. Rejects buffers smaller than VersionSize, buffers larger than maxSize, version mismatches, and trailing bytes.

Satisfies the proto/{p,x,block,warp}.Codec interface.

type MultiManager

type MultiManager interface {
	RegisterCodec(version uint16, codec Codec) error
	Marshal(version uint16, source interface{}) ([]byte, error)
	Unmarshal(bytes []byte, dest interface{}) (uint16, error)
	Size(version uint16, value interface{}) (int, error)
}

MultiManager is a multi-codec-version dispatcher. It owns the version-prefix outer layer and routes Marshal/Unmarshal to the registered inner Codec for the supplied version. Wire layout:

[uint16 LE codec version] [inner codec body]

The version-prefix byte order is LE to match the inner zapcodec body byte order. This deliberately diverges from the legacy luxfi/codec.Manager (which prepends BE) — see proto/zap_codec/README.md for the wire-format break note.

Thread-safety: RegisterCodec is NOT goroutine-safe — call it once at init time before any Marshal/Unmarshal. After that, Marshal / Unmarshal / Size are concurrency-safe to the extent the inner Codec implementations are (zapcodec.Codec is, via its RWMutex-protected type registry).

func NewDefaultManager

func NewDefaultManager() MultiManager

NewDefaultManager returns a multi-version Manager with the default 1 MiB wire-payload budget. Equivalent to NewManager(DefaultMaxSize).

func NewManager

func NewManager(maxSize uint64) MultiManager

NewManager returns a multi-version Manager with the supplied wire- payload budget. Versions are registered after construction via MultiManager.RegisterCodec.

func NewMaxInt32Manager

func NewMaxInt32Manager() MultiManager

NewMaxInt32Manager returns a multi-version Manager sized for genesis-style blobs (math.MaxInt32 budget). VM genesis codec wiring reaches for this rather than re-deriving the budget at every call site.

func NewMaxIntManager

func NewMaxIntManager() MultiManager

NewMaxIntManager returns a multi-version Manager sized for warp / proposervm-block style payloads (math.MaxInt budget — effectively unbounded; the p2p layer caps real-world wire sizes well below this).

type Packer

type Packer = wrappers.Packer

Packer re-exports wrappers.Packer for callers that need to drive MarshalInto / UnmarshalFrom byte streams directly (fuzz tests, size-accounting code).

type Registry

type Registry interface {
	RegisterType(interface{}) error
}

Registry is the local type-registration interface. Structurally identical to github.com/luxfi/codec.Registry. Every zapcodec.Codec instance satisfies it via the embedded zapcodec.RegisterType.

type ZAPCodec

type ZAPCodec = zapcodec.Codec

ZAPCodec is an explicit alias for the same zapcodec.Codec surface callers reach for when they want to emphasise the ZAP-native wire layout (e.g. platformvm txs V2). Same backing type as LinearCodec; the alias exists for readability at call sites.

func NewZAPCodec

func NewZAPCodec() ZAPCodec

NewZAPCodec is an alias for NewLinearCodec — both return a zapcodec-backed Codec. Call sites that want to emphasise the ZAP wire layout use this name; otherwise they're interchangeable.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL