fsmwire

package
v0.0.0-...-d37b0a0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 3, 2026 License: AGPL-3.0 Imports: 3 Imported by: 0

Documentation

Overview

Package fsmwire defines the §6.3 / §11 binary wire format for the FSM-internal encryption Raft entry types (opcodes 0x03 / 0x04 / 0x05). The encoding is hand-rolled binary with a leading version byte rather than proto3 because:

  • Each replica's apply path must reproduce the bytes exactly to keep state machine determinism. proto3's lenient unknown-field handling would let a future leader-built field silently drop on a stale follower, leaving registry rows or active-pointer entries missing on that follower — exactly the silent divergence the encryption tag was added to detect.

  • The schema is small (~30 lines of encode/decode per opcode), so the maintenance cost of hand-rolled binary is negligible while keeping every state-machine-affecting bit explicit.

  • It matches the in-house style: internal/encryption/envelope.go, raft_envelope.go, and kv/raft_payload_wrapper.go all hand-roll binary with a leading version byte.

Version byte. Every payload starts with WireVersionV1 (0x01). Future schema changes bump the version and the decoder fails closed (ErrFSMWireVersion) — operators upgrade clusters fully before flipping flags that produce the new format. There is no "skip unknown fields" path because skipping a field on the apply path is the divergence we are guarding against.

Index

Constants

View Source
const (
	OpRegistration byte = 0x03
	OpBootstrap    byte = 0x04
	OpRotation     byte = 0x05
)

Opcode tags consumed by kv/fsm.go's Apply dispatch. Reserved per design §11.3:

0x00 raftEncodeSingle    (kv legacy)
0x01 raftEncodeBatch     (kv legacy)
0x02 raftEncodeHLCLease  (kv HLC ceiling)
0x03 OpRegistration      (writer-registry register, §4.1)
0x04 OpBootstrap         (initial DEK install, §5.6)
0x05 OpRotation          (DEK rotation / cluster flag flip, §5.2 + §7.1)
View Source
const (
	// RotateSubRotateDEK is a fresh DEK install + active-pointer
	// update. Ships in Stage 4. See §5.2.
	RotateSubRotateDEK byte = 0x01

	// RotateSubEnableStorageEnvelope is the one-shot storage-layer
	// cutover (§2.2 / §7.1 Phase 1). Ships in Stage 6D. The wire
	// shape reuses the rotate-dek triple but with `Wrapped` empty
	// (`len(Wrapped) == 0`) and `DEKID == sidecar.Active.Storage`;
	// the applier re-validates both at apply time. The byte value
	// `0x04` matches the original `0x02..0x0F` reservation block
	// so older binaries that pre-date Stage 6D continue to halt on
	// the existing "unknown sub-tag" branch.
	RotateSubEnableStorageEnvelope byte = 0x04

	// RotateSubEnableRaftEnvelope is the one-shot raft-layer cutover
	// (§7.1 Phase 2). Ships in Stage 6E-1. The wire shape mirrors
	// RotateSubEnableStorageEnvelope but with the raft slot:
	// `Wrapped` empty (`len(Wrapped) == 0`), `DEKID ==
	// sidecar.Active.Raft`, `Purpose == PurposeRaft`. The applier
	// re-validates all three at apply time. On a fresh-success
	// apply the sidecar's `RaftEnvelopeCutoverIndex` is set to
	// `raftIdx` (the cutover entry's own apply index, which Stage
	// 6E-2's engine apply-hook compares against `entry.Index` via
	// strict `>` to decide unwrap-or-not).
	RotateSubEnableRaftEnvelope byte = 0x05
)

Sub-tags used by OpRotation. Encoded as the first byte after the version byte.

View Source
const (
	OpEncryptionMin byte = 0x03
	OpEncryptionMax byte = 0x07
)

OpEncryptionMin / OpEncryptionMax delimit the FSM-internal opcode range RESERVED for the encryption subsystem. kv/fsm.go's Apply dispatcher routes EVERY byte in the closed range [OpEncryptionMin, OpEncryptionMax] through applyEncryption, which fails closed via ErrEncryptionApply for any byte that is not yet implemented (Stage 4 implements only 0x03/0x04/0x05; 0x06/0x07 are reserved for later stages). The widened range is the codex P1 fix for PR748: previously only the three implemented opcodes were routed, so a future leader emitting 0x06 against a stale follower would fall through to the legacy proto3 decoder rather than halting the apply loop — the same divergence shape the §6.3 fail-closed halt was added to prevent.

Upper bound 0x07 (NOT 0x0F): proto3 wire tags for field 1 occupy 0x08..0x0D (field 1 with wire types varint/fixed64/length-delim/ start-group/end-group/fixed32). Routing those through the encryption dispatcher would short-circuit the legacy proto3 fallback in `decodeLegacyRaftRequest` for any RaftCommand/Request payload whose first encoded field is field 1 (e.g. `Request.is_txn` = true → first bytes `0x08 0x01`). Bytes 0x03..0x07 are SAFE because they encode either field 0 (proto3 disallows field 0) or reserved/invalid wire types (0x06/0x07 = wire types 6/7 which proto3 marks reserved), so no valid proto3 marshal output starts with them. Future encryption opcodes 0x08+ would need a different dispatch shape (e.g. a 2-byte sentinel) before they could be routed safely.

View Source
const WireVersionV1 byte = 0x01

WireVersionV1 is the current FSM-wire version. Carried as the second byte of every payload (after the opcode tag, which is stripped before fsmwire sees the payload).

Variables

View Source
var (
	// ErrFSMWireMalformed indicates the payload bytes do not
	// match the fixed-shape decoder for the supplied opcode.
	// Length mismatch, missing length-prefixed body, or trailing
	// garbage all surface here.
	ErrFSMWireMalformed = errors.New("fsmwire: payload is malformed")

	// ErrFSMWireVersion indicates the payload version byte is
	// not WireVersionV1. The decoder fails closed rather than
	// trying to interpret an unknown version under the v1
	// layout.
	ErrFSMWireVersion = errors.New("fsmwire: unknown wire version")

	// ErrFSMWireSubtag indicates the rotation opcode (0x05)
	// carries an unknown sub-tag. Reserved for forward-compat:
	// future opcodes that share the rotation tag (rewrap-deks,
	// retire-dek, enable-storage-envelope, enable-raft-envelope)
	// will allocate distinct sub-tags.
	ErrFSMWireSubtag = errors.New("fsmwire: unknown rotation sub-tag")
)

Errors returned by fsmwire's encoders/decoders. All three are terminal in the apply path — kv/fsm.go marks them with ErrEncryptionApply so internal/raftengine/etcd halts the apply loop instead of silently advancing setApplied.

Functions

func EncodeBootstrap

func EncodeBootstrap(p BootstrapPayload) []byte

EncodeBootstrap serialises p as the OpBootstrap payload (no leading opcode tag).

Wire layout:

[ver 1]
[storage_dek_id 4] [storage_wrapped_len 4] [storage_wrapped ...]
[raft_dek_id 4]    [raft_wrapped_len 4]    [raft_wrapped ...]
[batch_count 4]    [registration_payload]*batch_count

The two wrapped-DEK length prefixes are independent; the concrete length is whatever the configured KEK produced (the reference FileWrapper produces 60 bytes per 32-byte DEK).

func EncodeRegistration

func EncodeRegistration(p RegistrationPayload) []byte

EncodeRegistration serialises p as the OpRegistration payload (without the leading 0x03 opcode tag — that is added by the kv/fsm.go dispatch layer).

Wire layout:

[ver 1] [dek_id 4 BE] [full_node_id 8 BE] [local_epoch 2 BE]

func EncodeRotation

func EncodeRotation(p RotationPayload) []byte

EncodeRotation serialises p as the OpRotation payload (no leading opcode tag).

Wire layout (RotateSubRotateDEK):

[ver 1] [subtag 1] [dek_id 4] [purpose 1]
[wrapped_len 4] [wrapped ...]
[proposer_dek_id 4] [proposer_full_node_id 8] [proposer_local_epoch 2]

Types

type BootstrapPayload

type BootstrapPayload struct {
	StorageDEKID   uint32
	WrappedStorage []byte
	RaftDEKID      uint32
	WrappedRaft    []byte
	BatchRegistry  []RegistrationPayload
}

BootstrapPayload is the OpBootstrap body: §5.6 step 1a's "install the initial wrapped DEK pair plus a batch of writer-registry rows for every member that passed the capability pre-check".

The wrapped DEKs are KEK-output blobs the leader produced; the FSM apply layer installs them into the local keystore via Keystore.Set after the KEK Unwrap.

func DecodeBootstrap

func DecodeBootstrap(raw []byte) (BootstrapPayload, error)

DecodeBootstrap reverses EncodeBootstrap. Fails closed on any length-prefix overrun or version mismatch.

type Purpose

type Purpose byte

Purpose is the §5.1 sidecar purpose tag. The Cipher itself does not enforce purpose — that contract is maintained by the sidecar loader and the FSM apply handlers in kv/fsm_encryption.go. The fsmwire codec carries the purpose alongside each DEK so the applier can install it under the correct sidecar slot.

const (
	// PurposeStorage marks the storage-layer DEK (§4.1 envelopes).
	PurposeStorage Purpose = 1
	// PurposeRaft marks the raft-layer DEK (§4.2 envelopes).
	PurposeRaft Purpose = 2
)

type RegistrationPayload

type RegistrationPayload struct {
	DEKID      uint32
	FullNodeID uint64
	LocalEpoch uint16
}

RegistrationPayload is the OpRegistration body: a single (dek_id, full_node_id, local_epoch) triple inserted by FSM apply into the §4.1 writer registry. The kv/fsm_encryption.go handler decides between case 1 (insert), 2 (re-register / monotonic bump), 3 (rollback → ErrLocalEpochRollback) and 4 (collision → ErrNodeIDCollision).

func DecodeRegistration

func DecodeRegistration(raw []byte) (RegistrationPayload, error)

DecodeRegistration reverses EncodeRegistration. Fails closed on length mismatch or unknown version byte.

type RotationPayload

type RotationPayload struct {
	SubTag               byte
	DEKID                uint32
	Purpose              Purpose
	Wrapped              []byte
	ProposerRegistration RegistrationPayload
}

RotationPayload is the OpRotation body. SubTag selects between rotate-dek / rewrap / enable-flag variants. Stage 4 ships RotateSubRotateDEK; Stage 6D adds RotateSubEnableStorageEnvelope.

For RotateSubRotateDEK the payload is:

  • DEKID: the new key id being installed
  • Purpose: which sidecar slot to update (PurposeStorage / PurposeRaft)
  • Wrapped: the KEK-wrapped DEK bytes
  • ProposerRegistration: the proposing node's writer-registry row, inserted in the same FSM apply transaction so the proposer's first encrypted write under the new DEK is covered by the §4.1 case 2 epoch monotonicity check.

For RotateSubEnableStorageEnvelope the payload reuses the same triple but with stricter constraints (validated in both the cutover RPC mutator on the propose side and `ApplyRotation` on the apply side, per the 6D design's §2.1):

  • DEKID: MUST equal sidecar.Active.Storage at apply time
  • Purpose: MUST be PurposeStorage
  • Wrapped: MUST be empty (`len(Wrapped) == 0`, NOT nil — the wire decoder materialises zero-length payloads as an allocated empty slice)
  • ProposerRegistration: covers the proposer's first write under the now-active envelope

func DecodeRotation

func DecodeRotation(raw []byte) (RotationPayload, error)

DecodeRotation reverses EncodeRotation. Fails closed on length prefix overrun, version mismatch, or unknown sub-tag.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL