consensus

package
v0.1.0-proto2g Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 6, 2026 License: MPL-2.0 Imports: 26 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrBootstrapExpectMismatch means peers disagree on how large the
	// seed set is. Each disagreeing subset would elect its own
	// bootstrapper and produce two clusters that gossip but never share
	// a log - the exact failure the seed model was introduced to avoid.
	ErrBootstrapExpectMismatch = errors.New("peers disagree on bootstrap-expect")

	// ErrDuplicateNodeID means two members advertise the same Raft
	// server ID. Raft would treat them as one server at two addresses.
	ErrDuplicateNodeID = errors.New("duplicate node id in bootstrap set")
)

Bootstrap faults are configuration faults, not transient conditions: a node that sees them must fail loudly rather than retry into a split cluster.

View Source 
var (
	// ErrOperatorRegistryEmpty means no operator key is registered.
	// Every mutating verb refuses while this holds - fail closed is the
	// point of the registry, not a degraded mode.
	ErrOperatorRegistryEmpty = errors.New("operator key registry is empty: no mutation is authorized")

	// ErrOperatorRegistrySeeded means a seed arrived for a registry that
	// is already populated with a different set. Two nodes configured
	// with different root keys is a misconfiguration we refuse to paper
	// over by merging them: quietly unioning the sets would widen the
	// trust root without anyone deciding to.
	ErrOperatorRegistrySeeded = errors.New("operator key registry already seeded with a different key set")

	// ErrOperatorRegistryStale means a signed change named a registry
	// serial that is no longer current. This is the replay guard: a
	// change is valid at exactly one serial, so re-submitting a captured
	// one fails once any other change has landed.
	ErrOperatorRegistryStale = errors.New("operator key change: stale registry serial")

	// ErrOperatorLastKey means a remove would empty the registry.
	// Refusing keeps the operator from locking the cluster out of its
	// own control plane, and is what guarantees the seed path stays
	// reachable exactly once.
	ErrOperatorLastKey = errors.New("operator key change: refusing to remove the last registered key")
)
View Source 
var (
	// ErrSnapshotRegistryUnverifiable means the snapshot carries operator
	// keys and this node holds no root to check them against.
	ErrSnapshotRegistryUnverifiable = errors.New(
		"snapshot carries an operator key registry and this node has no configured operator keys to authenticate it against")

	// ErrSnapshotRegistryUnprovenanced means the snapshot carries a
	// registry with no provenance - the shape every snapshot had before
	// GOBLIN-DIV-047.
	ErrSnapshotRegistryUnprovenanced = errors.New(
		"snapshot carries an operator key registry with no provenance")

	// ErrSnapshotSeedMismatch means the snapshot's founding key set is not
	// this node's configured root of trust.
	ErrSnapshotSeedMismatch = errors.New(
		"snapshot's founding operator key set is not this node's configured root of trust")

	// ErrSnapshotChainInvalid means the recorded provenance does not
	// replay: a signature failed, an ordering guard failed, or the result
	// disagreed with the registry the snapshot claims.
	ErrSnapshotChainInvalid = errors.New("snapshot operator key provenance does not replay")
)
View Source 
var ErrCASMismatch = errors.New("cas: version mismatch")

ErrCASMismatch is returned (as the Apply response) when a CAS command's expected version does not match the key's current version. Errors are data: callers distinguish "CAS lost the race" from transport failures.

View Source 
var ErrIllegalTransition = errors.New("instance lifecycle: illegal transition")

ErrIllegalTransition is returned (as the Apply response) when an InstanceTransition violates the lifecycle FSM. Errors are data: callers distinguish a rejected transition from transport failures.

View Source 
var (

	// An error indicating a given key does not exist
	ErrKeyNotFound = errors.New("not found")
)
View Source 
var ErrMigrationInFlight = errors.New("migration already in flight for this instance")

ErrMigrationInFlight is returned when a second MIGRATE_BEGIN arrives for an instance that is already migrating. Typed because a proposer retrying after a lost response must distinguish "someone else is moving this" from "your request was malformed".

View Source 
var ErrNoMigrationInFlight = errors.New("no migration in flight for this instance")

ErrNoMigrationInFlight is returned when a commit arrives with no matching begin.

Functions

func LegalTransition 

func LegalTransition(from, to goblinv1.InstanceState) bool

LegalTransition reports whether the instance lifecycle FSM permits from -> to. The lifecycle is forward-only (DDR-6): ADMITTED -> SCHEDULED -> STARTING -> RUNNING -> DRAINING -> STOPPING -> TERMINATED -> ARCHIVED, with forward skips permitted (a dispatch may go ADMITTED -> RUNNING directly) and TERMINATED reachable from any non-terminal state. Nothing leaves ARCHIVED, nothing re-enters an earlier state, and UNSPECIFIED is never a destination.

Types

type BoltStore 

type BoltStore struct {
	// contains filtered or unexported fields
}

BoltStore provides access to BoltDB for Raft to store and retrieve log entries. It also provides key/value storage, and can be used as a LogStore and StableStore.

func New 

func New(options Options) (*BoltStore, error)

New uses the supplied options to open the BoltDB and prepare it for use as a raft backend.

func NewBoltStore 

func NewBoltStore(path string) (*BoltStore, error)

NewBoltStore takes a file path and returns a connected Raft backend.

func (*BoltStore) Close 

func (b *BoltStore) Close() error

Close is used to gracefully close the DB connection.

func (*BoltStore) DeleteRange 

func (b *BoltStore) DeleteRange(min, max uint64) error

DeleteRange is used to delete logs within a given range inclusively.

func (*BoltStore) FirstIndex 

func (b *BoltStore) FirstIndex() (uint64, error)

FirstIndex returns the first known index from the Raft log.

func (*BoltStore) Get 

func (b *BoltStore) Get(k []byte) ([]byte, error)

Get is used to retrieve a value from the k/v store by key

func (*BoltStore) GetLog 

func (b *BoltStore) GetLog(idx uint64, raftlog *raft.Log) error

GetLog is used to retrieve a log from BoltDB at a given index.

func (*BoltStore) GetUint64 

func (b *BoltStore) GetUint64(key []byte) (uint64, error)

GetUint64 is like Get, but handles uint64 values

func (*BoltStore) LastIndex 

func (b *BoltStore) LastIndex() (uint64, error)

LastIndex returns the last known index from the Raft log.

func (*BoltStore) Set 

func (b *BoltStore) Set(k, v []byte) error

Set is used to set a key/value set outside of the raft log

func (*BoltStore) SetUint64 

func (b *BoltStore) SetUint64(key []byte, val uint64) error

SetUint64 is like Set, but handles uint64 values

func (*BoltStore) StoreLog 

func (b *BoltStore) StoreLog(log *raft.Log) error

StoreLog is used to store a single raft log

func (*BoltStore) StoreLogs 

func (b *BoltStore) StoreLogs(logs []*raft.Log) error

StoreLogs is used to store a set of raft logs

func (*BoltStore) Sync 

func (b *BoltStore) Sync() error

Sync performs an fsync on the database file handle. This is not necessary under normal operation unless NoSync is enabled, in which this forces the database file to sync against the disk.

type BootstrapPeer 

type BootstrapPeer struct {
	NodeID string
	Addr   string
	Expect int
}

BootstrapPeer is one member's view as gossip reports it: its Raft server ID, its dialable control-plane address, and the bootstrap-expect size it was configured with (0 when it advertised none, which marks it a plain joiner rather than a seed).

type BootstrapPlan 

type BootstrapPlan struct {
	// Servers is the initial Raft configuration, ordered by node ID.
	Servers []raft.Server
	// Bootstrapper is the node ID that issues BootstrapCluster. The
	// other seeds do nothing and join through the resulting log.
	Bootstrapper string
}

BootstrapPlan is the decision every seed node reaches independently once it can see the whole expected set. Because it is a pure function of that set, all seeds compute the same plan and exactly one of them acts on it.

func PlanBootstrap 

func PlanBootstrap(expect int, peers []BootstrapPeer) (BootstrapPlan, bool, error)

PlanBootstrap decides whether the seed set is complete and, if so, which member seeds the cluster. It reports ready=false while peers are still missing; callers poll until it turns true, the context expires, or it returns an error.

expect < 2 means bootstrap-expect is not in play: the caller keeps the seed model (bootstrap alone when there is no join target).

type Consensus 

type Consensus struct {
	// contains filtered or unexported fields
}

Consensus manages cluster consensus using Raft

func NewConsensus 

func NewConsensus(nodeID, dataDir string, stream raft.StreamLayer, bootstrap bool,
	snapshotThreshold uint64, snapshotInterval time.Duration, trailingLogs uint64,
	trustedRoots []*goblinv1.OperatorKey) (*Consensus, error)

NewConsensus builds the Raft engine over the provided stream layer (the raft-quic plane of the shared control-plane listener; the layer's Addr() is this server's advertised raft address). bootstrap must be true on exactly one seed node (the one with no join target): every node bootstrapping its own single-node cluster yields N independent rafts that gossip but never share state - the failure mode the 2b e2e exposed.

snapshotThreshold, snapshotInterval, and trailingLogs tune Raft's snapshot-compaction behavior (raft.Config.SnapshotThreshold / SnapshotInterval / TrailingLogs); zero keeps raft.DefaultConfig's value for that field (GOBLIN-DIV-040: operators need these to bound trailing-log size and replay-on-join cost, not just the library defaults tuned for a generic workload). trustedRoots is this node's configured operator keys (--operator-key). They reach the FSM at construction so Restore can authenticate a snapshot's registry against material this node already holds rather than material the snapshot supplies (GOBLIN-DIV-047). Threaded through the constructor rather than set afterwards because raft may call Restore as soon as the FSM is handed to it.

func (*Consensus) AddVoter 

func (c *Consensus) AddVoter(id, address string) error

AddVoter adds a new voting member to the cluster

func (*Consensus) Apply 

func (c *Consensus) Apply(data []byte, timeout time.Duration) error

Apply applies a command to the Raft log

func (*Consensus) ApplyWithResponse 

func (c *Consensus) ApplyWithResponse(data []byte, timeout time.Duration) (interface{}, error)

ApplyWithResponse applies a command and returns the FSM's response value alongside any commit error. Commands whose outcome is data (CAS) return a typed error as the response; a nil, nil result is an applied success.

func (*Consensus) Bootstrap 

func (c *Consensus) Bootstrap(servers []raft.Server) error

Bootstrap installs an initial Raft configuration. It is separate from construction because bootstrap-expect cannot decide the server set until gossip has converged: the engine comes up as a configuration-less follower and is seeded once the peers are known.

ErrCantBootstrap means state already exists (a restart, or another seed won the race) - not an error.

func (*Consensus) GetInstance 

func (c *Consensus) GetInstance(instanceID string) (*goblinv1.AgentInstance, bool)

GetInstance returns a live instance record by canonical UUID string.

func (*Consensus) GetState 

func (c *Consensus) GetState(namespace, key string) ([]byte, bool)

GetState returns the current FSM state

func (*Consensus) GetStateWithVersion 

func (c *Consensus) GetStateWithVersion(namespace, key string) ([]byte, uint64, bool)

GetStateWithVersion returns the current FSM state and the key's CAS version (0 when the key is absent).

func (*Consensus) IsLeader 

func (c *Consensus) IsLeader() bool

IsLeader returns true if this node is the leader

func (*Consensus) IsTombstoned 

func (c *Consensus) IsTombstoned(instanceID string) bool

IsTombstoned reports whether an instance UUID was ever terminated.

func (*Consensus) Leader 

func (c *Consensus) Leader() string

Leader returns the current leader address

func (*Consensus) LeaderID 

func (c *Consensus) LeaderID() string

LeaderID returns the current leader ID

func (*Consensus) ListInstances 

func (c *Consensus) ListInstances() []*goblinv1.AgentInstance

ListInstances returns every live instance record.

func (*Consensus) MigrationInFlight 

func (c *Consensus) MigrationInFlight(instanceUUID []byte) (*goblinv1.MigrationRecord, bool)

MigrationInFlight reports the in-flight migration for one instance. Local read: the reconciler asks about an instance it is already looking at, and a stale "no migration" only costs the recovery it would have done anyway.

func (*Consensus) MigrationsInFlight 

func (c *Consensus) MigrationsInFlight() []*goblinv1.MigrationRecord

MigrationsInFlight lists every recorded in-flight migration, for the orphan sweep (GOBLIN-DIV-049).

func (*Consensus) NodeID 

func (c *Consensus) NodeID() string

NodeID is this node's raft server id. It exists so a refusal can name WHICH node refused without threading the supervisor's config through every gate (GOBLIN-DIV-048): a diagnostic that says "the registry was empty" is useless if it cannot say empty on whom.

func (*Consensus) OperatorKeyCountLocal 

func (c *Consensus) OperatorKeyCountLocal() int

OperatorKeyCountLocal reports how many operator keys THIS NODE has applied. Zero is the fail-closed condition: no key, no mutation. No leadership check, deliberately - see OperatorKeyCountLocal on the FSM for why staleness can only move this answer toward refusal.

func (*Consensus) OperatorKeysLocal 

func (c *Consensus) OperatorKeysLocal() ([]*goblinv1.OperatorKey, uint64)

OperatorKeysLocal returns THIS NODE's applied operator key registry and its serial (GOBLIN-DIV-015 piece 1). It performs no leadership check, so the answer may predate a committed change. Use OperatorKeysVerified for anything that authorizes; use this only where a stale answer cannot produce a yes, and say so at the call site.

func (*Consensus) OperatorKeysVerified 

func (c *Consensus) OperatorKeysVerified() ([]*goblinv1.OperatorKey, uint64, error)

OperatorKeysVerified returns the operator key registry and its serial only if this node is the leader with a live quorum (GOBLIN-DIV-044).

This is the accessor for anything that AUTHORIZES from key material. The unverified reads below answer from whatever this replica happens to have applied, and a follower that has not applied an OPERATOR_KEY_CHANGE remove still resolves the removed key - so a consumer that says yes on a successful lookup would mint for a revoked operator by asking a lagging replica. VerifyLeader is what makes that unreachable: a follower cannot answer at all, and a leader that lost quorum cannot either.

It is a refusal, not a forward. Routing to the leader is the caller's policy decision; this surface's job is only that a stale reader cannot say yes.

func (*Consensus) RemoveServer 

func (c *Consensus) RemoveServer(id string) error

RemoveServer removes a server from the cluster

func (*Consensus) Scan 

func (c *Consensus) Scan(namespace, prefix string) map[string][]byte

Scan returns all keys matching the prefix

func (*Consensus) Shutdown 

func (c *Consensus) Shutdown() error

Shutdown stops the consensus manager

func (*Consensus) Stats 

func (c *Consensus) Stats() map[string]string

Stats returns Raft statistics

func (*Consensus) VerifyLeader 

func (c *Consensus) VerifyLeader() error

VerifyLeader checks if this node is the leader and has a quorum

type FSM 

type FSM struct {
	// contains filtered or unexported fields
}

FSM implements the Raft finite state machine

func NewFSM 

func NewFSM(trustedRoots []*goblinv1.OperatorKey) *FSM

NewFSM creates a new FSM NewFSM builds the state machine. trustedRoots is this node's configured operator keys (--operator-key), which Restore uses to authenticate a snapshot's registry; pass nil for a node configured with none.

It is a CONSTRUCTOR PARAMETER rather than a setter, and that is not a style preference. A setter reintroduces an ordering hazard with real consequences: raft can call Restore as soon as the FSM is handed over, so an anchor installed "shortly after" construction is an anchor that might not be there when the first snapshot lands - and the failure would be silent acceptance of an unverified registry, which is the defect this closes.

func (*FSM) Apply 

func (f *FSM) Apply(log *raft.Log) interface{}

Apply applies a Raft log entry to the FSM. The returned value is the command's response (retrieved via raft.ApplyFuture.Response): nil on success, ErrCASMismatch on a failed CAS, an error for undecodable or unknown commands. A silent no-op on a distributed write is a data consistency hazard, so every path answers.

func (*FSM) Get 

func (f *FSM) Get(namespace, key string) ([]byte, bool)

Get retrieves a value from the FSM state

func (*FSM) GetInstance 

func (f *FSM) GetInstance(instanceID string) (*goblinv1.AgentInstance, bool)

GetInstance returns a copy of a live instance record by canonical UUID string.

func (*FSM) GetWithVersion 

func (f *FSM) GetWithVersion(namespace, key string) ([]byte, uint64, bool)

GetWithVersion retrieves a value and its CAS version. A missing key reports version 0 (the value CAS create-if-absent expects).

func (*FSM) IsTombstoned 

func (f *FSM) IsTombstoned(instanceID string) bool

IsTombstoned reports whether a UUID has ever been terminated. Tombstones are append-only forever.

func (*FSM) ListInstances 

func (f *FSM) ListInstances() []*goblinv1.AgentInstance

ListInstances returns copies of every live instance record.

func (*FSM) MigrationInFlight 

func (f *FSM) MigrationInFlight(instanceUUID []byte) (*goblinv1.MigrationRecord, bool)

MigrationInFlight reports the in-flight migration for an instance. Read path for goblinctl and the reconciler; takes the read lock.

func (*FSM) MigrationsInFlight 

func (f *FSM) MigrationsInFlight() []*goblinv1.MigrationRecord

MigrationsInFlight lists every migration currently recorded as in flight.

It exists for the orphan sweep (GOBLIN-DIV-049): nothing else ever clears one of these records. Only a MIGRATE_COMMIT does, and only the leader can propose one - so a leader that dies mid-migration leaves a record no surviving node will ever retire, and the reconciler now honours those records. Without a sweep that would trade a duplicated instance for a permanently unrecoverable one.

func (*FSM) OperatorKeyCountLocal 

func (f *FSM) OperatorKeyCountLocal() int

OperatorKeyCountLocal reports how many operator keys THIS REPLICA has applied. Zero is the fail-closed condition every mutating verb checks, and the count is the one reading that is safe to take locally: a replica behind the seed reads zero and refuses, and a seeded registry can never return to empty because removing the last key is refused. So this can be stale only in the direction that refuses.

func (*FSM) OperatorKeysLocal 

func (f *FSM) OperatorKeysLocal() ([]*goblinv1.OperatorKey, uint64)

OperatorKeysLocal returns THIS REPLICA's applied registry sorted by key id, plus the current serial. The Local suffix is the contract, not decoration: an FSM knows only what it has applied, so a follower that has not yet applied an OPERATOR_KEY_CHANGE answers from the registry as it was before the change - including a remove. Callers for whom a stale yes is wrong (anything that authorizes on key material, e.g. a mint path) must use Consensus.OperatorKeysVerified instead. Callers for whom a stale answer can only fail closed may use this and must say at the call site why.

func (*FSM) Restore 

func (f *FSM) Restore(rc io.ReadCloser) (err error)

Restore restores the FSM from a snapshot. Only the proto encoding is accepted (GOBLIN-DIV-040 schema reset): a snapshot written by the old JSON encoder is refused outright rather than dual-read, mirroring the CommandType-0 rejection above for the same reason - a compatibility path nothing forces anyone to remove never gets removed.

func (*FSM) Scan 

func (f *FSM) Scan(namespace, prefix string) map[string][]byte

Scan returns all key-value pairs in a namespace that match the prefix

func (*FSM) Snapshot 

func (f *FSM) Snapshot() (raft.FSMSnapshot, error)

Snapshot returns a snapshot of the FSM state

type Options 

type Options struct {
	// Path is the file path to the BoltDB to use
	Path string

	// BoltOptions contains any specific BoltDB options you might
	// want to specify [e.g. open timeout]
	BoltOptions *bolt.Options

	// NoSync causes the database to skip fsync calls after each
	// write to the log. This is unsafe, so it should be used
	// with caution.
	NoSync bool
}

Options contains all the configuration used to open the BoltDB

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.