Documentation
¶
Overview ¶
Package consensus — ranges 0.2.57.* (ADR 0152, the domain's outcomes) and 0.3.92.* (ADR 0152, the shipped storage and transport's outcomes, declared here since ADR 0160).
Package consensus declares the SDK's consensus DOMAIN: agreement of several machines on the order of their writes. It holds every value a port speaks — node and cluster identities, positions, entries, messages, snapshots, membership — and the frozen ports themselves: Replicated, the consumer's algorithm-neutral port, StateMachine and its siblings, and the storage and transport ports LogStore, SnapshotStore and Transport, which speak Raft's terms (ADR 0152 D1).
A write is committed when a majority holds it on disk ¶
An entry is committed once a majority of the voters have made it durable. Every port in this package exists to keep that sentence true across crashes, partitions, restarts, recoveries and upgrades: LogStore.Sync and LogStore.WriteStamp carry the safety of the whole domain, and a failed sync stops the node rather than being retried.
This is not ADR 0120's state machine ¶
StateMachine is HOW a deterministic program is replicated: a pure transition applied in log order on every node. The statemachine domain decides WHAT state an entity is in and moves it on a timer. The two share a textbook name and nothing else.
The engine, its storage and its transport live in internal/service/app/consensus; the facade is pkg/v1/app/consensus.
Index ¶
- Constants
- Variables
- type AlarmID
- type AuthenticatedPeerValue
- type BatchingStateMachine
- type ClusterID
- type DurableStateMachine
- type EntryKind
- type EntryValue
- type FencedAttestationValue
- type FrameLane
- type GroupID
- type HardState
- type Incarnation
- type InstallAcceptValue
- type InstallOfferValue
- type InstallParams
- type JoinAuthorizer
- type JoinConfig
- type JoinReasonBinder
- type JoinRequestValue
- type Joiner
- type LeadershipWatcher
- type LogIdentityValue
- type LogReader
- type LogStore
- type MemberValue
- type MembershipValue
- type MessageFlags
- type MessageKind
- type MessageValue
- type NodeID
- type Operator
- type PositionValue
- type PreconditionSpec
- type Replicated
- type Role
- type Snapshot
- type SnapshotHandle
- type SnapshotMetaValue
- type SnapshotSink
- type SnapshotStore
- type StampState
- type StampValue
- type StateMachine
- type StatusValue
- type StopReasonValue
- type TimingContractSpec
- type TokenValue
- type Transport
- type UnreachableReporter
- type VersionsValue
Constants ¶
const CodeAdminPreconditionFailed errs.Code = 0x00_02_39_19 // 0.2.57.25
CodeAdminPreconditionFailed identifies an administrative operation that named an alarm, version or contract that is not the current one.
const CodeBootstrapRefused errs.Code = 0x00_03_5C_0B // 0.3.92.11
CodeBootstrapRefused identifies a bootstrap against a conflicting history.
const CodeClusterMismatch errs.Code = 0x00_03_5C_08 // 0.3.92.8
CodeClusterMismatch identifies a peer's first frame naming another cluster or group.
const CodeCompacted errs.Code = 0x00_02_39_09 // 0.2.57.9
CodeCompacted identifies an index below the first retained one.
const CodeConsensusMisconfigured errs.Code = 0x00_02_39_01 // 0.2.57.1
CodeConsensusMisconfigured identifies a node refused at construction: a bound missing or out of range, a timing rule broken, a lease without a drift.
const CodeDataDirInUse errs.Code = 0x00_03_5C_10 // 0.3.92.16
CodeDataDirInUse identifies an offline operation on a directory a live node holds.
const CodeDataDirLocked errs.Code = 0x00_03_5C_05 // 0.3.92.5
CodeDataDirLocked identifies a data directory another process holds.
const CodeDataDirMismatch errs.Code = 0x00_03_5C_06 // 0.3.92.6
CodeDataDirMismatch identifies a stamp naming another cluster, node or incarnation.
const CodeEntrySuperseded errs.Code = 0x00_02_39_0C // 0.2.57.12
CodeEntrySuperseded identifies a proposal whose position holds a COMMITTED entry of another term: certainly not committed.
const CodeEntryTooLarge errs.Code = 0x00_02_39_08 // 0.2.57.8
CodeEntryTooLarge identifies a command above MaxEntryBytes.
const CodeFrameInvalid errs.Code = 0x00_03_5C_09 // 0.3.92.9
CodeFrameInvalid identifies a frame that does not parse or is too large.
const CodeHardStateCorrupt errs.Code = 0x00_03_5C_02 // 0.3.92.2
CodeHardStateCorrupt identifies an unreadable hard-state record.
const CodeJoinRefused errs.Code = 0x00_03_5C_11 // 0.3.92.17
CodeJoinRefused identifies a pre-admission JOIN that was refused.
const CodeLeadershipUnconfirmed errs.Code = 0x00_02_39_04 // 0.2.57.4
CodeLeadershipUnconfirmed identifies a barrier whose confirmation round did not reach a majority in time.
const CodeLineageMismatch errs.Code = 0x00_02_39_14 // 0.2.57.20
CodeLineageMismatch identifies a position, outbox identity or fence from another LogIdentity lineage.
const CodeLogCorrupt errs.Code = 0x00_03_5C_01 // 0.3.92.1
CodeLogCorrupt identifies damage to the log that is not a torn tail.
const CodeLogQuarantined errs.Code = 0x00_03_5C_0E // 0.3.92.14
CodeLogQuarantined identifies a data directory carrying a quarantine marker.
const CodeMembershipChangePending errs.Code = 0x00_02_39_05 // 0.2.57.5
CodeMembershipChangePending identifies a second membership change proposed while one is still uncommitted.
const CodeMembershipInvalid errs.Code = 0x00_02_39_06 // 0.2.57.6
CodeMembershipInvalid identifies a membership change refused on its merits.
const CodeNoSpace errs.Code = 0x00_02_39_11 // 0.2.57.17
CodeNoSpace identifies a write refused while the no-space alarm is raised.
const CodeNodeIDRetired errs.Code = 0x00_02_39_10 // 0.2.57.16
CodeNodeIDRetired identifies a removed NodeID offered again.
const CodeNodeStopped errs.Code = 0x00_02_39_0B // 0.2.57.11
CodeNodeStopped identifies a call on a node that was closed or stopped itself.
const CodeNotLeader errs.Code = 0x00_02_39_02 // 0.2.57.2
CodeNotLeader identifies a proposal that was certainly NOT appended, because this node is not the leader and could not forward it.
const CodeOutcomeUnknown errs.Code = 0x00_02_39_03 // 0.2.57.3
CodeOutcomeUnknown identifies a proposal that was appended, or possibly delivered, and whose answer was lost. It may still commit.
const CodeOverloaded errs.Code = 0x00_02_39_0D // 0.2.57.13
CodeOverloaded identifies a proposal refused before append.
const CodePayloadVersionRefused errs.Code = 0x00_02_39_13 // 0.2.57.19
CodePayloadVersionRefused identifies a proposal whose payload version is above what some member announced.
const CodePeerRejected errs.Code = 0x00_03_5C_07 // 0.3.92.7
CodePeerRejected identifies a peer whose certificate or key is not admitted.
const CodeRecoverRefused errs.Code = 0x00_03_5C_0D // 0.3.92.13
CodeRecoverRefused identifies a Recover whose preconditions do not hold.
const CodeRestoreRefused errs.Code = 0x00_03_5C_0F // 0.3.92.15
CodeRestoreRefused identifies a RestoreSnapshot whose preconditions do not hold.
const CodeResultTypeMismatch errs.Code = 0x00_02_39_12 // 0.2.57.18
CodeResultTypeMismatch identifies an Apply result of another type than the one ProposeAs asked for. The entry IS committed.
const CodeSnapshotCorrupt errs.Code = 0x00_03_5C_03 // 0.3.92.3
CodeSnapshotCorrupt identifies a snapshot whose digest or size does not match its metadata.
const CodeSnapshotTooLarge errs.Code = 0x00_03_5C_0A // 0.3.92.10
CodeSnapshotTooLarge identifies an announced snapshot above MaxSnapshotBytes.
const CodeStateMachineFailed errs.Code = 0x00_02_39_0A // 0.2.57.10
CodeStateMachineFailed identifies a node whose Apply panicked or could not complete. The entry IS committed.
const CodeStopMarkerPresent errs.Code = 0x00_03_5C_12 // 0.3.92.18
CodeStopMarkerPresent identifies a node refused at start by a stop marker.
const CodeStorageFailed errs.Code = 0x00_03_5C_04 // 0.3.92.4
CodeStorageFailed identifies a failed write, sync or read.
const CodeTransferFailed errs.Code = 0x00_02_39_07 // 0.2.57.7
CodeTransferFailed identifies a leadership transfer whose target did not win within one election timeout.
const CodeTransportMisconfigured errs.Code = 0x00_03_5C_0C // 0.3.92.12
CodeTransportMisconfigured identifies a transport refused at construction.
const CodeVersionUnsupported errs.Code = 0x00_02_39_0F // 0.2.57.15
CodeVersionUnsupported identifies an entry kind, record, frame or snapshot version this node does not know.
Variables ¶
var ( // ConsensusMisconfigured is returned by a constructor whose configuration // it cannot honour: a bound out of range, a heartbeat not below a fifth of // the minimum election timeout, a lease without a stated clock drift, a // durability that disagrees with the state machine. ConsensusMisconfigured = errs.Static(CodeConsensusMisconfigured, "CONSENSUS_MISCONFIGURED", "The consensus node cannot be built from the given configuration", "core/app/consensus: a configuration field is missing, out of range, or contradicts another; the fields name it", 0, 78) // NotLeader is returned when a proposal was certainly NOT appended. It is // safe to retry; LeaderHint reads the known leader from its fields. NotLeader = errs.Static(CodeNotLeader, "NOT_LEADER", "This node is not the leader; the proposal was not appended", "core/app/consensus: the proposal was refused before append because this node does not lead and could not forward it", 0, 75) // OutcomeUnknown is returned when a proposal was appended, or possibly // delivered, and its answer was lost. The entry may still commit: never // retry blindly, read the state through Barrier first. OutcomeUnknown = errs.Static(CodeOutcomeUnknown, "OUTCOME_UNKNOWN", "The proposal may or may not have been committed", "core/app/consensus: the proposal was appended or possibly delivered, then the answer was lost to a leadership change, a context's end or a broken connection", 0, 75) // LeadershipUnconfirmed is returned by Barrier when its round did not reach // a majority in time. LeadershipUnconfirmed = errs.Static(CodeLeadershipUnconfirmed, "LEADERSHIP_UNCONFIRMED", "The leader could not confirm its leadership", "core/app/consensus: a read barrier's heartbeat round did not reach a majority of the configuration in time", 0, 75) // MembershipChangePending is returned while another membership change is // still uncommitted: one change at a time. MembershipChangePending = errs.Static(CodeMembershipChangePending, "MEMBERSHIP_CHANGE_PENDING", "Another membership change is still pending", "core/app/consensus: a membership change was proposed while a previous one is uncommitted", 0, 75) // MembershipInvalid is returned for a change refused on its merits: an // unknown node, the last voter, a removal below MinVoters, a promotion // without durable progress, a change without a reachable majority. MembershipInvalid = errs.Static(CodeMembershipInvalid, "MEMBERSHIP_INVALID", "The membership change is not allowed", "core/app/consensus: the membership change names an unknown node, leaves too few voters, or would leave no reachable majority", 0, 65) // TransferFailed is returned when a leadership transfer's target did not // win within one election timeout. The old leader resumes, without its // lease for the rest of the term. TransferFailed = errs.Static(CodeTransferFailed, "TRANSFER_FAILED", "The leadership transfer did not complete", "core/app/consensus: the transfer target did not win an election within one election timeout", 0, 75) // EntryTooLarge is returned for a command above MaxEntryBytes. EntryTooLarge = errs.Static(CodeEntryTooLarge, "ENTRY_TOO_LARGE", "The proposed entry is too large", "core/app/consensus: the command exceeds MaxEntryBytes", 0, 65) // Compacted is the answer for an index below the first retained one. It is // an expected outcome the engine acts on, never a failure. Compacted = errs.Static(CodeCompacted, "COMPACTED", "The requested log position has been compacted", "core/app/consensus: the index is below the first index the log retains", 0, 75) // StateMachineFailed stops the node when Apply panics or cannot complete. // The entry IS committed: this never means "not written". StateMachineFailed = errs.Static(CodeStateMachineFailed, "STATE_MACHINE_FAILED", "The replicated state machine failed and the node stopped", "core/app/consensus: Apply panicked or could not complete; the entry is committed and the node stopped applying", 0, 69) // NodeStopped is returned by a call on a node that was closed or stopped // itself. NodeStopped = errs.Static(CodeNodeStopped, "NODE_STOPPED", "The consensus node is stopped", "core/app/consensus: the node was closed, or stopped itself; its StopReason says why", 0, 75) // EntrySuperseded is returned when a COMMITTED entry at the proposal's // index has another term: the proposal certainly did not commit. EntrySuperseded = errs.Static(CodeEntrySuperseded, "ENTRY_SUPERSEDED", "Another entry was committed at the proposal's position", "core/app/consensus: the entry committed at the proposal's index carries another term", 0, 75) // Overloaded is returned when a proposal is refused before append: pending // bytes or apply lag over their bound. It is safe to retry. Overloaded = errs.Static(CodeOverloaded, "OVERLOADED", "The cluster is overloaded; the proposal was not appended", "core/app/consensus: pending proposal bytes or apply lag are over their bound; nothing was appended", 0, 75) // VersionUnsupported stops the node on an entry kind, record, frame or // snapshot version it does not know. Skipping one would be a fork. VersionUnsupported = errs.Static(CodeVersionUnsupported, "VERSION_UNSUPPORTED", "A version this node does not support was met", "core/app/consensus: an entry kind, log record, frame or snapshot version is unknown to this node, which stops", 0, 76) // NodeIDRetired is returned when a removed NodeID is offered again. NodeIDRetired = errs.Static(CodeNodeIDRetired, "NODE_ID_RETIRED", "The node ID has been retired and cannot be used again", "core/app/consensus: the NodeID is in the replicated retired set", 0, 65) // NoSpace is returned for an ordinary proposal while the replicated // no-space alarm is raised. NoSpace = errs.Static(CodeNoSpace, "NO_SPACE", "Writes are refused until the no-space alarm is cleared", "core/app/consensus: the replicated no-space alarm is raised; only the alarm, its clearing and administrative entries are admitted", 0, 75) // ResultTypeMismatch is returned by ProposeAs for an Apply result of // another type. The entry IS committed; TokenOf reads its token. ResultTypeMismatch = errs.Static(CodeResultTypeMismatch, "RESULT_TYPE_MISMATCH", "The entry was committed but its result has an unexpected type", "core/app/consensus: ProposeAs got an Apply result of another dynamic type; the entry is committed", 0, 65) // PayloadVersionRefused is returned for a proposal whose payload version is // above what some member announced; retryable after the upgrade. PayloadVersionRefused = errs.Static(CodePayloadVersionRefused, "PAYLOAD_VERSION_REFUSED", "The payload version is not yet supported by every member", "core/app/consensus: the proposal's payload version exceeds the cluster's committed floor; nothing was appended", 0, 75) // LineageMismatch is returned for a position, outbox identity or fence // from another LogIdentity lineage. LineageMismatch = errs.Static(CodeLineageMismatch, "LINEAGE_MISMATCH", "The position belongs to another log lineage", "core/app/consensus: a token or precondition names another ClusterID or log epoch than this log's", 0, 65) // AdminPreconditionFailed is returned when a clearing or raising // operation names an alarm, version or contract that is not current. AdminPreconditionFailed = errs.Static(CodeAdminPreconditionFailed, "ADMIN_PRECONDITION_FAILED", "The administrative operation's precondition no longer holds", "core/app/consensus: the precondition names an alarm, contract or version generation that is not the current one", 0, 75) // LogCorrupt is a CRC failure that is not a torn tail, or a gap in the // effective log. It is never repaired. LogCorrupt = errs.Static(CodeLogCorrupt, "LOG_CORRUPT", "The consensus log is corrupt", "core/app/consensus: a log record failed its checksum outside a torn tail, or the effective indexes have a gap", 0, 65) // HardStateCorrupt is an unreadable hard-state record or checkpoint; it is // never defaulted. HardStateCorrupt = errs.Static(CodeHardStateCorrupt, "HARD_STATE_CORRUPT", "The consensus hard state is corrupt", "core/app/consensus: the hard-state record or the checkpoint holding it is unreadable", 0, 65) // SnapshotCorrupt is a snapshot whose SHA-256 or size does not match its // metadata. SnapshotCorrupt = errs.Static(CodeSnapshotCorrupt, "SNAPSHOT_CORRUPT", "The snapshot is corrupt", "core/app/consensus: a snapshot's digest, chunk checksum or size does not match its metadata", 0, 65) // StorageFailed stops the node on a failed write, sync or read. It is // never retried: a retried fsync can report success over lost pages. StorageFailed = errs.Static(CodeStorageFailed, "STORAGE_FAILED", "The consensus storage failed and the node stopped", "core/app/consensus: a write, fsync or read failed; the node stops and refuses to start again until the machine reboots", 0, 74) // DataDirLocked is returned when another process holds the data directory. DataDirLocked = errs.Static(CodeDataDirLocked, "DATA_DIR_LOCKED", "The data directory is held by another process", "core/app/consensus: the data directory lock is held by another process", 0, 75) // DataDirMismatch is returned when the stamp names another cluster, node // or incarnation, or an applied position cannot be trusted. DataDirMismatch = errs.Static(CodeDataDirMismatch, "DATA_DIR_MISMATCH", "The data directory belongs to another node or cluster", "core/app/consensus: the stamp or the applied position names another cluster, node, incarnation or lineage", 0, 78) // PeerRejected is returned when a peer's certificate is not canonical, // names another cluster or slot, or carries a key that is not pinned. PeerRejected = errs.Static(CodePeerRejected, "PEER_REJECTED", "The peer was rejected", "core/app/consensus: the peer certificate's URI SAN is not canonical, names another cluster or slot, or its key is not admitted", 0, 75) // ClusterMismatch is returned when a peer's first frame names another // cluster or group. ClusterMismatch = errs.Static(CodeClusterMismatch, "CLUSTER_MISMATCH", "The peer belongs to another cluster", "core/app/consensus: the first frame names another cluster or group", 0, 78) // FrameInvalid is a frame that does not parse or exceeds MaxFrameBytes; // the connection closes. FrameInvalid = errs.Static(CodeFrameInvalid, "FRAME_INVALID", "A malformed frame was received", "core/app/consensus: a frame does not parse, or its length exceeds the frame cap", 0, 65) // SnapshotTooLarge is an announced snapshot above the local // MaxSnapshotBytes, refused before anything is spooled. SnapshotTooLarge = errs.Static(CodeSnapshotTooLarge, "SNAPSHOT_TOO_LARGE", "The snapshot is larger than this node accepts", "core/app/consensus: the announced snapshot size exceeds MaxSnapshotBytes", 0, 65) // BootstrapRefused is returned when a listed peer already holds a history, // or the founding member lists differ. BootstrapRefused = errs.Static(CodeBootstrapRefused, "BOOTSTRAP_REFUSED", "The cluster cannot be bootstrapped", "core/app/consensus: the data directory already holds log entries, or a bootstrap with another member list is pending", 0, 78) // TransportMisconfigured is returned when the transport cannot be built // from its configuration. TransportMisconfigured = errs.Static(CodeTransportMisconfigured, "TRANSPORT_MISCONFIGURED", "The consensus transport cannot be built from the given configuration", "core/app/consensus: an identity without roots, a listener without client-certificate verification, or a limit out of range", 0, 78) // RecoverRefused is returned when Recover's preconditions do not hold. RecoverRefused = errs.Static(CodeRecoverRefused, "RECOVER_REFUSED", "The recovery was refused", "core/app/consensus: Recover needs a stopped, locked directory, a new ClusterID, the typed confirmation and a signed fence for every old voter", 0, 78) // LogQuarantined is returned at start while the data directory carries a // quarantine marker; only ClearQuarantine lifts it. LogQuarantined = errs.Static(CodeLogQuarantined, "LOG_QUARANTINED", "The consensus log is quarantined", "core/app/consensus: the data directory carries a quarantine marker from a LOG_CORRUPT; an operator must clear it", 0, 65) // RestoreRefused is returned when RestoreSnapshot's preconditions do not // hold. RestoreRefused = errs.Static(CodeRestoreRefused, "RESTORE_REFUSED", "The snapshot restore was refused", "core/app/consensus: RestoreSnapshot needs an empty directory, a non-zero expected digest that matches, and the typed confirmation", 0, 78) // DataDirInUse is returned by an offline operation on a directory a live // node holds. DataDirInUse = errs.Static(CodeDataDirInUse, "DATA_DIR_IN_USE", "The data directory is in use by a running node", "core/app/consensus: Close, Recover or Inspect was called while a live Node holds the directory", 0, 75) // JoinRefused is returned to a pre-admission JOIN that was refused. JoinRefused = errs.Static(CodeJoinRefused, "JOIN_REFUSED", "The join request was refused", "core/app/consensus: the JOIN names a slot whose member is not fenced, exceeds its rate, or reuses a retired id", 0, 75) // StopMarkerPresent is returned by NewRaft when a stop marker applies. StopMarkerPresent = errs.Static(CodeStopMarkerPresent, "STOP_MARKER_PRESENT", "The node refuses to start until its stop marker is cleared", "core/app/consensus: a stop marker is present; STORAGE_FAILED clears at reboot, the others by an operator", 0, 65) )
Functions ¶
This section is empty.
Types ¶
type AlarmID ¶
type AlarmID uint64
AlarmID names an alarm by the index of the entry that raised it, so that a clearing operation names the alarm it means to clear.
type AuthenticatedPeerValue ¶ added in v0.3.0
type AuthenticatedPeerValue struct {
// ID is the admitted member, 0 for a joiner.
ID NodeID
// Incarnation is that member's incarnation.
Incarnation Incarnation
// Slot is the slot the certificate's URI SAN names.
Slot string
// SPKI is the SHA-256 of the certificate's SubjectPublicKeyInfo.
SPKI [32]byte
}
AuthenticatedPeerValue is what the TLS handshake proved, never what a frame claims. The certificate proves the slot and the key; the ID and the incarnation are the member the transport admitted on that key. A peer that is not yet a member, during a JOIN, has ID 0 and only Slot and SPKI set.
type BatchingStateMachine ¶
type BatchingStateMachine interface {
StateMachine
// ApplyBatch applies es in order and writes each result into results.
ApplyBatch(es []EntryValue, results []any)
}
BatchingStateMachine is the sibling of a state machine that applies committed commands in batches, one result slot per entry.
type ClusterID ¶
type ClusterID [16]byte
ClusterID names one cluster: the founding, or a Recover or RestoreSnapshot that started a new history. It is 16 bytes everywhere it is encoded, the URI SAN included, where it is spelt in hex (see String).
type DurableStateMachine ¶
type DurableStateMachine interface {
StateMachine
// AppliedPosition returns the log identity and the position of the last
// entry whose effects are durable.
AppliedPosition() (LogIdentityValue, PositionValue, error)
}
DurableStateMachine is the sibling of a state machine that keeps its state on disk: it persists the applied position atomically with the state that entry produced, and the engine resumes after it.
type EntryKind ¶
type EntryKind uint8
EntryKind says what a log entry carries. Only EntryCommand reaches the state machine; every other kind is the engine's own replicated state. A kind a node does not know stops it with VERSION_UNSUPPORTED: it is never skipped.
const ( // EntryCommand is a caller's command, handed to StateMachine.Apply. EntryCommand EntryKind = iota + 1 // EntryNoOp is the empty entry a new leader commits in its term before it // serves a read or accepts a membership change. EntryNoOp // EntryMembership carries a whole new Membership and the change that // produced it. It takes effect when APPENDED. EntryMembership // EntryTimingContract carries a new committed timing contract. EntryTimingContract // EntryClusterVersion raises the protocol version every member must speak. EntryClusterVersion // EntryNoSpaceRaise raises the replicated no-space alarm. EntryNoSpaceRaise // EntryNoSpaceClear clears the no-space alarm it names. EntryNoSpaceClear // EntryTransfer records a leadership transfer and its operator. EntryTransfer )
The entry kinds of protocol version 1. The range 0x80-0x8F is reserved for ADR 0173 and refused until the cluster version enables it.
type EntryValue ¶ added in v0.3.0
type EntryValue struct {
// Position is the entry's term and index.
Position PositionValue
// Kind says what Data carries.
Kind EntryKind
// Version is the payload version the proposer set; it selects the frozen
// code path that applies the entry.
Version uint32
// BodySum is the CRC-32C of Data, computed by the proposer.
BodySum uint32
// ChainSum chains the previous entry's header: computed by the LEADER at
// its serialized append point, verified at append and before Apply.
ChainSum uint32
// Data is the payload: a command for EntryCommand, the engine's encoding
// for the other kinds.
Data []byte
// Token is the entry's lineage-scoped command identity, set in memory by
// the engine before Apply and never encoded in the log.
Token TokenValue
}
EntryValue is one record of the replicated log. Data is immutable once appended: the store and the engine never write into it, and a caller that hands bytes to Propose gives up their ownership.
type FencedAttestationValue ¶ added in v0.3.0
type FencedAttestationValue struct {
// Voter is the old voter.
Voter NodeID
// Method is how it was fenced: "stopped", "powered-off" or "cut".
Method string
// Receipts are Ed25519 signatures, one per witness key, over the old
// cluster id, the new cluster id and the voter.
Receipts [][]byte
}
FencedAttestationValue is the operator's statement that an old voter was fenced before a Recover: the voter, the method that fenced it, and one signed receipt per witness (one per NAS).
type FrameLane ¶
type FrameLane uint8
FrameLane is the connection a frame belongs on: control, data or snapshot. It is carried in the two high bits of the frame's type byte, so a transport routes a frame of a type it does not parse.
type GroupID ¶
type GroupID uint64
GroupID names a consensus group. v1 runs exactly one group, 1, and every frame carries it so that a later version can multiplex groups without changing the wire.
type HardState ¶
type HardState struct {
// Term is the node's current term.
Term uint64
// Vote is the candidate voted for in Term, 0 for none.
Vote NodeID
// Commit is the highest index known committed.
Commit uint64
}
HardState is what a node must never forget: its term, its vote in that term and the commit index it knows of. Term and vote are durable before any reply depends on them; the commit index is written lazily, never above the last synced index.
type Incarnation ¶
type Incarnation [16]byte
Incarnation is a random 128-bit value drawn when a data directory is first stamped. A wiped machine that kept its certificate and its NodeID is still a different incarnation, and is refused. It is an identifier, not a secret.
func (Incarnation) IsZero ¶
func (i Incarnation) IsZero() bool
IsZero reports whether no incarnation was drawn.
func (Incarnation) String ¶
func (i Incarnation) String() string
String renders the incarnation in lowercase hex.
type InstallAcceptValue ¶ added in v0.3.0
type InstallAcceptValue struct {
// SnapshotID is the snapshot answered.
SnapshotID string
// Offset is the chunk-aligned ENCODED offset to resume from, 0 for a
// fresh spool.
Offset int64
// Refused is zero, or why the offer is refused.
Refused errs.Code
// Held reports that the receiver already holds the snapshot: an
// idempotent success.
Held bool
}
InstallAcceptValue is the receiver's answer to an offer. The RECEIVER chooses the offset to resume from, from what it has durably spooled, so a resumed stream never appends to bytes the receiver did not keep (D7, R8-21).
type InstallOfferValue ¶ added in v0.3.0
type InstallOfferValue struct {
// Term is the sender's term.
Term uint64
// SnapshotID equals Meta.ID.
SnapshotID string
// Meta describes the snapshot.
Meta SnapshotMetaValue
}
InstallOfferValue is what a leader sends before streaming a snapshot: the whole metadata first, so the receiver refuses a snapshot too large, of another version or already covered before a single byte of it is spooled (D7).
type InstallParams ¶ added in v0.3.0
type InstallParams struct {
// Ctx ends with the connection.
Ctx context.Context
// From is the peer TLS authenticated.
From AuthenticatedPeerValue
// Offer is the accepted offer.
Offer InstallOfferValue
// Offset is the accepted offset.
Offset int64
// Body is the encoded stream from Offset, ending with the connection.
Body io.Reader
}
InstallParams is what the transport hands the engine once bytes flow, after the offer was accepted. Body ends with the connection: a stream cut short is a short read, and the spool keeps its last whole chunk for the next offer.
type JoinAuthorizer ¶
type JoinAuthorizer interface {
// Authorize accepts or refuses the request.
Authorize(ctx context.Context, r JoinRequestValue) (Operator, error)
}
JoinAuthorizer is the PRODUCT's port: a member calls it before it turns a JOIN into an AddLearner. It decides whether the slot's previous member is fenced, which only the product can know, and returns the Operator to record.
type JoinConfig ¶
type JoinConfig struct {
// Addresses are the members to try, in order.
Addresses []string
// Stamp is the joiner's stamp, from Init.
Stamp StampValue
// Address is where the joiner listens.
Address string
// Pins are the SPKIs of the members the joiner trusts, from the
// operator. An answer counts only from a peer presenting one of them,
// and only when the membership it returns lists that peer under that
// key; an empty list refuses the JOIN.
Pins [][32]byte
// MaxSnapshotBytes is the joiner's own MaxSnapshotBytes, announced in
// the JOIN so the members hold their cuts to it (R2-30); 0 announces
// nothing, and the JOIN is the one a member that predates the
// announcement reads.
MaxSnapshotBytes int64
}
JoinConfig is what a joining node needs to send its JOIN: where to reach the members, and the stamp Init wrote, whose id and incarnation the member it reaches proposes as a learner.
type JoinReasonBinder ¶
type JoinReasonBinder interface {
// BindJoinReason registers the group's JOIN callback that also returns
// the refusal's reason, empty for none.
BindJoinReason(group GroupID, join func(ctx context.Context, from AuthenticatedPeerValue, r JoinRequestValue) (PositionValue, errs.Code, string))
}
JoinReasonBinder is the sibling of a transport that carries the reason of a refused JOIN back to the joiner. A node that finds it binds this join callback in place of Bind's: the reason is what the JoinAuthorizer chose to say, sanitized and bounded, or the engine's own refusal, never more.
type JoinRequestValue ¶ added in v0.3.0
type JoinRequestValue struct {
// Cluster is the cluster the joiner claims.
Cluster ClusterID
// ID is the joiner's new NodeID.
ID NodeID
// Incarnation is the joiner's incarnation.
Incarnation Incarnation
// Slot is the slot its certificate names.
Slot string
// SPKI is its key, from the handshake.
SPKI [32]byte
// Address is where it listens.
Address string
// MaxSnapshotBytes is the largest snapshot the joiner accepts, as it
// announced it; 0 when it announced none, as a joiner that predates
// the announcement does.
MaxSnapshotBytes int64
}
JoinRequestValue is a pre-admission JOIN as a member received it: the slot and the key from TLS, the new node's id and incarnation from its frame.
type Joiner ¶
type Joiner interface {
// Join sends the JOIN and returns the position its AddLearner was
// appended at.
Join(ctx context.Context, cfg JoinConfig) (PositionValue, error)
}
Joiner is the sibling a transport implements to carry a pre-admission JOIN.
type LeadershipWatcher ¶
type LeadershipWatcher interface {
// Watch returns its OWN channel, of capacity one, holding the latest
// status; it is unsubscribed and closed when ctx ends or the node closes.
Watch(ctx context.Context) <-chan StatusValue
}
LeadershipWatcher is the sibling of a node that notifies leadership changes.
type LogIdentityValue ¶ added in v0.3.0
type LogIdentityValue struct {
// Cluster is the cluster the log belongs to.
Cluster ClusterID
// Epoch counts the histories of this cluster id: 1 at founding.
Epoch uint64
// Predecessor is the cluster of the history this one was recovered from,
// zero for a founding.
Predecessor ClusterID
// PredecessorEpoch is the predecessor's epoch.
PredecessorEpoch uint64
// KeptIndex is the last index Recover kept from the predecessor. An
// applied position of the predecessor at or below it is still trusted
// when its term is the one kept there: Recover adopts the verified
// suffix past the commit the survivor persisted, which may hold
// another term than the write the old cluster acknowledged.
// Every node of the history applies the entries at or below it under the
// predecessor's token: one admitted after the Recover learns it from the
// snapshot it installs, never from an append.
KeptIndex uint64
// AdoptedFrom is the last index of the kept range the predecessor had
// committed when Recover read it: the commit the survivor persisted,
// or the snapshot a restore started from. An entry at or below it is
// the one every node of the predecessor holds (Log Matching), so a
// predecessor's token at or below it is trusted by its index once its
// term can no longer be read; the adopted range above it, up to
// KeptIndex, is not. 0 records nothing, and nothing is trusted that
// way. It is recorded only in a cluster at protocol version 2, in the
// stamp of the node Recover or RestoreSnapshot ran on: a snapshot does
// not carry it, so a node that learns the lineage from one trusts
// nothing that way (#509).
AdoptedFrom uint64
}
LogIdentityValue scopes positions: the ClusterID and a log epoch that Recover and RestoreSnapshot bump, and the predecessor this history descends from with the last index it kept. A position or a token from another identity is not comparable with this log's.
func (LogIdentityValue) Same ¶ added in v0.3.0
func (l LogIdentityValue) Same(o LogIdentityValue) bool
Same reports whether two identities name the same history (cluster and epoch), whatever lineage they record.
type LogReader ¶
type LogReader interface {
// Entries returns entries in [lo, hi), at most maxBytes, and always at
// least one entry when lo < hi, even above maxBytes. lo below first is
// COMPACTED.
Entries(lo, hi uint64, maxBytes int, dst []EntryValue) ([]EntryValue, error)
// Term answers for first-1 too, from the term kept at compaction.
Term(index uint64) (uint64, error)
// Bounds of an empty log after a snapshot at s are (s+1, s).
Bounds() (first, last uint64)
}
LogReader reads the log. Expected outcomes are answers, not failures: an index below the first retained one is COMPACTED.
type LogStore ¶
type LogStore interface {
LogReader
// Append buffers entries and, when hs is not nil, a hard state, in ONE
// record batch.
Append(entries []EntryValue, hs *HardState) error
// Sync makes everything appended so far durable.
Sync() error
// TruncateSuffix writes ONE durable truncate record removing every entry
// from index from on, and nothing else.
TruncateSuffix(from uint64) error
// CompactPrefix drops what a snapshot covers, through index through.
CompactPrefix(through uint64) error
// HardState returns the last hard state appended.
HardState() (HardState, error)
// Stamp returns the directory's stamp.
Stamp() (StampValue, error)
// WriteStamp writes the stamp, durable on return.
WriteStamp(StampValue) error
// Close releases the store.
Close() error
}
LogStore is the durable log. Term, vote and entries travel in one Append and become durable under one Sync; readers see appended entries before they are synced. One writer, many readers. When Sync and WriteStamp return nil, what they cover survives a power loss: the safety of the whole domain rests on it. FROZEN under ADR 0039.
type MemberValue ¶ added in v0.3.0
type MemberValue struct {
// ID is the node.
ID NodeID
// Incarnation is the node's data-directory incarnation.
Incarnation Incarnation
// Slot is the placement slot its certificate names.
Slot string
// SPKI is the SHA-256 of its certificate's SubjectPublicKeyInfo. A zero
// pin admits nobody: it is what DenyPin leaves.
SPKI [32]byte
// Address is where its transport listens.
Address string
// Voter is true for a voter, false for a learner.
Voter bool
// Joining marks an empty learner admitted before it holds any log.
Joining bool
// PayloadMax is the highest payload version its state machine supports; 0
// means only the lowest.
PayloadMax uint32
// MaxSnapshotBytes is the largest snapshot it accepts, 0 when unknown.
MaxSnapshotBytes int64
}
MemberValue is one node of the membership: its identity, the key it is pinned to, where it listens, and whether it votes.
type MembershipValue ¶ added in v0.3.0
type MembershipValue struct {
// Index is the index of the entry that produced it, 0 at founding.
Index uint64
// Members lists every voter and learner.
Members []MemberValue
// Retired lists every removed NodeID.
Retired []NodeID
// RetiredKeys lists the SPKI pin of every member removed, and every pin
// DenyPin withdrew, since the cluster version is 3: a key held once is
// never admitted again under another id. Empty below version 3.
RetiredKeys [][32]byte
}
MembershipValue is the configuration at a position: its members, the set of retired IDs, which is never reused, and from protocol 3 the set of retired keys, which is never admitted again.
func (MembershipValue) Clone ¶ added in v0.3.0
func (m MembershipValue) Clone() MembershipValue
Clone returns a deep copy, so that a membership handed out cannot be changed under the engine.
func (MembershipValue) IsRetired ¶ added in v0.3.0
func (m MembershipValue) IsRetired(id NodeID) bool
IsRetired reports whether the id was removed.
func (MembershipValue) IsRetiredKey ¶ added in v0.26.17
func (m MembershipValue) IsRetiredKey(spki [32]byte) bool
IsRetiredKey reports whether the key was retired with a member removed, or with a pin denied.
func (MembershipValue) Member ¶ added in v0.3.0
func (m MembershipValue) Member(id NodeID) (MemberValue, bool)
Member returns the member with that id.
func (MembershipValue) Voters ¶ added in v0.3.0
func (m MembershipValue) Voters() []NodeID
Voters returns the ids of the voters, in membership order.
type MessageFlags ¶
type MessageFlags uint8
MessageFlags are a message's boolean qualifiers.
const ( // FlagTransfer marks a vote request of a leadership transfer. It lifts the // live-leader refusal for ONE campaign, never the election restriction. FlagTransfer MessageFlags = 1 << iota // FlagNeedSnapshot marks an append rejection from a node that holds no // usable snapshot and asks for one. FlagNeedSnapshot // FlagNoLease marks a read the follower wants confirmed by a round, never // by a lease. FlagNoLease // FlagTruncated marks an append rejection from a node that truncated // acknowledged entries through TruncateAndStaySilent: the leader lowers // its match to the node's log, the one exception to "match only grows". FlagTruncated // FlagProtocol2 marks a frame from a node that speaks protocol version // 2. It is the announcement D18 asks for: a node of protocol 1 tests // the bits it names only and ignores it, and a leader refuses to raise // the cluster version to 2 while a member's last frame of the term // lacks it. FlagProtocol2 // FlagProtocol3 marks a frame from a node that speaks protocol version // 3, which sets FlagProtocol2 beside it. A node of protocol 2 or below // tests the bits it names only and ignores it, and a leader refuses to // raise the cluster version to 3 while a member's last frame of the term // lacks it. FlagProtocol3 )
The flags.
type MessageKind ¶
type MessageKind uint8
MessageKind says what a Message is. The two high bits are its lane, so that the frame header alone routes it (ADR 0152 D18).
const ( // MsgPreVote asks whether the sender COULD win at Term, without raising // anybody's term. MsgPreVote MessageKind = 0x01 // MsgPreVoteResp answers a MsgPreVote. MsgPreVoteResp MessageKind = 0x02 // MsgVote asks for a vote at Term. MsgVote MessageKind = 0x03 // MsgVoteResp answers a MsgVote. MsgVoteResp MessageKind = 0x04 // MsgHeartbeat confirms the leader and carries a commit index clamped to // the follower's match. MsgHeartbeat MessageKind = 0x05 // MsgHeartbeatResp answers a heartbeat and carries the applied contract // generation. MsgHeartbeatResp MessageKind = 0x06 // MsgTimeoutNow tells the transfer target to campaign at once. MsgTimeoutNow MessageKind = 0x07 // MsgReadIndex asks the leader for a confirmed read index. MsgReadIndex MessageKind = 0x08 // MsgReadIndexResp answers a MsgReadIndex. MsgReadIndexResp MessageKind = 0x09 // MsgForwardAdmin forwards an administrative operation to the leader. MsgForwardAdmin MessageKind = 0x0A // MsgForwardAdminResp answers a MsgForwardAdmin. MsgForwardAdminResp MessageKind = 0x0B // MsgLeaderRecord, of protocol version 2, tells a member that campaigns // behind the sender's commit the member record of the leader the sender // follows, read from the sender's committed configuration: a voter that // was down while that leader was admitted pins it provisionally, for // replication only, and learns its configuration from it (ADR 0152 D4, // #542). Its one entry's data is that record, as AppendMembership // encodes a configuration of one member. MsgLeaderRecord MessageKind = 0x0C // MsgApp appends entries after (LogTerm, Index). MsgApp MessageKind = 0x41 // MsgAppResp acknowledges, or refuses, a MsgApp. MsgAppResp MessageKind = 0x42 // MsgForward forwards a proposal to the leader under a submission id. MsgForward MessageKind = 0x43 // MsgForwardResp answers a MsgForward with the position it appended at. MsgForwardResp MessageKind = 0x44 )
The message kinds of protocol version 1.
func (MessageKind) Known ¶
func (k MessageKind) Known() bool
Known reports whether protocol version 1 or 2 defines the kind.
func (MessageKind) Lane ¶
func (k MessageKind) Lane() FrameLane
Lane returns the connection the kind belongs on.
type MessageValue ¶ added in v0.3.0
type MessageValue struct {
// Version is the frame layout version, 1.
Version uint8
// Group is the consensus group, 1 in v1.
Group GroupID
// Kind says what the message is.
Kind MessageKind
// Flags qualify it.
Flags MessageFlags
// From is the sender, as the sender wrote it.
From NodeID
// To is the receiver.
To NodeID
// Term is the sender's term.
Term uint64
// LogTerm is the term of the entry at Index: the previous entry of an
// append, the last entry of a vote request.
LogTerm uint64
// Index is the previous index of an append, the last index of a vote
// request, the verified prefix of an acknowledgement, a read index.
Index uint64
// Commit is the commit index the sender lets the receiver advance to.
Commit uint64
// Reject marks a refusal.
Reject bool
// RejectHint is the receiver's last index, a hint where to probe from.
RejectHint uint64
// Context correlates a reply with its request: a heartbeat round, a read
// request, a submission sequence.
Context uint64
// TransferIssuer is the leader that issued a transfer flag.
TransferIssuer NodeID
// TransferTerm is the term the transfer flag was issued in.
TransferTerm uint64
// Contract is the committed contract generation the sender has applied.
Contract uint64
// Code is a refusal's error code, 0 for none.
Code errs.Code
// Entries are the entries an append carries, or the proposal a forward
// carries with a zero position.
Entries []EntryValue
}
MessageValue is one protocol message: a plain value, encoded by the service's AppendMessage and decoded by ParseMessage, never by a method. The engine trusts the transport's authenticated peer over From, and drops a message whose From differs from it.
type NodeID ¶
type NodeID uint64
NodeID names one member of a group for its whole life. It is a random 64-bit value drawn at Init, never 0 and never reused: a node whose disk was lost rejoins under a NEW id, because one that rejoined under its old id could vote twice in one term.
type Operator ¶
type Operator string
Operator is the opaque identity an administrative operation is performed under. The product vouches for it and the SDK records it in the entry, never authenticating it. It is printable ASCII 0x21-0x7E, at most 128 bytes, and must not carry personal data: the log is replicated and kept.
type PositionValue ¶ added in v0.3.0
type PositionValue struct {
// Term is the leader term the entry was appended in.
Term uint64
// Index is the entry's place in the log, from 1.
Index uint64
}
PositionValue is an entry's place in the log: the term it was created in and its index. The index is a fencing token: unique, strictly increasing over the group's life and identical on every node. The term says WHICH command, never who may act on it now (ADR 0152 D10).
type PreconditionSpec ¶ added in v0.3.0
type PreconditionSpec struct {
// Log is the identity the precondition was read in.
Log LogIdentityValue
// Alarm is the alarm a clearing operation names.
Alarm AlarmID
// Generation is the contract or cluster-version generation an operation
// expects to replace.
Generation uint64
}
PreconditionSpec is what a clearing or raising operation expects to act on: the log identity it was read in, and the alarm or the contract or cluster-version generation it expects to replace. Apply refuses a mismatch with ADMIN_PRECONDITION_FAILED, so a stale page cannot clear an alarm raised since.
type Replicated ¶
type Replicated interface {
// Propose replicates data and returns, once the entry is applied
// locally, its lineage-scoped token and the value Apply returned.
// NOT_LEADER means certainly not appended; OUTCOME_UNKNOWN means it may
// still commit, and the caller must read the state before deciding.
Propose(ctx context.Context, data []byte) (TokenValue, any, error)
// Barrier returns once the local state machine reflects every write
// committed before the call, with the token of the applied index.
Barrier(ctx context.Context) (TokenValue, error)
// WaitApplied returns once the local node has applied t, the entry of
// t's term at t's index. A token of another lineage is
// LINEAGE_MISMATCH. A token whose index holds an entry of another term
// is ENTRY_SUPERSEDED, or LINEAGE_MISMATCH when it is one of the
// predecessor's within the index Recover kept, as is a token of the
// predecessor's whose term the node can no longer read there.
WaitApplied(ctx context.Context, t TokenValue) error
// Status returns the node's view after its last turn.
Status() StatusValue
}
Replicated is the consumer's port, and the only algorithm-neutral one: propose a command, wait for a linearizable read point, wait for a position, read the node's status. FROZEN under ADR 0039.
type Role ¶
type Role uint8
Role is a node's part in its group.
const ( // RoleFollower follows a leader, or waits for one. RoleFollower Role = iota + 1 // RolePreCandidate asks whether it COULD win, without raising its term. RolePreCandidate // RoleCandidate campaigns at a raised term. RoleCandidate // RoleLeader leads its term. RoleLeader // RoleLearner receives the log and votes when asked, but never campaigns. RoleLearner )
The roles.
type Snapshot ¶
type Snapshot interface {
// WriteTo writes the view's state.
WriteTo(w io.Writer) (int64, error)
// Release frees the view.
Release()
}
Snapshot is a point-in-time view a StateMachine captured. WriteTo runs on the snapshot goroutine while Apply continues; Release frees the view.
type SnapshotHandle ¶
type SnapshotHandle interface {
io.ReaderAt
// Meta describes the snapshot.
Meta() SnapshotMetaValue
// Close unpins it.
Close() error
}
SnapshotHandle is a published snapshot, pinned until Close.
type SnapshotMetaValue ¶ added in v0.3.0
type SnapshotMetaValue struct {
// ID names the snapshot, the same in a spool, an offer and a handle.
ID string
// Version is the snapshot format version, 1.
Version uint8
// Position is the last entry the snapshot covers.
Position PositionValue
// Log is the identity of the log the snapshot was cut from.
Log LogIdentityValue
// Membership is the configuration valid at Position, never a later one.
Membership MembershipValue
// ChainTip is the authenticated chain value at Position: the chain is
// carried across the boundary, never reseeded.
ChainTip uint32
// Codec names the compression of the chunks: "flate" or "gzip".
Codec string
// EncodedSize is the size of the encoded stream, chunk headers included.
EncodedSize int64
// DecodedSize is the size of the decoded stream.
DecodedSize int64
// ChunkSize is the largest decoded chunk.
ChunkSize int64
// SHA256 is the digest of the DECODED stream, computed by the engine.
SHA256 [32]byte
// Timing is the committed timing contract at Position.
Timing TimingContractSpec
// ClusterVersion is the committed cluster version at Position.
ClusterVersion uint32
// ClusterGeneration counts its raises.
ClusterGeneration uint64
// NoSpace is the no-space alarm raised at Position, 0 for none.
NoSpace AlarmID
// Reserved is carried uninterpreted for ADR 0173.
Reserved []byte
}
SnapshotMetaValue describes one snapshot: where it cuts the log, the engine state it carries, and what its bytes must hash to. It is versioned; a version a node does not know is refused cleanly.
type SnapshotSink ¶
type SnapshotSink interface {
io.Writer
// Commit publishes the snapshot; final carries the SHA-256 the engine
// computed over the DECODED stream.
Commit(final SnapshotMetaValue) error
// Abort drops the spool.
Abort() error
}
SnapshotSink receives the ENCODED chunks of one snapshot and checks each chunk's integrity as it arrives.
type SnapshotStore ¶
type SnapshotStore interface {
// Create opens the spool of id. When a spool of that id exists, it is
// returned with the ENCODED offset durably spooled so far.
Create(id string, meta SnapshotMetaValue) (sink SnapshotSink, spooled int64, err error)
// List returns the complete snapshots, newest first.
List() ([]SnapshotMetaValue, error)
// Open pins a snapshot until the handle is closed.
Open(id string) (SnapshotHandle, error)
// Remove deletes a snapshot; refused while a handle is open.
Remove(id string) error
}
SnapshotStore holds snapshots. FROZEN under ADR 0039.
type StampState ¶
type StampState uint8
StampState says how far a data directory has come.
const ( // StampJoining is a directory Init prepared: it may join as an empty // learner, never vote or acknowledge until it is admitted. StampJoining StampState = iota + 1 // StampBootstrapPending is a founding in progress: Bootstrap wrote it // before it wrote the founding configuration. StampBootstrapPending // StampMember is a directory that belongs to a cluster. StampMember )
The states of a stamp.
type StampValue ¶ added in v0.3.0
type StampValue struct {
// Log is the log identity the directory belongs to.
Log LogIdentityValue
// ID is the node.
ID NodeID
// Incarnation is drawn when the stamp is first written.
Incarnation Incarnation
// Slot is the placement slot the node's certificate names.
Slot string
// State is how far the directory has come.
State StampState
// Founding is the digest of the founding member list while a bootstrap
// is pending, so that a rerun with another list is refused.
Founding [32]byte
}
StampValue binds a data directory to one node of one cluster: written by LogStore.WriteStamp, durable on return, and verified at every start. A mismatch is DATA_DIR_MISMATCH.
type StateMachine ¶
type StateMachine interface {
// Apply applies one committed command and returns its result, delivered
// to the Propose that created it on the node that created it.
Apply(e EntryValue) any
// Snapshot captures a point-in-time view of the state, cheaply: it is
// called on the apply goroutine.
Snapshot() (Snapshot, error)
// Restore replaces the state with a snapshot's. It is serialised with
// Apply. An error coded VERSION_UNSUPPORTED says the snapshot is of a
// format this build does not read: the node stops, as for an entry
// kind it does not know, and a restart over that snapshot is refused
// (D18). Any other error refuses that one install, and the leader
// offers a snapshot again.
Restore(r io.Reader, meta SnapshotMetaValue) error
}
StateMachine is the caller's replicated program. Apply is a PURE transition of state the state machine owns: no side effect outside that state, no precondition that can differ between nodes. It is called for every committed command, in log order, on one goroutine, and must be deterministic. A refusal is an answer carried in the result; an Apply that cannot complete must panic, and the node stops with STATE_MACHINE_FAILED — the entry IS committed. FROZEN under ADR 0039.
type StatusValue ¶ added in v0.3.0
type StatusValue struct {
// ID is this node.
ID NodeID
// Role is its role.
Role Role
// Term is its current term.
Term uint64
// Leader is the leader it knows of in Term, 0 for none.
Leader NodeID
// Commit is the highest index it knows committed.
Commit uint64
// Applied is the highest index its state machine applied.
Applied uint64
// Log is the identity of its log.
Log LogIdentityValue
// Versions are its capabilities and the committed cluster version.
Versions VersionsValue
// Timing is the committed timing contract.
Timing TimingContractSpec
// NoSpace is the raised no-space alarm, 0 when none.
NoSpace AlarmID
}
StatusValue is a node's view of its group after one turn of its event loop. It is a copy taken on the loop, never a live view, so every field in it describes the same instant.
func (*StatusValue) Precondition ¶ added in v0.3.0
func (s *StatusValue) Precondition() PreconditionSpec
Precondition returns the precondition an operation clearing the current alarm, or replacing the current contract, would carry.
type StopReasonValue ¶ added in v0.3.0
type StopReasonValue struct {
// Err is the stop's error, built on its sentinel.
Err error
// Unmarked is set when a STORAGE_FAILED stop could write its marker
// nowhere: the exit code is then 73.
Unmarked bool
}
StopReasonValue says why a node stopped itself. Err carries the sentinel of the stop; ExitCode is the process exit status the product should use.
func (StopReasonValue) ExitCode ¶ added in v0.3.0
func (s StopReasonValue) ExitCode() int
ExitCode returns errs.ExitCodeOf of the stop's error, or 73 for a storage failure whose marker could not be written (ADR 0152 D4).
type TimingContractSpec ¶ added in v0.3.0
type TimingContractSpec struct {
// Generation counts the committed contracts, 1 at founding.
Generation uint64
// HeartbeatInterval is how often the leader confirms itself.
HeartbeatInterval time.Duration
// ElectionTimeoutMin is the lower bound of the randomized election
// timeout, and the vote-refusal window.
ElectionTimeoutMin time.Duration
// ElectionTimeoutMax is its upper bound, exclusive.
ElectionTimeoutMax time.Duration
// MaxClockDrift is the largest rate difference between two clocks, as a
// fraction in (0, 1). Zero refuses lease reads.
MaxClockDrift float64
}
TimingContractSpec is the set of timings every member must agree on for leases and vote refusals to be sound. The committed contract entry is its ONLY authority; a node's own configuration is merely the proposal used at Bootstrap.
type TokenValue ¶ added in v0.3.0
type TokenValue struct {
// Cluster is the cluster of the log that committed the entry.
Cluster ClusterID
// Epoch is that log's epoch.
Epoch uint64
// Term is the entry's term.
Term uint64
// Index is the entry's index.
Index uint64
}
TokenValue is a lineage-scoped position: the cluster, the log epoch, the term and the index. It is comparable, and its encoding is fixed at 41 bytes: a version byte, the 16-byte ClusterID, then epoch, term and index as big-endian uint64 (see AppendToken in the service's wire package).
func (TokenValue) Position ¶ added in v0.3.0
func (t TokenValue) Position() PositionValue
Position returns the token's position, without its lineage.
type Transport ¶
type Transport interface {
// Send hands over an encoded frame. It never blocks and may drop; the
// transport owns frame after the call.
Send(to NodeID, frame []byte)
// Stream sends a snapshot from the offset the receiver accepted; open
// returns a reader at an encoded offset on a chunk boundary, which
// Stream closes.
Stream(ctx context.Context, to NodeID, offer InstallOfferValue, open func(offset int64) (io.ReadCloser, error)) error
// Bind registers the group's callbacks. deliver gets the frame and the
// peer TLS authenticated, valid until deliver returns; offer is judged
// on the event loop and its spool then positioned off it, and the
// shipped transport reads its answer under JoinTimeout; join is
// answered off the loop, under a deadline.
Bind(group GroupID, deliver func(from AuthenticatedPeerValue, frame []byte), offer func(from AuthenticatedPeerValue, o InstallOfferValue) InstallAcceptValue, install func(InstallParams) error, join func(ctx context.Context, from AuthenticatedPeerValue, r JoinRequestValue) (PositionValue, errs.Code)) error
// SetMembership pushes the group's admitted members; the transport
// computes the union over its groups.
SetMembership(group GroupID, admitted []MemberValue)
}
Transport carries ENCODED frames between the nodes of a group. FROZEN under ADR 0039.
type UnreachableReporter ¶
type UnreachableReporter interface {
// Unreachable registers the callback.
Unreachable(func(NodeID))
}
UnreachableReporter is the sibling of a transport that reports a peer it could not reach, so that the leader moves its progress to probe.
type VersionsValue ¶ added in v0.3.0
type VersionsValue struct {
// ProtocolMin is the lowest protocol version it speaks.
ProtocolMin uint32
// ProtocolMax is the highest.
ProtocolMax uint32
// PayloadMax is the highest payload version it applies.
PayloadMax uint32
// Cluster is the committed cluster version.
Cluster uint32
// ClusterGeneration counts cluster-version raises.
ClusterGeneration uint64
}
VersionsValue are a node's capabilities: the protocol range it speaks, the payload versions its state machine supports, and the committed cluster version.