Documentation
¶
Overview ¶
Package cluster holds the shared domain types for go-faster/fs cluster mode: the failure-domain topology (rack → server → disk) that the etcd control plane publishes and the placement function consumes. It has no external dependencies and no I/O — the etcd integration and the replication transport live in sibling packages.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Disk ¶
type Disk struct {
// ID identifies the disk within its node.
ID DiskID
// Weight is the relative capacity weight used by weighted placement. A disk
// with weight <= 0 is drained: it is never chosen for new placement (used
// when decommissioning or when a disk is full/faulted).
Weight float64
// TotalBytes/FreeBytes are the disk's reported filesystem capacity,
// refreshed by its node's registration; 0/0 means unknown. Observability
// input only — placement uses Weight, and Signature ignores capacity so
// usage churn never reads as a membership change.
TotalBytes uint64
FreeBytes uint64
}
Disk is one storage device on a node.
type DiskID ¶
type DiskID string
DiskID uniquely identifies a disk. It only needs to be unique within its node; placement treats (NodeID, DiskID) as the physical target.
type Node ¶
type Node struct {
// ID identifies the node.
ID NodeID
// Addr is the node's cluster-listener address (host:port). Placement does
// not use it; it is carried for the caller that dispatches to targets.
Addr string
// Rack is the failure-domain label. Nodes sharing a rack share fate (power,
// switch); an empty rack means "unknown", treated as its own single domain
// keyed by node so such nodes never falsely appear rack-diverse.
Rack string
// Disks are the node's storage devices.
Disks []Disk
}
Node is a cluster member and the disks it exposes. Rack is its coarsest failure domain; placement spreads copies across distinct racks first, then distinct nodes, then distinct disks.
func (Node) FailureDomain ¶
FailureDomain returns the node's coarsest failure-domain key: its rack, or a node-specific key when the rack is unset so unlabeled nodes are never treated as sharing a rack (they degrade to node-level spread instead of falsely appearing rack-diverse).
type ScrubState ¶ added in v0.12.0
type ScrubState struct {
// Cursor is the last object namespace fully processed, in the walk's
// lexicographic order. Empty means no pass is in flight: the next one
// starts from the beginning.
Cursor string
// PassStarted is when the in-flight pass began.
PassStarted time.Time
// LastCompleted is when a pass over this disk last ran to the end. It is
// the honest measure of scrub coverage — how long ago every object on this
// disk was last verified — and the one number that reveals a cycle failing
// to keep up, which counters of work done cannot.
LastCompleted time.Time
}
ScrubState is one disk's scrub progress, durable across restarts.
A deep scrub reads every payload on the disk, so a pass over a large disk runs for hours or days. Without somewhere to record how far it got, a restart — a rolling upgrade, an OOM, a crash — throws the whole pass away and the next one starts at the beginning. A node restarted more often than a sweep takes then verifies the front of each disk over and over and never reaches the back, while every counter says the scrubber is working.
It lives here, with the other domain types, so the store that persists it and the repairer that keeps it need not know about each other.
func (ScrubState) InProgress ¶ added in v0.12.0
func (s ScrubState) InProgress() bool
InProgress reports whether a pass was interrupted and has somewhere to resume from.
type Topology ¶
type Topology struct {
// Epoch is the monotonically increasing version of this snapshot (the etcd
// revision it was derived from). Placement output is stable within an epoch.
Epoch uint64
// Nodes are the members. Order is not significant to placement.
Nodes []Node
}
Topology is an immutable snapshot of cluster membership at a given placement epoch. It is produced by the control plane (etcd revision → epoch) and is a pure input to placement; callers must not mutate a Topology after publishing it.
func (*Topology) DiskCount ¶
DiskCount reports the number of disks across all nodes, including drained ones.
func (*Topology) Signature ¶
Signature is a stable digest of everything placement depends on: node IDs, racks, disk IDs and weights — sorted, so registration order does not matter. Addresses and the epoch are excluded: a node re-registering at a new address (or any registry churn that bumps the epoch without changing membership) moves no data and must not read as a placement change. Equal signatures ⇒ identical placement for every object.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package diskstore is the filesystem-backed fragment store for go-faster/fs cluster mode: the durable transport.Store a production node serves its fragments from, one root directory per disk.
|
Package diskstore is the filesystem-backed fragment store for go-faster/fs cluster mode: the durable transport.Store a production node serves its fragments from, one root directory per disk. |
|
Package etcd is the go-faster/fs cluster control plane: the only component that talks to etcd.
|
Package etcd is the go-faster/fs cluster control plane: the only component that talks to etcd. |
|
Package fragment is the connective layer between placement and the erasure schemes: it encodes an object into the per-target fragments a scheme stores, and reconstructs the object from whatever fragments survive.
|
Package fragment is the connective layer between placement and the erasure schemes: it encodes an object into the per-target fragments a scheme stores, and reconstructs the object from whatever fragments survive. |
|
Package placement is the pure, failure-domain-aware placement function for go-faster/fs cluster mode (DESIGN.md FR-16).
|
Package placement is the pure, failure-domain-aware placement function for go-faster/fs cluster mode (DESIGN.md FR-16). |
|
Package scheme models the go-faster/fs replication and erasure schemes (DESIGN.md FR-17) and implements the pure codecs they need.
|
Package scheme models the go-faster/fs replication and erasure schemes (DESIGN.md FR-17) and implements the pure codecs they need. |
|
Package transport is the peer replication transport for go-faster/fs cluster mode: the internal HTTP API nodes use to store, fetch and delete object fragments on each other (the payloads produced by internal/cluster/fragment).
|
Package transport is the peer replication transport for go-faster/fs cluster mode: the internal HTTP API nodes use to store, fetch and delete object fragments on each other (the payloads produced by internal/cluster/fragment). |