Documentation
¶
Overview ¶
Package replication provides distributed replication for NornicDB.
This package implements a Cassandra-style shared-nothing architecture with Raft consensus for strong consistency. It supports multiple deployment modes:
- Standalone: Single node, no replication (default)
- HAStandby: Hot standby with automatic failover (2 nodes)
- Raft: Raft consensus cluster (3+ nodes)
- MultiRegion: Raft clusters with cross-region async replication
The design follows these principles:
- Shared-nothing: Each node owns its data, no shared storage
- Tunable consistency: Choose between AP (fast) and CP (safe) per query
- Linear scalability: Add nodes for increased capacity
- Operational simplicity: Minimal configuration, sensible defaults
Environment Variables (NORNICDB_CLUSTER_*):
NORNICDB_CLUSTER_MODE=standalone|ha_standby|raft|multi_region NORNICDB_CLUSTER_NODE_ID=node-1 NORNICDB_CLUSTER_BIND_ADDR=0.0.0.0:7000 NORNICDB_CLUSTER_ADVERTISE_ADDR=192.168.1.10:7000 NORNICDB_CLUSTER_REPLICATION_SECRET=your-shared-secret-min-32-chars
Hot Standby:
NORNICDB_CLUSTER_HA_ROLE=primary|standby NORNICDB_CLUSTER_HA_PEER_ADDR=standby-host:7000 NORNICDB_CLUSTER_HA_SYNC_MODE=async|quorum NORNICDB_CLUSTER_HA_HEARTBEAT_MS=1000 NORNICDB_CLUSTER_HA_FAILOVER_TIMEOUT=30s NORNICDB_CLUSTER_HA_AUTO_FAILOVER=true
Raft Cluster:
NORNICDB_CLUSTER_RAFT_PEERS=node-2:7000,node-3:7000 NORNICDB_CLUSTER_RAFT_BOOTSTRAP=true NORNICDB_CLUSTER_RAFT_ELECTION_TIMEOUT=1s NORNICDB_CLUSTER_RAFT_HEARTBEAT_TIMEOUT=100ms NORNICDB_CLUSTER_RAFT_SNAPSHOT_INTERVAL=300 NORNICDB_CLUSTER_RAFT_SNAPSHOT_THRESHOLD=10000
Multi-Region:
NORNICDB_CLUSTER_REGION_ID=us-east NORNICDB_CLUSTER_REMOTE_REGIONS=eu-west:coord1:7000,ap-south:coord2:7000 NORNICDB_CLUSTER_CROSS_REGION_MODE=async|quorum
Example Usage:
// Load config from environment
config := replication.LoadFromEnv()
// Create replicator based on mode
replicator, err := replication.NewReplicator(config, storage)
if err != nil {
log.Fatal(err)
}
// Start replication
if err := replicator.Start(ctx); err != nil {
log.Fatal(err)
}
// Apply writes through replicator
if err := replicator.Apply(cmd); err != nil {
if errors.Is(err, replication.ErrNotLeader) {
// Forward to leader
leaderAddr := replicator.LeaderAddr()
}
}
ELI12 (Explain Like I'm 12):
Imagine you have a diary that you want to keep safe:
**Standalone**: You have one diary. Simple, but if you lose it, everything's gone.
**Hot Standby**: You write in your diary, and your friend copies everything you write into their diary. If you're sick, they can take over.
**Raft Cluster**: You and 2 friends all have diaries. When you want to write something, everyone votes. If most agree, everyone writes the same thing. Even if one person is absent, the group still works.
**Multi-Region**: Like Raft, but your friends are in different cities. Within each city, the Raft voting happens. Between cities, changes are copied but not voted on (it would be too slow).
The "shared-nothing" part means each person has their own complete diary - no one shares pages with anyone else. This makes it easy to add more friends without things getting complicated.
Plan 04-06-04: D-05b stale-peer GC lifecycle.Component populating no metric — instead, periodically calling DeleteLabelValues on the per-peer GaugeVecs in *observability.ReplicationMetrics to evict stale peers (peers we have not observed within the staleness threshold).
Why a GC component (RESEARCH §Q7 / Pitfall 3):
- Per-peer label values accumulate on every reconnect — even when the peer label is stable per RISK-3, a misconfiguration that rotates PeerConfig.ID across restarts would explode cardinality without GC.
- Without DeleteLabelValues, GaugeVec series persist forever in the registry — a multi-region churn scenario could push the cardinality past D-05a ceilings.
- DeleteLabelValues races with concurrent observation per the prometheus client_golang documentation (the Bound observer cached by the replicator becomes stale after delete). The replicator's contract is to REBIND on reconnect — see replicator.go documentation.
Lifecycle semantics (mirrors pkg/storage.BytesMetricsSweeper — Plan 04-04 reference):
- Start spawns a goroutine that ticks every interval (default 5min).
- Each tick wraps `defer recover()` so any panic during DeleteLabelValues does not crash the supervisor errgroup (RISK-8 / T-04-08).
- Shutdown closes the ticker and waits for the goroutine to exit within the supervisor's drain budget (5s typical).
- Registered between pprof and workersC components per RESEARCH §Q4.
Plan 04-06-03: D-15a per-event observation chokepoint for the replicator.
This file holds the small "observability seam" that each Replicator implementation calls at the SAME sites that already emit role-transition log lines. No background polling — single-line additions at existing log sites per CONTEXT D-15a.
Why a thin observation helper instead of weaving metric calls through raft.go / ha_standby.go / multi_region.go directly:
- **DRY**: the Role/Term/Counter triplet at every leader-boundary transition is identical across implementations. Centralizing in `observeRoleTransition` keeps the pkg/observability dependency surface narrow.
- **Test seam**: TestRoleTransition_GaugeUpdate drives the helper directly without spinning up the full Raft state machine.
- **Nil-safe**: production code injects metrics via SetReplicatorMetrics; test fixtures that build a Replicator without metrics get a clean no-op (existing replication tests do not need to know about metrics).
**Pitfall 3 mitigation (RESEARCH §Q7)**: this helper does NOT cache per-peer Bound observers. Per-peer observation goes through `observePeerLag` / `observePeerRTT` / `observeLastContact` which call WithLabelValues every time AND Mark the tracker. The peer_metrics_gc component evicts stale series via DeleteLabelValues; a held Bound observer would point to a detached series after eviction. Re-binding on every observation site is the simple, correct contract.
Package replication provides distributed replication for NornicDB.
Package replication provides cluster replication for NornicDB.
Transport Architecture:
NornicDB cluster communication uses a hybrid approach:
1. **Client Bolt Protocol (default port 7687)** - Used for:
Neo4j driver compatibility for client queries
Writes can be sent to any node: if a write hits a follower, the cluster transport automatically forwards it to the leader (ForwardApply) and returns the leader's response, so clients do not need to route writes to the leader.
2. **Cluster Protocol (default port 7000)** - Used for:
- Raft consensus (RequestVote, AppendEntries)
- Write forwarding (ForwardApply from follower to leader)
- WAL streaming for HA standby
- Heartbeats and health checks
- Cluster coordination
This separation allows:
- Client-facing Bolt remains pure Neo4j compatible
- Cluster protocol is optimized for low-latency consensus
- Existing Bolt infrastructure reused where appropriate
Example Configuration:
NORNICDB_CLUSTER_MODE=raft NORNICDB_CLUSTER_BIND_ADDR=0.0.0.0:7000 NORNICDB_BOLT_PORT=7687
Index ¶
- Constants
- Variables
- func RegisterClusterHandlers(t *ClusterTransport, r Replicator)
- type AppendEntriesRequest
- type AppendEntriesResponse
- type ClusterConnection
- func (c *ClusterConnection) Close() error
- func (c *ClusterConnection) IsConnected() bool
- func (c *ClusterConnection) SendFence(ctx context.Context, req *FenceRequest) (*FenceResponse, error)
- func (c *ClusterConnection) SendForwardApply(ctx context.Context, cmd *Command, timeout time.Duration) error
- func (c *ClusterConnection) SendHeartbeat(ctx context.Context, req *HeartbeatRequest) (*HeartbeatResponse, error)
- func (c *ClusterConnection) SendPromote(ctx context.Context, req *PromoteRequest) (*PromoteResponse, error)
- func (c *ClusterConnection) SendRaftAppendEntries(ctx context.Context, req *RaftAppendEntriesRequest) (*RaftAppendEntriesResponse, error)
- func (c *ClusterConnection) SendRaftVote(ctx context.Context, req *RaftVoteRequest) (*RaftVoteResponse, error)
- func (c *ClusterConnection) SendWALBatch(ctx context.Context, entries []*WALEntry) (*WALBatchResponse, error)
- type ClusterMessage
- type ClusterMessageType
- type ClusterTransport
- func (t *ClusterTransport) Close() error
- func (t *ClusterTransport) Connect(ctx context.Context, addr string) (PeerConnection, error)
- func (t *ClusterTransport) Listen(ctx context.Context, addr string, handler ConnectionHandler) error
- func (t *ClusterTransport) RegisterHandler(msgType ClusterMessageType, handler MessageHandler)
- type ClusterTransportConfig
- type Command
- type CommandType
- type Config
- type ConnectionHandler
- type ConsistencyConfig
- type ConsistencyLevel
- type EdgePayload
- type FenceRequest
- type FenceResponse
- type HAStandbyConfig
- type HAStandbyReplicator
- func (r *HAStandbyReplicator) Apply(cmd *Command, timeout time.Duration) error
- func (r *HAStandbyReplicator) ApplyBatch(cmds []*Command, timeout time.Duration) error
- func (r *HAStandbyReplicator) HandleFence(req *FenceRequest) (*FenceResponse, error)
- func (r *HAStandbyReplicator) HandleHeartbeat(req *HeartbeatRequest) (*HeartbeatResponse, error)
- func (r *HAStandbyReplicator) HandlePromote(req *PromoteRequest) (*PromoteResponse, error)
- func (r *HAStandbyReplicator) HandleWALBatch(entries []*WALEntry) (*WALBatchResponse, error)
- func (r *HAStandbyReplicator) Health() *HealthStatus
- func (r *HAStandbyReplicator) IsLeader() bool
- func (r *HAStandbyReplicator) LeaderAddr() string
- func (r *HAStandbyReplicator) LeaderID() string
- func (r *HAStandbyReplicator) Mode() ReplicationMode
- func (r *HAStandbyReplicator) NodeID() string
- func (r *HAStandbyReplicator) Promote(ctx context.Context) error
- func (r *HAStandbyReplicator) SetReplicatorMetrics(bag *observability.ReplicationMetrics, tracker *PeerTracker)
- func (r *HAStandbyReplicator) SetTransport(t Transport)
- func (r *HAStandbyReplicator) Shutdown() error
- func (r *HAStandbyReplicator) Start(ctx context.Context) error
- func (r *HAStandbyReplicator) WaitForLeader(ctx context.Context) error
- type HealthStatus
- type HeartbeatRequest
- type HeartbeatResponse
- type MessageHandler
- type MetricsAware
- type MultiRegionConfig
- type MultiRegionReplicator
- func (r *MultiRegionReplicator) Apply(cmd *Command, timeout time.Duration) error
- func (r *MultiRegionReplicator) ApplyBatch(cmds []*Command, timeout time.Duration) error
- func (r *MultiRegionReplicator) Health() *HealthStatus
- func (r *MultiRegionReplicator) IsLeader() bool
- func (r *MultiRegionReplicator) IsPrimaryRegion() bool
- func (r *MultiRegionReplicator) LeaderAddr() string
- func (r *MultiRegionReplicator) LeaderID() string
- func (r *MultiRegionReplicator) Mode() ReplicationMode
- func (r *MultiRegionReplicator) NodeID() string
- func (r *MultiRegionReplicator) RegionFailover(ctx context.Context) error
- func (r *MultiRegionReplicator) RegionID() string
- func (r *MultiRegionReplicator) SetTransport(transport Transport)
- func (r *MultiRegionReplicator) Shutdown() error
- func (r *MultiRegionReplicator) Start(ctx context.Context) error
- func (r *MultiRegionReplicator) WaitForLeader(ctx context.Context) error
- type NodePayload
- type PeerConfig
- type PeerConnection
- type PeerMetricsGC
- type PeerStatus
- type PeerTracker
- type PromoteRequest
- type PromoteResponse
- type RaftAppendEntriesRequest
- type RaftAppendEntriesResponse
- type RaftConfig
- type RaftLogEntry
- type RaftRPCMessage
- type RaftRPCType
- type RaftReplicator
- func (r *RaftReplicator) AddVoter(id, addr string) error
- func (r *RaftReplicator) Apply(cmd *Command, timeout time.Duration) error
- func (r *RaftReplicator) ApplyBatch(cmds []*Command, timeout time.Duration) error
- func (r *RaftReplicator) GetConfiguration() ([]PeerStatus, error)
- func (r *RaftReplicator) HandleForwardApply(cmd *Command, timeout time.Duration) error
- func (r *RaftReplicator) HandleRaftAppendEntries(req *RaftAppendEntriesRequest) (*RaftAppendEntriesResponse, error)
- func (r *RaftReplicator) HandleRaftVote(req *RaftVoteRequest) (*RaftVoteResponse, error)
- func (r *RaftReplicator) Health() *HealthStatus
- func (r *RaftReplicator) IsLeader() bool
- func (r *RaftReplicator) LeaderAddr() string
- func (r *RaftReplicator) LeaderID() string
- func (r *RaftReplicator) Mode() ReplicationMode
- func (r *RaftReplicator) NodeID() string
- func (r *RaftReplicator) RemoveServer(id string) error
- func (r *RaftReplicator) SetReplicatorMetrics(bag *observability.ReplicationMetrics, tracker *PeerTracker)
- func (r *RaftReplicator) SetTransport(t Transport)
- func (r *RaftReplicator) Shutdown() error
- func (r *RaftReplicator) Start(ctx context.Context) error
- func (r *RaftReplicator) WaitForLeader(ctx context.Context) error
- type RaftState
- type RaftVoteRequest
- type RaftVoteResponse
- type RemoteRegionConfig
- type ReplicatedEngine
- func (e *ReplicatedEngine) BulkCreateEdges(edges []*storage.Edge) error
- func (e *ReplicatedEngine) BulkCreateNodes(nodes []*storage.Node) error
- func (e *ReplicatedEngine) BulkDeleteEdges(ids []storage.EdgeID) error
- func (e *ReplicatedEngine) BulkDeleteNodes(ids []storage.NodeID) error
- func (e *ReplicatedEngine) CreateEdge(edge *storage.Edge) error
- func (e *ReplicatedEngine) CreateNode(node *storage.Node) (storage.NodeID, error)
- func (e *ReplicatedEngine) DeleteByPrefix(prefix string) (nodesDeleted int64, edgesDeleted int64, err error)
- func (e *ReplicatedEngine) DeleteEdge(id storage.EdgeID) error
- func (e *ReplicatedEngine) DeleteNode(id storage.NodeID) error
- func (e *ReplicatedEngine) IsLeader() bool
- func (e *ReplicatedEngine) UpdateEdge(edge *storage.Edge) error
- func (e *ReplicatedEngine) UpdateNode(node *storage.Node) error
- type ReplicationMode
- type Replicator
- type SnapshotReader
- type SnapshotWriter
- type StandaloneReplicator
- func (r *StandaloneReplicator) Apply(cmd *Command, timeout time.Duration) error
- func (r *StandaloneReplicator) ApplyBatch(cmds []*Command, timeout time.Duration) error
- func (r *StandaloneReplicator) Health() *HealthStatus
- func (r *StandaloneReplicator) IsLeader() bool
- func (r *StandaloneReplicator) LeaderAddr() string
- func (r *StandaloneReplicator) LeaderID() string
- func (r *StandaloneReplicator) Mode() ReplicationMode
- func (r *StandaloneReplicator) NodeID() string
- func (r *StandaloneReplicator) SetReplicatorMetrics(bag *observability.ReplicationMetrics, tracker *PeerTracker)
- func (r *StandaloneReplicator) Shutdown() error
- func (r *StandaloneReplicator) Start(ctx context.Context) error
- func (r *StandaloneReplicator) WaitForLeader(ctx context.Context) error
- type Storage
- type StorageAdapter
- func (a *StorageAdapter) ApplyCommand(cmd *Command) error
- func (a *StorageAdapter) Close() error
- func (a *StorageAdapter) Engine() storage.Engine
- func (a *StorageAdapter) FlushWAL() error
- func (a *StorageAdapter) GetWALEntries(fromPosition uint64, maxEntries int) ([]*WALEntry, error)
- func (a *StorageAdapter) GetWALPosition() (uint64, error)
- func (a *StorageAdapter) PruneWALEntries(uptoPosition uint64)
- func (a *StorageAdapter) RestoreSnapshot(r SnapshotReader) error
- func (a *StorageAdapter) SetExecutor(executor *cypher.StorageExecutor)
- func (a *StorageAdapter) WriteSnapshot(w SnapshotWriter) error
- type SyncMode
- type TLSConfig
- type Transport
- type VoteRequest
- type VoteResponse
- type WALApplier
- type WALBatchResponse
- type WALEntry
- type WALStreamer
Constants ¶
const CurrentCodecVersion uint32 = 1
CurrentCodecVersion is the codec generation this binary speaks. Pre-version peers omit the field (wire value 0). Version 1 is the first versioned frame that supports the optional traceparent field added in Phase 8.
const DefaultPeerGCInterval = 5 * time.Minute
DefaultPeerGCInterval is the D-05b default sweep cadence.
const DefaultPeerGCStaleness = 24 * time.Hour
DefaultPeerGCStaleness is the D-05b default staleness threshold — peers not observed within this window are evicted from the registry.
Variables ¶
var ( // ErrNotLeader is returned when a write is attempted on a non-leader node. ErrNotLeader = errors.New("not leader") // ErrNoLeader is returned when no leader is available in the cluster. ErrNoLeader = errors.New("no leader available") // ErrTimeout is returned when an operation times out. ErrTimeout = errors.New("operation timed out") // ErrClosed is returned when operating on a closed replicator. ErrClosed = errors.New("replicator is closed") // ErrStandbyMode is returned when writes are attempted on a standby node. ErrStandbyMode = errors.New("node is in standby mode") // ErrNotReady is returned when the replicator hasn't finished initialization. ErrNotReady = errors.New("replicator not ready") )
Errors returned by replication operations.
Functions ¶
func RegisterClusterHandlers ¶
func RegisterClusterHandlers(t *ClusterTransport, r Replicator)
RegisterClusterHandlers wires a Replicator's handler methods into a ClusterTransport. This is intentionally transport-specific because other Transport implementations may not support message-type dispatch.
Types ¶
type AppendEntriesRequest ¶
type AppendEntriesRequest struct {
Term uint64 `json:"term"`
LeaderID string `json:"leader_id"`
LeaderAddr string `json:"leader_addr"`
PrevLogIndex uint64 `json:"prev_log_index"`
PrevLogTerm uint64 `json:"prev_log_term"`
Entries []*RaftLogEntry `json:"entries"`
LeaderCommit uint64 `json:"leader_commit"`
// CodecVersion identifies the wire format generation (TRC-21/TRC-22).
// 0 (or absent via omitempty) = pre-version frame; 1 = first versioned frame.
// Rolling-upgrade compat: receivers treat absent/0 as pre-version and accept
// gracefully; leaders omit the field when any peer is at version 0.
CodecVersion uint32 `json:"codec_version,omitempty"`
}
AppendEntriesRequest is sent by the leader to replicate log entries.
type AppendEntriesResponse ¶
type AppendEntriesResponse struct {
Term uint64 `json:"term"`
Success bool `json:"success"`
MatchIndex uint64 `json:"match_index"`
ConflictIndex uint64 `json:"conflict_index"`
ConflictTerm uint64 `json:"conflict_term"`
ResponderID string `json:"responder_id"`
// CodecVersion echoes the responder's supported codec version so the leader
// can track peer capabilities for mixed-cluster compat (TRC-22).
CodecVersion uint32 `json:"codec_version,omitempty"`
}
AppendEntriesResponse is the response to an append entries request.
type ClusterConnection ¶
type ClusterConnection struct {
// contains filtered or unexported fields
}
ClusterConnection implements PeerConnection for cluster communication.
func (*ClusterConnection) Close ¶
func (c *ClusterConnection) Close() error
Close closes the connection.
func (*ClusterConnection) IsConnected ¶
func (c *ClusterConnection) IsConnected() bool
IsConnected returns true if the connection is active.
func (*ClusterConnection) SendFence ¶
func (c *ClusterConnection) SendFence(ctx context.Context, req *FenceRequest) (*FenceResponse, error)
SendFence sends a fence request to the peer.
func (*ClusterConnection) SendForwardApply ¶
func (c *ClusterConnection) SendForwardApply(ctx context.Context, cmd *Command, timeout time.Duration) error
SendForwardApply sends a write command to the leader for application. Used by followers to forward writes to the leader automatically.
func (*ClusterConnection) SendHeartbeat ¶
func (c *ClusterConnection) SendHeartbeat(ctx context.Context, req *HeartbeatRequest) (*HeartbeatResponse, error)
SendHeartbeat sends a heartbeat to the peer.
func (*ClusterConnection) SendPromote ¶
func (c *ClusterConnection) SendPromote(ctx context.Context, req *PromoteRequest) (*PromoteResponse, error)
SendPromote sends a promote request to the peer.
func (*ClusterConnection) SendRaftAppendEntries ¶
func (c *ClusterConnection) SendRaftAppendEntries(ctx context.Context, req *RaftAppendEntriesRequest) (*RaftAppendEntriesResponse, error)
SendRaftAppendEntries sends Raft append entries to the peer.
func (*ClusterConnection) SendRaftVote ¶
func (c *ClusterConnection) SendRaftVote(ctx context.Context, req *RaftVoteRequest) (*RaftVoteResponse, error)
SendRaftVote sends a Raft vote request to the peer.
func (*ClusterConnection) SendWALBatch ¶
func (c *ClusterConnection) SendWALBatch(ctx context.Context, entries []*WALEntry) (*WALBatchResponse, error)
SendWALBatch sends WAL entries to the peer.
type ClusterMessage ¶
type ClusterMessage struct {
Type ClusterMessageType
NodeID string
Timestamp int64
Signature string
Payload []byte
}
ClusterMessage is the on-wire format for cluster communication.
type ClusterMessageType ¶
type ClusterMessageType uint8
ClusterMessageType identifies cluster protocol messages.
const ( // Raft consensus messages ClusterMsgVoteRequest ClusterMessageType = iota + 1 ClusterMsgVoteResponse ClusterMsgAppendEntries ClusterMsgAppendEntriesResponse // HA standby messages ClusterMsgWALBatch ClusterMsgWALBatchResponse ClusterMsgHeartbeat ClusterMsgHeartbeatResponse ClusterMsgFence ClusterMsgFenceResponse ClusterMsgPromote ClusterMsgPromoteResponse // Cluster management ClusterMsgJoin ClusterMsgJoinResponse ClusterMsgLeave ClusterMsgLeaveResponse ClusterMsgStatus ClusterMsgStatusResponse // Write forwarding: follower sends write to leader for application ClusterMsgForwardApply ClusterMsgForwardApplyResponse )
type ClusterTransport ¶
type ClusterTransport struct {
// contains filtered or unexported fields
}
ClusterTransport handles cluster-to-cluster communication.
For client-facing queries, use the standard Bolt server (pkg/bolt). ClusterTransport is specifically for:
- Raft consensus protocol
- WAL streaming for HA
- Cluster coordination
func NewClusterTransport ¶
func NewClusterTransport(config *ClusterTransportConfig) *ClusterTransport
NewClusterTransport creates a cluster transport.
func (*ClusterTransport) Close ¶
func (t *ClusterTransport) Close() error
Close shuts down the transport.
func (*ClusterTransport) Connect ¶
func (t *ClusterTransport) Connect(ctx context.Context, addr string) (PeerConnection, error)
Connect establishes a connection to a peer node.
func (*ClusterTransport) Listen ¶
func (t *ClusterTransport) Listen(ctx context.Context, addr string, handler ConnectionHandler) error
Listen starts accepting cluster connections.
func (*ClusterTransport) RegisterHandler ¶
func (t *ClusterTransport) RegisterHandler(msgType ClusterMessageType, handler MessageHandler)
RegisterHandler registers a handler for a message type.
type ClusterTransportConfig ¶
type ClusterTransportConfig struct {
NodeID string
BindAddr string
DialTimeout time.Duration
ReadTimeout time.Duration
WriteTimeout time.Duration
MaxMsgSize int
TLSServer *tls.Config
TLSClient *tls.Config
AuthSecret []byte
AuthMaxSkew time.Duration
}
ClusterTransportConfig configures the cluster transport.
func DefaultClusterTransportConfig ¶
func DefaultClusterTransportConfig() *ClusterTransportConfig
DefaultClusterTransportConfig returns production defaults.
type Command ¶
type Command struct {
// Type identifies the operation type.
Type CommandType
// Data is the serialized operation data.
Data []byte
// Timestamp when the command was created.
Timestamp time.Time
// RequestID for idempotency/deduplication.
RequestID string
}
Command represents a write operation to be replicated.
type CommandType ¶
type CommandType uint8
CommandType identifies the type of write operation.
const ( // CmdUnknown is an unknown command type. CmdUnknown CommandType = iota // CmdCreateNode creates a new node. CmdCreateNode // CmdUpdateNode updates an existing node. CmdUpdateNode // CmdDeleteNode deletes a node. CmdDeleteNode // CmdCreateEdge creates a new edge. CmdCreateEdge // CmdDeleteEdge deletes an edge. CmdDeleteEdge // CmdUpdateEdge updates an existing edge. CmdUpdateEdge // CmdSetProperty sets a property on a node. CmdSetProperty // CmdBatchWrite is a batch of multiple writes. CmdBatchWrite // CmdCypher is a Cypher write query. CmdCypher // CmdVoteRequest is a Raft vote request. CmdVoteRequest // CmdVoteResponse is a Raft vote response. CmdVoteResponse // CmdAppendEntries is a Raft append entries request. CmdAppendEntries // CmdAppendEntriesResponse is a Raft append entries response. CmdAppendEntriesResponse // CmdDeleteByPrefix deletes all nodes/edges under an ID prefix (e.g. database drop). CmdDeleteByPrefix // CmdBulkCreateNodes creates multiple nodes. CmdBulkCreateNodes // CmdBulkCreateEdges creates multiple edges. CmdBulkCreateEdges // CmdBulkDeleteNodes deletes multiple nodes. CmdBulkDeleteNodes // CmdBulkDeleteEdges deletes multiple edges. CmdBulkDeleteEdges )
type Config ¶
type Config struct {
// Mode selects the replication mode (default: standalone)
// Environment: NORNICDB_CLUSTER_MODE
Mode ReplicationMode
// NodeID uniquely identifies this node in the cluster.
// Auto-generated if not set.
// Environment: NORNICDB_CLUSTER_NODE_ID
NodeID string
// BindAddr is the address to bind the replication server.
// Environment: NORNICDB_CLUSTER_BIND_ADDR
BindAddr string
// AdvertiseAddr is the address advertised to other nodes.
// Defaults to BindAddr if not set.
// Environment: NORNICDB_CLUSTER_ADVERTISE_ADDR
AdvertiseAddr string
// ReplicationSecret is a shared secret used to authenticate cluster messages.
// When set, nodes must present valid HMAC signatures for replication traffic.
// Environment: NORNICDB_CLUSTER_REPLICATION_SECRET
ReplicationSecret string
// DataDir for replication state (Raft logs, snapshots).
// Defaults to main database DataDir + "/replication"
// Environment: NORNICDB_CLUSTER_DATA_DIR
DataDir string
// HAStandby holds hot standby configuration.
HAStandby HAStandbyConfig
// Raft holds Raft consensus configuration.
Raft RaftConfig
// MultiRegion holds multi-region configuration.
MultiRegion MultiRegionConfig
// Consistency holds default consistency levels.
Consistency ConsistencyConfig
// TLS holds security configuration for encrypted connections.
TLS TLSConfig
}
Config holds all replication configuration. Designed to integrate with NornicDB's existing config patterns.
func DefaultConfig ¶
func DefaultConfig() *Config
DefaultConfig returns a Config with sensible defaults for standalone mode. All values are tuned for single-node operation with zero overhead.
func LoadFromEnv ¶
func LoadFromEnv() *Config
LoadFromEnv loads replication configuration from environment variables. Uses NORNICDB_CLUSTER_* prefix for all replication settings.
Example environment for HA Standby:
NORNICDB_CLUSTER_MODE=ha_standby NORNICDB_CLUSTER_NODE_ID=primary-1 NORNICDB_CLUSTER_HA_ROLE=primary NORNICDB_CLUSTER_HA_PEER_ADDR=standby-1:7000 NORNICDB_CLUSTER_HA_AUTO_FAILOVER=true
Example environment for Raft:
NORNICDB_CLUSTER_MODE=raft NORNICDB_CLUSTER_NODE_ID=node-1 NORNICDB_CLUSTER_RAFT_BOOTSTRAP=true NORNICDB_CLUSTER_RAFT_PEERS=node-2:node2:7000,node-3:node3:7000
func (*Config) IsStandalone ¶
IsStandalone returns true if running in standalone mode.
type ConnectionHandler ¶
type ConnectionHandler func(conn PeerConnection)
ConnectionHandler handles incoming connections.
type ConsistencyConfig ¶
type ConsistencyConfig struct {
// DefaultWriteConsistency for write operations.
// Environment: NORNICDB_CLUSTER_WRITE_CONSISTENCY
DefaultWriteConsistency ConsistencyLevel
// DefaultReadConsistency for read operations.
// Environment: NORNICDB_CLUSTER_READ_CONSISTENCY
DefaultReadConsistency ConsistencyLevel
}
ConsistencyConfig holds default consistency levels.
type ConsistencyLevel ¶
type ConsistencyLevel string
ConsistencyLevel defines read/write consistency requirements. Follows Cassandra naming conventions.
const ( // ConsistencyOne requires acknowledgment from one node. ConsistencyOne ConsistencyLevel = "ONE" // ConsistencyQuorum requires acknowledgment from majority (N/2+1). ConsistencyQuorum ConsistencyLevel = "QUORUM" // ConsistencyAll requires acknowledgment from all nodes. ConsistencyAll ConsistencyLevel = "ALL" // ConsistencyLocalOne requires acknowledgment from one node in local region. ConsistencyLocalOne ConsistencyLevel = "LOCAL_ONE" // ConsistencyLocalQuorum requires quorum from local region only. ConsistencyLocalQuorum ConsistencyLevel = "LOCAL_QUORUM" )
type EdgePayload ¶
type EdgePayload struct {
ID string
StartNode string
EndNode string
Type string
Properties map[string]any
CreatedAt int64
Confidence float64
AutoGenerated bool
}
EdgePayload is the replication-safe representation of storage.Edge.
type FenceRequest ¶
FenceRequest requests the peer to stop accepting writes.
type FenceResponse ¶
type FenceResponse struct {
Fenced bool
}
FenceResponse is the response to a fence request.
type HAStandbyConfig ¶
type HAStandbyConfig struct {
// Role is "primary" or "standby".
// Environment: NORNICDB_CLUSTER_HA_ROLE
Role string
// PeerAddr is the address of the other node.
// Environment: NORNICDB_CLUSTER_HA_PEER_ADDR
PeerAddr string
// SyncMode controls replication synchronization.
// Environment: NORNICDB_CLUSTER_HA_SYNC_MODE
SyncMode SyncMode
// HeartbeatInterval is how often to send heartbeats.
// Environment: NORNICDB_CLUSTER_HA_HEARTBEAT_MS (in milliseconds)
HeartbeatInterval time.Duration
// FailoverTimeout is how long to wait before triggering failover.
// Environment: NORNICDB_CLUSTER_HA_FAILOVER_TIMEOUT
FailoverTimeout time.Duration
// AutoFailover enables automatic failover on primary failure.
// Environment: NORNICDB_CLUSTER_HA_AUTO_FAILOVER
AutoFailover bool
// WALBatchSize is max entries per WAL batch.
// Environment: NORNICDB_CLUSTER_HA_WAL_BATCH_SIZE
WALBatchSize int
// WALBatchTimeout is max time to wait for batch fill.
// Environment: NORNICDB_CLUSTER_HA_WAL_BATCH_TIMEOUT
WALBatchTimeout time.Duration
// ReconnectInterval is how often to retry connection to peer.
// Environment: NORNICDB_CLUSTER_HA_RECONNECT_INTERVAL
ReconnectInterval time.Duration
// MaxReconnectBackoff is maximum backoff for reconnection.
// Environment: NORNICDB_CLUSTER_HA_MAX_RECONNECT_BACKOFF
MaxReconnectBackoff time.Duration
}
HAStandbyConfig configures 2-node hot standby replication.
type HAStandbyReplicator ¶
type HAStandbyReplicator struct {
// contains filtered or unexported fields
}
HAStandbyReplicator implements hot standby replication between 2 nodes. One node is primary (accepts writes), the other is standby (receives WAL). Supports automatic failover when primary fails.
func NewHAStandbyReplicator ¶
func NewHAStandbyReplicator(config *Config, storage Storage) (*HAStandbyReplicator, error)
NewHAStandbyReplicator creates a new HA standby replicator.
func (*HAStandbyReplicator) Apply ¶
func (r *HAStandbyReplicator) Apply(cmd *Command, timeout time.Duration) error
Apply applies a command to storage.
func (*HAStandbyReplicator) ApplyBatch ¶
func (r *HAStandbyReplicator) ApplyBatch(cmds []*Command, timeout time.Duration) error
ApplyBatch applies multiple commands.
func (*HAStandbyReplicator) HandleFence ¶
func (r *HAStandbyReplicator) HandleFence(req *FenceRequest) (*FenceResponse, error)
HandleFence handles incoming fence request (for primary).
func (*HAStandbyReplicator) HandleHeartbeat ¶
func (r *HAStandbyReplicator) HandleHeartbeat(req *HeartbeatRequest) (*HeartbeatResponse, error)
HandleHeartbeat handles incoming heartbeat (for standby).
func (*HAStandbyReplicator) HandlePromote ¶
func (r *HAStandbyReplicator) HandlePromote(req *PromoteRequest) (*PromoteResponse, error)
HandlePromote handles an incoming promote request from the peer. Today this is used as a lightweight coordination hook; the actual promotion decision remains local (based on health/auto-failover config).
func (*HAStandbyReplicator) HandleWALBatch ¶
func (r *HAStandbyReplicator) HandleWALBatch(entries []*WALEntry) (*WALBatchResponse, error)
HandleWALBatch handles incoming WAL batch (for standby).
func (*HAStandbyReplicator) Health ¶
func (r *HAStandbyReplicator) Health() *HealthStatus
Health returns health status.
func (*HAStandbyReplicator) IsLeader ¶
func (r *HAStandbyReplicator) IsLeader() bool
IsLeader returns true if this is the primary.
func (*HAStandbyReplicator) LeaderAddr ¶
func (r *HAStandbyReplicator) LeaderAddr() string
LeaderAddr returns the primary address.
func (*HAStandbyReplicator) LeaderID ¶
func (r *HAStandbyReplicator) LeaderID() string
LeaderID returns the primary's node ID.
func (*HAStandbyReplicator) Mode ¶
func (r *HAStandbyReplicator) Mode() ReplicationMode
Mode returns the replication mode.
func (*HAStandbyReplicator) NodeID ¶
func (r *HAStandbyReplicator) NodeID() string
NodeID returns this node's ID.
func (*HAStandbyReplicator) Promote ¶
func (r *HAStandbyReplicator) Promote(ctx context.Context) error
Promote promotes this standby to primary.
func (*HAStandbyReplicator) SetReplicatorMetrics ¶ added in v1.1.0
func (r *HAStandbyReplicator) SetReplicatorMetrics(bag *observability.ReplicationMetrics, tracker *PeerTracker)
SetReplicatorMetrics implements MetricsAware — Plan-04-06 D-15a observation seam. Idempotent. Calling with nil disables observation.
func (*HAStandbyReplicator) SetTransport ¶
func (r *HAStandbyReplicator) SetTransport(t Transport)
SetTransport sets the transport for peer communication.
Safe to call before or after Start(). The chaos / partition-recovery tests swap transports on a running replicator to simulate network failure and recovery; r.mu serializes those writes against the connectToStandbyLoop / Listen reads of r.transport.
func (*HAStandbyReplicator) Shutdown ¶
func (r *HAStandbyReplicator) Shutdown() error
Shutdown stops the replicator.
func (*HAStandbyReplicator) Start ¶
func (r *HAStandbyReplicator) Start(ctx context.Context) error
Start initializes and starts the HA standby replicator.
func (*HAStandbyReplicator) WaitForLeader ¶
func (r *HAStandbyReplicator) WaitForLeader(ctx context.Context) error
WaitForLeader blocks until primary is available.
type HealthStatus ¶
type HealthStatus struct {
// Mode is the replication mode.
Mode ReplicationMode `json:"mode"`
// NodeID is this node's identifier.
NodeID string `json:"node_id"`
// Role is the current role (leader, follower, standby, etc.).
Role string `json:"role"`
// IsLeader indicates if this node accepts writes.
IsLeader bool `json:"is_leader"`
// LeaderID is the current leader's ID (if known).
LeaderID string `json:"leader_id,omitempty"`
// LeaderAddr is the current leader's address.
LeaderAddr string `json:"leader_addr,omitempty"`
// State is the current state (ready, initializing, etc.).
State string `json:"state"`
// Healthy indicates overall health.
Healthy bool `json:"healthy"`
// ReplicationLag is the lag behind leader (for followers).
ReplicationLag time.Duration `json:"replication_lag,omitempty"`
// LastContact is the last successful contact with leader/peer.
LastContact time.Time `json:"last_contact,omitempty"`
// Peers contains status of peer nodes.
Peers []PeerStatus `json:"peers,omitempty"`
// Region is the region ID (for multi-region).
Region string `json:"region,omitempty"`
// CommitIndex is the last committed log index (for Raft).
CommitIndex uint64 `json:"commit_index,omitempty"`
// AppliedIndex is the last applied log index.
AppliedIndex uint64 `json:"applied_index,omitempty"`
// Term is the current Raft term.
Term uint64 `json:"term,omitempty"`
}
HealthStatus represents the health state of the replicator.
type HeartbeatRequest ¶
HeartbeatRequest is a heartbeat message.
type HeartbeatResponse ¶
HeartbeatResponse is the response to a heartbeat.
type MessageHandler ¶
type MessageHandler func(ctx context.Context, nodeID string, msg *ClusterMessage) (*ClusterMessage, error)
MessageHandler processes incoming cluster messages.
type MetricsAware ¶ added in v1.1.0
type MetricsAware interface {
// SetReplicatorMetrics injects the observability bag + peer tracker.
// Idempotent. Calling with nil disables observation (defensive).
SetReplicatorMetrics(bag *observability.ReplicationMetrics, tracker *PeerTracker)
}
MetricsAware is the optional interface a Replicator implementation implements to accept the observability bag. Implementations may embed *atomic.Pointer[replicatorMetrics] or guard with their own mutex; the interface itself is intentionally minimal.
type MultiRegionConfig ¶
type MultiRegionConfig struct {
// RegionID identifies this region.
// Environment: NORNICDB_CLUSTER_REGION_ID
RegionID string
// LocalCluster is the local Raft cluster configuration.
LocalCluster RaftConfig
// RemoteRegions are other regions to replicate to.
// Environment: NORNICDB_CLUSTER_REMOTE_REGIONS (format: "region:host:port,...")
RemoteRegions []RemoteRegionConfig
// CrossRegionSyncMode controls cross-region replication mode.
// Environment: NORNICDB_CLUSTER_CROSS_REGION_MODE
CrossRegionSyncMode SyncMode
// CrossRegionBatchSize is entries per cross-region batch.
// Environment: NORNICDB_CLUSTER_CROSS_REGION_BATCH_SIZE
CrossRegionBatchSize int
// CrossRegionBatchTimeout is max time to wait for batch fill.
// Environment: NORNICDB_CLUSTER_CROSS_REGION_BATCH_TIMEOUT
CrossRegionBatchTimeout time.Duration
// ConflictStrategy handles write conflicts between regions.
// Environment: NORNICDB_CLUSTER_CONFLICT_STRATEGY
ConflictStrategy string // "last_write_wins", "manual"
}
MultiRegionConfig configures cross-region replication.
type MultiRegionReplicator ¶
type MultiRegionReplicator struct {
// contains filtered or unexported fields
}
MultiRegionReplicator implements multi-region replication. Each region runs a Raft cluster, with async replication between regions.
Architecture: - Each region has its own Raft cluster for strong local consistency - Cross-region replication is asynchronous via WAL streaming - One region is designated as "primary" for write coordination - Failover promotes a remote region to primary
func NewMultiRegionReplicator ¶
func NewMultiRegionReplicator(config *Config, storage Storage) (*MultiRegionReplicator, error)
NewMultiRegionReplicator creates a new multi-region replicator.
func (*MultiRegionReplicator) Apply ¶
func (r *MultiRegionReplicator) Apply(cmd *Command, timeout time.Duration) error
Apply applies a command through the local Raft cluster.
func (*MultiRegionReplicator) ApplyBatch ¶
func (r *MultiRegionReplicator) ApplyBatch(cmds []*Command, timeout time.Duration) error
ApplyBatch applies multiple commands.
func (*MultiRegionReplicator) Health ¶
func (r *MultiRegionReplicator) Health() *HealthStatus
Health returns health status.
func (*MultiRegionReplicator) IsLeader ¶
func (r *MultiRegionReplicator) IsLeader() bool
IsLeader returns true if this node is the local Raft leader.
func (*MultiRegionReplicator) IsPrimaryRegion ¶
func (r *MultiRegionReplicator) IsPrimaryRegion() bool
IsPrimaryRegion returns true if this is the primary region.
func (*MultiRegionReplicator) LeaderAddr ¶
func (r *MultiRegionReplicator) LeaderAddr() string
LeaderAddr returns the address of the local leader.
func (*MultiRegionReplicator) LeaderID ¶
func (r *MultiRegionReplicator) LeaderID() string
LeaderID returns the ID of the local leader.
func (*MultiRegionReplicator) Mode ¶
func (r *MultiRegionReplicator) Mode() ReplicationMode
Mode returns the replication mode.
func (*MultiRegionReplicator) NodeID ¶
func (r *MultiRegionReplicator) NodeID() string
NodeID returns this node's ID.
func (*MultiRegionReplicator) RegionFailover ¶
func (r *MultiRegionReplicator) RegionFailover(ctx context.Context) error
RegionFailover promotes this region to primary.
func (*MultiRegionReplicator) RegionID ¶
func (r *MultiRegionReplicator) RegionID() string
RegionID returns this region's ID.
func (*MultiRegionReplicator) SetTransport ¶
func (r *MultiRegionReplicator) SetTransport(transport Transport)
SetTransport sets the transport for cross-region communication.
func (*MultiRegionReplicator) Shutdown ¶
func (r *MultiRegionReplicator) Shutdown() error
Shutdown stops the replicator.
func (*MultiRegionReplicator) Start ¶
func (r *MultiRegionReplicator) Start(ctx context.Context) error
Start initializes and starts the multi-region replicator.
func (*MultiRegionReplicator) WaitForLeader ¶
func (r *MultiRegionReplicator) WaitForLeader(ctx context.Context) error
WaitForLeader blocks until a leader is elected.
type NodePayload ¶
type NodePayload struct {
ID string
Labels []string
Properties map[string]any
NamedEmbeddings map[string][]float32
ChunkEmbeddings [][]float32
CreatedAtUnixNs int64
UpdatedAtUnixNs int64
}
NodePayload is the replication-safe representation of storage.Node, including embeddings.
type PeerConfig ¶
type PeerConfig struct {
// ID is the unique identifier for this peer.
ID string
// Addr is the network address (host:port).
Addr string
}
PeerConfig holds configuration for a single peer node.
type PeerConnection ¶
type PeerConnection interface {
// SendWALBatch sends a batch of WAL entries to the peer.
SendWALBatch(ctx context.Context, entries []*WALEntry) (*WALBatchResponse, error)
// SendHeartbeat sends a heartbeat to the peer.
SendHeartbeat(ctx context.Context, req *HeartbeatRequest) (*HeartbeatResponse, error)
// SendFence sends a fence request to prevent split-brain.
SendFence(ctx context.Context, req *FenceRequest) (*FenceResponse, error)
// SendPromote notifies the peer to prepare for promotion.
SendPromote(ctx context.Context, req *PromoteRequest) (*PromoteResponse, error)
// SendRaftVote sends a Raft vote request and returns the response.
SendRaftVote(ctx context.Context, req *RaftVoteRequest) (*RaftVoteResponse, error)
// SendRaftAppendEntries sends Raft append entries and returns the response.
SendRaftAppendEntries(ctx context.Context, req *RaftAppendEntriesRequest) (*RaftAppendEntriesResponse, error)
// Close closes the connection.
Close() error
// IsConnected returns true if the connection is active.
IsConnected() bool
}
PeerConnection represents a connection to a peer node.
type PeerMetricsGC ¶ added in v1.1.0
type PeerMetricsGC struct {
// contains filtered or unexported fields
}
PeerMetricsGC is the lifecycle.Component that GCs stale peer label values from the per-peer GaugeVecs in *observability.ReplicationMetrics.
func NewPeerMetricsGC ¶ added in v1.1.0
func NewPeerMetricsGC(metrics *observability.ReplicationMetrics, interval, staleness time.Duration) *PeerMetricsGC
NewPeerMetricsGC constructs the GC. interval ≤ 0 falls back to DefaultPeerGCInterval; staleness ≤ 0 falls back to DefaultPeerGCStaleness. metrics nil disables the sweep (Start returns nil immediately) — defensive for partial-init.
func (*PeerMetricsGC) Name ¶ added in v1.1.0
func (g *PeerMetricsGC) Name() string
Name implements lifecycle.Component.
func (*PeerMetricsGC) Shutdown ¶ added in v1.1.0
func (g *PeerMetricsGC) Shutdown(ctx context.Context) error
Shutdown implements lifecycle.Component. Idempotent; subsequent calls are no-ops.
func (*PeerMetricsGC) Start ¶ added in v1.1.0
func (g *PeerMetricsGC) Start(ctx context.Context) error
Start implements lifecycle.Component. Blocks until ctx is cancelled or Shutdown is called. Does NOT perform an initial sweep — peer state at startup is empty; sweeping immediately would be a no-op anyway.
func (*PeerMetricsGC) Tracker ¶ added in v1.1.0
func (g *PeerMetricsGC) Tracker() *PeerTracker
Tracker exposes the PeerTracker so the replicator can call Mark at observation sites. The GC owns the tracker so its sweep + Mark calls share the same mutex (no cross-component locking surprises).
type PeerStatus ¶
type PeerStatus struct {
// ID is the peer's identifier.
ID string `json:"id"`
// Address is the peer's network address.
Address string `json:"address"`
// Healthy indicates if the peer is reachable.
Healthy bool `json:"healthy"`
// Lag is the replication lag for this peer.
Lag uint64 `json:"lag,omitempty"`
// LastContact is the last successful contact with this peer.
LastContact time.Time `json:"last_contact,omitempty"`
// State is the peer's current state.
State string `json:"state,omitempty"`
}
PeerStatus represents the status of a peer node.
type PeerTracker ¶ added in v1.1.0
type PeerTracker struct {
// contains filtered or unexported fields
}
PeerTracker maintains a thread-safe map of peer-label → last-seen time. The replicator code calls Mark on every observation site (heartbeat, replicate, RTT measurement); the GC walks the map to find stale entries.
Rebind-on-reconnect contract (Pitfall 3 mitigation per RESEARCH §Q7): after PeerMetricsGC.sweep evicts a peer via DeleteLabelValues, any held Bound observer in the replicator goes stale — observations land on a detached series that will not be re-aggregated. The replicator MUST re-bind via WithLabelValues at the next observation site (or simply avoid caching Bound observers across reconnects).
func NewPeerTracker ¶ added in v1.1.0
func NewPeerTracker() *PeerTracker
NewPeerTracker constructs an empty tracker.
func (*PeerTracker) Forget ¶ added in v1.1.0
func (t *PeerTracker) Forget(peer string)
Forget removes a peer entry — called by the GC after eviction.
func (*PeerTracker) Len ¶ added in v1.1.0
func (t *PeerTracker) Len() int
Len reports the current number of tracked peers — diagnostic for tests.
func (*PeerTracker) Mark ¶ added in v1.1.0
func (t *PeerTracker) Mark(peer string)
Mark records that we just observed `peer`. Callers (replicator observation sites) invoke this every time they touch a per-peer metric.
func (*PeerTracker) StaleSince ¶ added in v1.1.0
func (t *PeerTracker) StaleSince(cutoff time.Time) []string
StaleSince returns peers not observed since `cutoff`. The result is a fresh slice owned by the caller — modifying it is safe.
type PromoteRequest ¶
type PromoteRequest struct {
Reason string
}
PromoteRequest notifies the peer to prepare for promotion.
type PromoteResponse ¶
type PromoteResponse struct {
Ready bool
}
PromoteResponse is the response to a promote request.
type RaftAppendEntriesRequest ¶
type RaftAppendEntriesRequest struct {
Term uint64 `json:"term"`
LeaderID string `json:"leader_id"`
LeaderAddr string `json:"leader_addr"`
PrevLogIndex uint64 `json:"prev_log_index"`
PrevLogTerm uint64 `json:"prev_log_term"`
Entries []*RaftLogEntry `json:"entries"`
LeaderCommit uint64 `json:"leader_commit"`
CodecVersion uint32 `json:"codec_version,omitempty"`
}
RaftAppendEntriesRequest is a Raft AppendEntries RPC request.
type RaftAppendEntriesResponse ¶
type RaftAppendEntriesResponse struct {
Term uint64 `json:"term"`
Success bool `json:"success"`
MatchIndex uint64 `json:"match_index"`
ConflictIndex uint64 `json:"conflict_index,omitempty"`
ConflictTerm uint64 `json:"conflict_term,omitempty"`
ResponderID string `json:"responder_id"`
CodecVersion uint32 `json:"codec_version,omitempty"`
}
RaftAppendEntriesResponse is a Raft AppendEntries RPC response.
type RaftConfig ¶
type RaftConfig struct {
// ClusterID identifies the Raft cluster.
// Environment: NORNICDB_CLUSTER_RAFT_CLUSTER_ID
ClusterID string
// Bootstrap initializes a new cluster (only on first node).
// Environment: NORNICDB_CLUSTER_RAFT_BOOTSTRAP
Bootstrap bool
// Peers is the initial set of peer addresses.
// Format: "node-id:addr,node-id:addr"
// Environment: NORNICDB_CLUSTER_RAFT_PEERS
Peers []PeerConfig
// ElectionTimeout is the time before starting an election.
// Environment: NORNICDB_CLUSTER_RAFT_ELECTION_TIMEOUT
ElectionTimeout time.Duration
// HeartbeatTimeout is the interval for leader heartbeats.
// Environment: NORNICDB_CLUSTER_RAFT_HEARTBEAT_TIMEOUT
HeartbeatTimeout time.Duration
// LeaderLeaseTimeout is how long a leader maintains its lease.
// Environment: NORNICDB_CLUSTER_RAFT_LEADER_LEASE_TIMEOUT
LeaderLeaseTimeout time.Duration
// SnapshotInterval is seconds between automatic snapshots.
// Environment: NORNICDB_CLUSTER_RAFT_SNAPSHOT_INTERVAL
SnapshotInterval int
// SnapshotThreshold is log entries before triggering snapshot.
// Environment: NORNICDB_CLUSTER_RAFT_SNAPSHOT_THRESHOLD
SnapshotThreshold uint64
// TrailingLogs is entries to retain after snapshot.
// Environment: NORNICDB_CLUSTER_RAFT_TRAILING_LOGS
TrailingLogs uint64
// MaxAppendEntries is max entries per AppendEntries RPC.
// Environment: NORNICDB_CLUSTER_RAFT_MAX_APPEND_ENTRIES
MaxAppendEntries uint64
// CommitTimeout is max time to wait for commit.
// Environment: NORNICDB_CLUSTER_RAFT_COMMIT_TIMEOUT
CommitTimeout time.Duration
// MaxInflightLogs is max in-flight log entries for replication.
// Environment: NORNICDB_CLUSTER_RAFT_MAX_INFLIGHT_LOGS
MaxInflightLogs int
// SnapshotRetain is number of snapshots to retain.
// Environment: NORNICDB_CLUSTER_RAFT_SNAPSHOT_RETAIN
SnapshotRetain int
}
RaftConfig configures Raft consensus clustering.
type RaftLogEntry ¶
type RaftLogEntry struct {
Index uint64 `json:"index"`
Term uint64 `json:"term"`
Command *Command `json:"command"`
}
RaftLogEntry represents an entry in the Raft log.
type RaftRPCMessage ¶
type RaftRPCMessage struct {
Type RaftRPCType `json:"type"`
Payload []byte `json:"payload"`
}
RaftRPCMessage wraps all Raft RPC messages for transport.
type RaftRPCType ¶
type RaftRPCType uint8
RaftRPCType identifies the type of Raft RPC message.
const ( RPCVoteRequest RaftRPCType = iota + 1 RPCVoteResponse RPCAppendEntries RPCAppendEntriesResponse )
type RaftReplicator ¶
type RaftReplicator struct {
// contains filtered or unexported fields
}
RaftReplicator implements full Raft consensus-based replication. This provides strong consistency with automatic leader election.
func NewRaftReplicator ¶
func NewRaftReplicator(config *Config, storage Storage) (*RaftReplicator, error)
NewRaftReplicator creates a new Raft replicator.
func (*RaftReplicator) AddVoter ¶
func (r *RaftReplicator) AddVoter(id, addr string) error
AddVoter adds a voting member to the cluster.
func (*RaftReplicator) Apply ¶
func (r *RaftReplicator) Apply(cmd *Command, timeout time.Duration) error
Apply applies a command through Raft consensus. If this node is not the leader, it forwards the command to the leader automatically.
func (*RaftReplicator) ApplyBatch ¶
func (r *RaftReplicator) ApplyBatch(cmds []*Command, timeout time.Duration) error
ApplyBatch applies multiple commands atomically.
func (*RaftReplicator) GetConfiguration ¶
func (r *RaftReplicator) GetConfiguration() ([]PeerStatus, error)
GetConfiguration returns the current cluster configuration.
func (*RaftReplicator) HandleForwardApply ¶
func (r *RaftReplicator) HandleForwardApply(cmd *Command, timeout time.Duration) error
HandleForwardApply applies a command received from a follower (write forwarding). Only the leader should receive these; it applies the command through Raft as usual.
func (*RaftReplicator) HandleRaftAppendEntries ¶
func (r *RaftReplicator) HandleRaftAppendEntries(req *RaftAppendEntriesRequest) (*RaftAppendEntriesResponse, error)
HandleRaftAppendEntries handles an incoming AppendEntries RPC via the cluster transport.
func (*RaftReplicator) HandleRaftVote ¶
func (r *RaftReplicator) HandleRaftVote(req *RaftVoteRequest) (*RaftVoteResponse, error)
HandleRaftVote handles an incoming RequestVote RPC via the cluster transport.
func (*RaftReplicator) Health ¶
func (r *RaftReplicator) Health() *HealthStatus
Health returns health status.
func (*RaftReplicator) IsLeader ¶
func (r *RaftReplicator) IsLeader() bool
IsLeader returns true if this node is the Raft leader.
func (*RaftReplicator) LeaderAddr ¶
func (r *RaftReplicator) LeaderAddr() string
LeaderAddr returns the address of the current leader.
func (*RaftReplicator) LeaderID ¶
func (r *RaftReplicator) LeaderID() string
LeaderID returns the ID of the current leader.
func (*RaftReplicator) Mode ¶
func (r *RaftReplicator) Mode() ReplicationMode
Mode returns the replication mode.
func (*RaftReplicator) NodeID ¶
func (r *RaftReplicator) NodeID() string
NodeID returns this node's ID.
func (*RaftReplicator) RemoveServer ¶
func (r *RaftReplicator) RemoveServer(id string) error
RemoveServer removes a server from the cluster.
func (*RaftReplicator) SetReplicatorMetrics ¶ added in v1.1.0
func (r *RaftReplicator) SetReplicatorMetrics(bag *observability.ReplicationMetrics, tracker *PeerTracker)
SetReplicatorMetrics implements MetricsAware — Plan-04-06 D-15a observation seam. Idempotent. Calling with nil disables observation.
func (*RaftReplicator) SetTransport ¶
func (r *RaftReplicator) SetTransport(t Transport)
SetTransport sets the transport for peer communication.
func (*RaftReplicator) Shutdown ¶
func (r *RaftReplicator) Shutdown() error
Shutdown gracefully shuts down the replicator.
func (*RaftReplicator) Start ¶
func (r *RaftReplicator) Start(ctx context.Context) error
Start initializes and starts the Raft replicator.
func (*RaftReplicator) WaitForLeader ¶
func (r *RaftReplicator) WaitForLeader(ctx context.Context) error
WaitForLeader blocks until a leader is elected or context cancelled.
type RaftVoteRequest ¶
type RaftVoteRequest struct {
Term uint64 `json:"term"`
CandidateID string `json:"candidate_id"`
LastLogIndex uint64 `json:"last_log_index"`
LastLogTerm uint64 `json:"last_log_term"`
}
RaftVoteRequest is a Raft RequestVote RPC request.
type RaftVoteResponse ¶
type RaftVoteResponse struct {
Term uint64 `json:"term"`
VoteGranted bool `json:"vote_granted"`
VoterID string `json:"voter_id"`
}
RaftVoteResponse is a Raft RequestVote RPC response.
type RemoteRegionConfig ¶
type RemoteRegionConfig struct {
// RegionID identifies the remote region.
RegionID string
// Addrs are coordinator addresses for the remote region.
Addrs []string
// Priority for failover (lower = higher priority).
Priority int
}
RemoteRegionConfig holds configuration for a remote region.
type ReplicatedEngine ¶
ReplicatedEngine wraps a storage.Engine and routes write operations through a Replicator.
Design:
- Reads are served locally from the embedded Engine.
- Writes are turned into replication Commands and applied via the Replicator.
- The embedded Engine is used only for reads; replicated writes are applied to the *inner* engine by the StorageAdapter on each node.
This wrapper intentionally operates on the *base* storage (the engine that stores fully-qualified IDs like "<db>:<id>") so multi-database isolation is preserved.
func NewReplicatedEngine ¶
func NewReplicatedEngine(inner storage.Engine, replicator Replicator, timeout time.Duration) *ReplicatedEngine
func (*ReplicatedEngine) BulkCreateEdges ¶
func (e *ReplicatedEngine) BulkCreateEdges(edges []*storage.Edge) error
func (*ReplicatedEngine) BulkCreateNodes ¶
func (e *ReplicatedEngine) BulkCreateNodes(nodes []*storage.Node) error
func (*ReplicatedEngine) BulkDeleteEdges ¶
func (e *ReplicatedEngine) BulkDeleteEdges(ids []storage.EdgeID) error
func (*ReplicatedEngine) BulkDeleteNodes ¶
func (e *ReplicatedEngine) BulkDeleteNodes(ids []storage.NodeID) error
func (*ReplicatedEngine) CreateEdge ¶
func (e *ReplicatedEngine) CreateEdge(edge *storage.Edge) error
func (*ReplicatedEngine) CreateNode ¶
func (*ReplicatedEngine) DeleteByPrefix ¶
func (e *ReplicatedEngine) DeleteByPrefix(prefix string) (nodesDeleted int64, edgesDeleted int64, err error)
func (*ReplicatedEngine) DeleteEdge ¶
func (e *ReplicatedEngine) DeleteEdge(id storage.EdgeID) error
func (*ReplicatedEngine) DeleteNode ¶
func (e *ReplicatedEngine) DeleteNode(id storage.NodeID) error
func (*ReplicatedEngine) IsLeader ¶
func (e *ReplicatedEngine) IsLeader() bool
IsLeader reports whether this node can accept writes in the current replication mode. This is a convenience for higher-level components (e.g. multidb startup) that need to avoid performing metadata migrations on standby/followers.
func (*ReplicatedEngine) UpdateEdge ¶
func (e *ReplicatedEngine) UpdateEdge(edge *storage.Edge) error
func (*ReplicatedEngine) UpdateNode ¶
func (e *ReplicatedEngine) UpdateNode(node *storage.Node) error
type ReplicationMode ¶
type ReplicationMode string
ReplicationMode defines how the node participates in replication.
const ( // ModeStandalone is single-node operation with no replication (default). // This mode has zero overhead and is suitable for development, testing, // and small deployments where high availability is not required. ModeStandalone ReplicationMode = "standalone" // ModeHAStandby is hot standby replication between 2 nodes. // The primary accepts all writes and streams WAL to the standby. // Automatic failover promotes standby if primary fails. // Best for: Simple HA without the complexity of consensus. ModeHAStandby ReplicationMode = "ha_standby" // ModeRaft is Raft consensus with 3+ nodes. // Provides strong consistency with automatic leader election. // All writes go through Raft log for guaranteed ordering. // Best for: Production deployments requiring strong consistency. ModeRaft ReplicationMode = "raft" // ModeMultiRegion combines Raft clusters with cross-region replication. // Each region runs a Raft cluster for local strong consistency. // Cross-region replication is async for performance. // Best for: Global deployments with regional read replicas. ModeMultiRegion ReplicationMode = "multi_region" )
type Replicator ¶
type Replicator interface {
// Start initializes and starts the replicator.
// This should be called after the storage engine is ready.
Start(ctx context.Context) error
// Apply applies a write command to the cluster.
// Returns ErrNotLeader if this node cannot accept writes.
// The command is replicated according to the replication mode.
Apply(cmd *Command, timeout time.Duration) error
// ApplyBatch applies multiple commands atomically.
ApplyBatch(cmds []*Command, timeout time.Duration) error
// IsLeader returns true if this node can accept writes.
// For standalone mode, this always returns true.
// For HA standby, this returns true only on primary.
// For Raft, this returns true only on the Raft leader.
IsLeader() bool
// LeaderAddr returns the address of the current leader.
// Returns empty string if unknown or in standalone mode.
LeaderAddr() string
// LeaderID returns the ID of the current leader.
LeaderID() string
// Health returns the current health status of the replicator.
Health() *HealthStatus
// WaitForLeader blocks until a leader is elected or context is cancelled.
WaitForLeader(ctx context.Context) error
// Shutdown gracefully shuts down the replicator.
// This should be called before closing the storage engine.
Shutdown() error
// Mode returns the current replication mode.
Mode() ReplicationMode
// NodeID returns this node's unique identifier.
NodeID() string
}
Replicator is the unified interface for all replication modes. It abstracts the complexity of different replication strategies behind a simple interface that the rest of NornicDB uses.
The Replicator handles:
- Write routing (to leader)
- Read routing (based on consistency level)
- Health monitoring
- Failover orchestration
Example:
replicator, _ := replication.NewReplicator(config, storage)
replicator.Start(ctx)
defer replicator.Shutdown()
// All writes go through replicator
if err := replicator.Apply(cmd); err != nil {
if errors.Is(err, replication.ErrNotLeader) {
// Forward to leader
}
}
func NewReplicator ¶
func NewReplicator(config *Config, storage Storage) (Replicator, error)
NewReplicator creates the appropriate Replicator based on configuration. This is the main factory function for creating replicators.
For standalone mode, this returns a no-op replicator that has zero overhead. For other modes, it returns the appropriate distributed replicator.
Example:
config := replication.LoadFromEnv()
if err := config.Validate(); err != nil {
log.Fatal(err)
}
replicator, err := replication.NewReplicator(config, storage)
if err != nil {
log.Fatal(err)
}
type SnapshotReader ¶
SnapshotReader is used to read snapshot data.
type SnapshotWriter ¶
SnapshotWriter is used to write snapshot data.
type StandaloneReplicator ¶
type StandaloneReplicator struct {
// contains filtered or unexported fields
}
StandaloneReplicator is a no-op replicator for single-node operation. It implements the Replicator interface with zero overhead. All writes are applied directly to storage without any replication.
func NewStandaloneReplicator ¶
func NewStandaloneReplicator(config *Config, storage Storage) *StandaloneReplicator
NewStandaloneReplicator creates a new standalone (no-op) replicator.
func (*StandaloneReplicator) Apply ¶
func (r *StandaloneReplicator) Apply(cmd *Command, timeout time.Duration) error
Apply applies a command directly to storage.
func (*StandaloneReplicator) ApplyBatch ¶
func (r *StandaloneReplicator) ApplyBatch(cmds []*Command, timeout time.Duration) error
ApplyBatch applies multiple commands directly to storage.
func (*StandaloneReplicator) Health ¶
func (r *StandaloneReplicator) Health() *HealthStatus
Health returns health status.
func (*StandaloneReplicator) IsLeader ¶
func (r *StandaloneReplicator) IsLeader() bool
IsLeader always returns true for standalone mode.
func (*StandaloneReplicator) LeaderAddr ¶
func (r *StandaloneReplicator) LeaderAddr() string
LeaderAddr returns empty string for standalone mode.
func (*StandaloneReplicator) LeaderID ¶
func (r *StandaloneReplicator) LeaderID() string
LeaderID returns this node's ID for standalone mode.
func (*StandaloneReplicator) Mode ¶
func (r *StandaloneReplicator) Mode() ReplicationMode
Mode returns the replication mode.
func (*StandaloneReplicator) NodeID ¶
func (r *StandaloneReplicator) NodeID() string
NodeID returns this node's ID.
func (*StandaloneReplicator) SetReplicatorMetrics ¶ added in v1.1.0
func (r *StandaloneReplicator) SetReplicatorMetrics(bag *observability.ReplicationMetrics, tracker *PeerTracker)
SetReplicatorMetrics implements MetricsAware. Standalone uses metrics only to emit an initial Role=standalone (well, follower with no peers) observation at Start; the bag is otherwise quiescent.
func (*StandaloneReplicator) Shutdown ¶
func (r *StandaloneReplicator) Shutdown() error
Shutdown stops the standalone replicator.
func (*StandaloneReplicator) Start ¶
func (r *StandaloneReplicator) Start(ctx context.Context) error
Start initializes the standalone replicator (no-op).
func (*StandaloneReplicator) WaitForLeader ¶
func (r *StandaloneReplicator) WaitForLeader(ctx context.Context) error
WaitForLeader returns immediately for standalone mode.
type Storage ¶
type Storage interface {
// ApplyCommand applies a replicated command to storage.
ApplyCommand(cmd *Command) error
// GetWALPosition returns the current WAL position.
GetWALPosition() (uint64, error)
// GetWALEntries returns WAL entries starting from the given position.
GetWALEntries(fromPosition uint64, maxEntries int) ([]*WALEntry, error)
// WriteSnapshot writes a full snapshot to the given writer.
WriteSnapshot(w SnapshotWriter) error
// RestoreSnapshot restores state from a snapshot.
RestoreSnapshot(r SnapshotReader) error
}
Storage is the interface that the storage engine must implement to work with replication. This is a subset of the full storage.Engine interface, containing only what replication needs.
type StorageAdapter ¶
type StorageAdapter struct {
// contains filtered or unexported fields
}
StorageAdapter bridges the replication.Storage interface to storage.Engine. It translates replication commands into storage operations and maintains WAL state.
func NewStorageAdapter ¶
func NewStorageAdapter(engine storage.Engine) (*StorageAdapter, error)
NewStorageAdapter creates a new storage adapter wrapping the given engine. The WAL directory defaults to "data/replication/wal" if not specified.
func NewStorageAdapterWithWAL ¶
func NewStorageAdapterWithWAL(engine storage.Engine, walDir string) (*StorageAdapter, error)
NewStorageAdapterWithWAL creates a new storage adapter with a custom WAL directory. If walDir is empty, defaults to "data/replication/wal".
func (*StorageAdapter) ApplyCommand ¶
func (a *StorageAdapter) ApplyCommand(cmd *Command) error
ApplyCommand applies a replicated command to storage. WAL writes are now asynchronous for better performance.
func (*StorageAdapter) Close ¶
func (a *StorageAdapter) Close() error
Close releases replication resources (WAL file handles/background goroutines).
func (*StorageAdapter) Engine ¶
func (a *StorageAdapter) Engine() storage.Engine
Engine returns the underlying storage engine.
func (*StorageAdapter) FlushWAL ¶
func (a *StorageAdapter) FlushWAL() error
FlushWAL waits for all pending WAL writes to complete. This is useful for tests or when you need to ensure durability before proceeding.
The previous version slept 20ms hoping walWriterLoop would drain the queue — on slow runners (CI) goroutine scheduling jitter could leave the tail request still in walQueue at sync time, so callers that immediately observed GetWALPosition saw a count one short of what they queued. The fix is to send a barrier request through the same channel and wait for its acknowledgement. Because the writer reads requests from walQueue in FIFO order, every request queued before the barrier is guaranteed to be flushed before the barrier itself completes.
func (*StorageAdapter) GetWALEntries ¶
func (a *StorageAdapter) GetWALEntries(fromPosition uint64, maxEntries int) ([]*WALEntry, error)
GetWALEntries returns WAL entries starting from the given position.
func (*StorageAdapter) GetWALPosition ¶
func (a *StorageAdapter) GetWALPosition() (uint64, error)
GetWALPosition returns the current WAL position.
func (*StorageAdapter) PruneWALEntries ¶
func (a *StorageAdapter) PruneWALEntries(uptoPosition uint64)
PruneWALEntries drops in-memory WAL entries up to (and including) uptoPosition. This keeps memory bounded while streaming and does not affect the persistent WAL.
func (*StorageAdapter) RestoreSnapshot ¶
func (a *StorageAdapter) RestoreSnapshot(r SnapshotReader) error
RestoreSnapshot restores state from a snapshot.
func (*StorageAdapter) SetExecutor ¶
func (a *StorageAdapter) SetExecutor(executor *cypher.StorageExecutor)
SetExecutor sets a custom Cypher executor for the adapter. This allows using an executor with additional configuration (e.g., database manager, embedder).
func (*StorageAdapter) WriteSnapshot ¶
func (a *StorageAdapter) WriteSnapshot(w SnapshotWriter) error
WriteSnapshot writes a full snapshot to the given writer.
type SyncMode ¶
type SyncMode string
SyncMode defines the synchronization strategy for replication.
const ( // SyncAsync acknowledges writes immediately without waiting for replication. // Lowest latency, but potential data loss on failure. SyncAsync SyncMode = "async" // SyncQuorum waits until the replica acknowledges it has applied the write. // Highest durability, highest latency. SyncQuorum SyncMode = "quorum" )
type TLSConfig ¶
type TLSConfig struct {
// Enabled enables TLS for all replication connections.
// STRONGLY RECOMMENDED for production.
// Environment: NORNICDB_CLUSTER_TLS_ENABLED
Enabled bool
// CertFile is path to the server certificate (PEM format).
// Environment: NORNICDB_CLUSTER_TLS_CERT_FILE
CertFile string
// KeyFile is path to the server private key (PEM format).
// Environment: NORNICDB_CLUSTER_TLS_KEY_FILE
KeyFile string
// CAFile is path to the CA certificate for client verification (mTLS).
// If set, client certificates will be required and verified.
// Environment: NORNICDB_CLUSTER_TLS_CA_FILE
CAFile string
// VerifyClient requires client certificate verification (mTLS).
// Should be true in production for mutual authentication.
// Environment: NORNICDB_CLUSTER_TLS_VERIFY_CLIENT
VerifyClient bool
// InsecureSkipVerify skips server certificate verification.
// WARNING: Only use for testing. Never in production!
// Environment: NORNICDB_CLUSTER_TLS_INSECURE_SKIP_VERIFY
InsecureSkipVerify bool
// ServerName is the expected server name for certificate verification.
// Environment: NORNICDB_CLUSTER_TLS_SERVER_NAME
ServerName string
// MinVersion is the minimum TLS version (default: TLS 1.2).
// Environment: NORNICDB_CLUSTER_TLS_MIN_VERSION
MinVersion string // "1.2" or "1.3"
// CipherSuites restricts allowed cipher suites (optional).
// Leave empty to use secure defaults.
// Environment: NORNICDB_CLUSTER_TLS_CIPHER_SUITES
CipherSuites []string
}
TLSConfig configures TLS/mTLS for secure replication connections. ALL replication traffic should use TLS in production environments.
type Transport ¶
type Transport interface {
// Connect establishes a connection to a peer.
Connect(ctx context.Context, addr string) (PeerConnection, error)
// Listen starts accepting connections from peers.
Listen(ctx context.Context, addr string, handler ConnectionHandler) error
// Close shuts down the transport.
Close() error
}
Transport is the interface for peer-to-peer communication. This allows mocking in tests.
func NewDefaultTransport ¶
func NewDefaultTransport(config *ClusterTransportConfig) Transport
NewDefaultTransport creates a ClusterTransport for production use. See transport.go for the full implementation.
func NewDefaultTransportFromConfig ¶
NewDefaultTransportFromConfig creates a ClusterTransport using replication config.
type VoteRequest ¶
type VoteRequest struct {
Term uint64 `json:"term"`
CandidateID string `json:"candidate_id"`
LastLogIndex uint64 `json:"last_log_index"`
LastLogTerm uint64 `json:"last_log_term"`
}
VoteRequest is sent by candidates to request votes.
type VoteResponse ¶
type VoteResponse struct {
Term uint64 `json:"term"`
VoteGranted bool `json:"vote_granted"`
VoterID string `json:"voter_id"`
}
VoteResponse is the response to a vote request.
type WALApplier ¶
type WALApplier struct {
// contains filtered or unexported fields
}
WALApplier applies WAL entries to storage.
func NewWALApplier ¶
func NewWALApplier(storage Storage) *WALApplier
NewWALApplier creates a new WAL applier.
func (*WALApplier) ApplyBatch ¶
func (a *WALApplier) ApplyBatch(entries []*WALEntry) (uint64, error)
ApplyBatch applies a batch of WAL entries.
func (*WALApplier) Flush ¶
func (a *WALApplier) Flush() error
Flush ensures all pending entries are applied.
type WALBatchResponse ¶
WALBatchResponse is the response to a WAL batch.
type WALEntry ¶
type WALEntry struct {
// Position is the unique, monotonically increasing position.
Position uint64
// Timestamp when the entry was created.
Timestamp int64
// Command is the replicated command.
Command *Command
}
WALEntry represents a write-ahead log entry.
type WALStreamer ¶
type WALStreamer struct {
// contains filtered or unexported fields
}
WALStreamer manages WAL streaming from primary.
func NewWALStreamer ¶
func NewWALStreamer(storage Storage, batchSize int) *WALStreamer
NewWALStreamer creates a new WAL streamer.
func (*WALStreamer) AcknowledgePosition ¶
func (w *WALStreamer) AcknowledgePosition(pos uint64)
AcknowledgePosition marks entries up to this position as acknowledged.
func (*WALStreamer) GetPendingEntries ¶
func (w *WALStreamer) GetPendingEntries(maxEntries int) ([]*WALEntry, error)
GetPendingEntries returns WAL entries that haven't been acknowledged.
func (*WALStreamer) LastAcked ¶
func (w *WALStreamer) LastAcked() uint64
LastAcked returns the last acknowledged WAL position.