Documentation
¶
Overview ¶
Package forwarder is the agent's log forwarder subsystem (plan §4.1). It is a dual tap: per-running-instance workload logs (via the orchestrator's GetInstanceLogs, which fans out to runner.GetLogs) plus the agent's Outbox for system/subsystem events. Every record is stamped with NodeID and instance metadata, buffered through a durable on-disk spool for at-least-once delivery, and pushed to ingest.
On single-node, ingest is an in-process call into the control plane's ObserveService (LogStore.Write). On multi-node the same Ingester interface is implemented by an RPC client. The subsystem is OFF unless [observability] is enabled in the runefile — cmd/runed only registers it when a backend is configured.
The forwarder registers as an internal/agent.Subsystem so its lifecycle is governed by the agent alongside dataplane/dns/volumes/ingress.
Index ¶
Constants ¶
const ( // DefaultFlushInterval is how often the spool is drained to the ingester. DefaultFlushInterval = 2 * time.Second // DefaultBatchSize caps how many records are pushed per flush. DefaultBatchSize = 500 // DefaultPollInterval is how often the forwarder re-scans running // instances to start taps for newly-scheduled instances. DefaultPollInterval = 5 * time.Second // DefaultOutboxDrain caps how many outbox entries are pulled per tick. DefaultOutboxDrain = 1024 )
Defaults can be overridden via Config.
const DefaultSpoolCapacity = 100_000
DefaultSpoolCapacity bounds the in-memory spool before drop-oldest kicks in.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// Source taps workload logs. Required.
Source LogSource
// Ingester is the batch sink. Required.
Ingester Ingester
// Outbox is the agent's system-event buffer. Optional; when nil the
// event tap is disabled and only workload logs are forwarded.
Outbox *outbox.Outbox
// NodeID is stamped on every record. Required.
NodeID string
// Spool is the durable buffer for at-least-once delivery. When nil an
// in-memory spool is used (records are lost on crash, but never block the
// taps). Provide a disk-backed spool for production durability.
Spool Spool
// FlushInterval, BatchSize, PollInterval, OutboxDrain override the
// package defaults when > 0.
FlushInterval time.Duration
BatchSize int
PollInterval time.Duration
OutboxDrain int
// Logger; defaults to the global logger with component "agent.forwarder".
Logger log.Logger
}
Config bundles forwarder construction parameters.
type DiskSpool ¶
type DiskSpool struct {
*MemSpool
// contains filtered or unexported fields
}
DiskSpool wraps a MemSpool with a JSONL append-log for crash durability. Each Push is appended to the file; on startup the file is replayed into memory. Pop/Requeue operate on the in-memory view; the file is truncated when the in-memory spool drains to empty (a coarse compaction that is correct because a drained spool means every record was ingested). This is the MVP durable spool; a segment/offset design can replace it behind the Spool interface.
func NewDiskSpool ¶
NewDiskSpool opens (or creates) a JSONL spool file at path and replays any existing records into memory.
type Ingester ¶
Ingester is the sink the forwarder pushes batches to. On single-node it is the in-process ObserveService; on multi-node an RPC client. Write must be safe for concurrent callers and should be at-least-once friendly (the forwarder retries failed batches from the spool).
type LogSource ¶
type LogSource interface {
ListRunningInstances(ctx context.Context, namespace string) ([]*types.Instance, error)
GetInstanceLogs(ctx context.Context, namespace, instanceID string, opts types.LogOptions) (io.ReadCloser, error)
}
LogSource is the forwarder's read-only view of running instances and their logs. The orchestrator satisfies it (ListRunningInstances + GetInstanceLogs).
type MemSpool ¶
type MemSpool struct {
// contains filtered or unexported fields
}
MemSpool is an in-memory Spool with a drop-oldest bound. It is the default when no durable spool is configured: records survive transient ingest failures (via Requeue) but are lost on process crash.
func NewMemSpool ¶
NewMemSpool constructs an in-memory spool. capacity <= 0 uses DefaultSpoolCapacity.
type Spool ¶
type Spool interface {
// Push enqueues a record. Must be non-blocking.
Push(r observe.LogRecord)
// Pop removes and returns up to max records (oldest first). Empty => nil.
Pop(max int) []observe.LogRecord
// Requeue returns a previously-popped batch to the front of the spool
// (used after an ingest failure so records aren't dropped).
Requeue(batch []observe.LogRecord)
// Len reports the current depth.
Len() int
}
Spool is the forwarder's at-least-once buffer between the taps (producers) and the flush loop (consumer). Push never blocks the taps; Pop pulls a batch to ingest; Requeue returns a failed batch to the front for retry.
type Subsystem ¶
type Subsystem struct {
// contains filtered or unexported fields
}
Subsystem is the forwarder, an internal/agent.Subsystem.
func (*Subsystem) Ready ¶
func (s *Subsystem) Ready() <-chan struct{}
Ready closes once the forwarder's loops are running.