errorlog

package
v1.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 19, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Overview

Package errorlog is the laboratory's own error journal: every component (operator, node-agent, L7 proxy, wg-demux, VPN and gateway pods, the agent) counts its errors by a normalized fingerprint, keeps a few recent messages and publishes the aggregate as a Kubernetes Event that the management agent reads and forwards in MonitoringUpdate.errors. Nothing here may carry tenant or participant data, an address or a secret: every message passes Redact before it is stored, and the fingerprint is made from the further Normalize d text.

Index

Constants

View Source
const (
	// Reason of every journal event.
	Reason = "ErrorJournal"
	// Label marks the events of the journal (a cheap server-side selector for the reader).
	Label = "cybericebox.com/error-journal"
)

The journal rides on Kubernetes Events, which every component may already write and the management agent may already read (the API server keeps them for an hour, deduplicates and rate-limits them): no log access and no new kind of permission.

View Source
const MaxGroups = 200

MaxGroups bounds the distinct errors one process keeps; past it new ones count into the "overflow" group.

View Source
const (
	// MaxSampleLen is the longest message kept.
	MaxSampleLen = 300
)
View Source
const MaxSamples = 3

MaxSamples is how many recent messages a group keeps.

View Source
const OverflowKind = "overflow"

OverflowFingerprintKind is the kind of the group that takes the errors past MaxGroups.

Variables

This section is empty.

Functions

func Fingerprint

func Fingerprint(component, kind, normalized string) string

Fingerprint is the stable id of an error: the component, the kind of the error path and the normalized message.

func Instance

func Instance() string

Instance is the name of this process's pod (its hostname), never an address.

func Namespace

func Namespace(def string) string

Namespace is where this component publishes: ERROR_JOURNAL_NAMESPACE, else the default.

func Normalize

func Normalize(msg string) string

Normalize is Redact with the parts that vary between occurrences of the same error replaced, so that the occurrences group together: numbers become <n>, long hex runs <id>, generated pod names lose their suffix. The fingerprint is made from it.

func Redact

func Redact(msg string) string

Redact removes what must not leave the cluster from an error message: key material, tokens, passwords, addresses (IPv4, IPv6, MAC), email addresses, host names, the names of objects and any quoted string (they carry tenant and group names), and shortens it to MaxSampleLen. What is left says what failed, not for whom.

func Start

func Start(ctx context.Context, agg *Aggregator, cfg *rest.Config, namespace string, log logr.Logger)

Start publishes the aggregator's changed groups as Events in namespace until ctx ends. The journal is best effort: a cluster that will not take the events only loses the journal, the component is never stopped for it.

func WrapLogger

func WrapLogger(l logr.Logger, agg *Aggregator) logr.Logger

WrapLogger returns a logger that records its errors in agg and logs as l does.

Types

type Aggregator

type Aggregator struct {
	Component string
	Instance  string
	// contains filtered or unexported fields
}

Aggregator counts the errors of one process (one component instance) by fingerprint. It is safe for concurrent use.

func New

func New(component, instance string) *Aggregator

New makes an aggregator for a component instance (a node name or a pod name, never an address).

func Setup

func Setup(component, instance string, base logr.Logger) (logr.Logger, *Aggregator)

Setup wraps a logger so that its errors are journalled; the returned aggregator is what Start publishes.

func (*Aggregator) Dirty

func (a *Aggregator) Dirty() []Group

Dirty returns the groups that changed since the last call (copies) and forgets the mark.

func (*Aggregator) Record

func (a *Aggregator) Record(kind, msg string)

Record counts one error of the given kind (a short code of the error path: reconcile, ovs, registry, ...).

func (*Aggregator) Snapshot

func (a *Aggregator) Snapshot() []Group

Snapshot returns every group (copies).

type Group

type Group struct {
	Fingerprint string
	Kind        string
	Normalized  string
	// Total counts the occurrences since the process started; the reader takes the difference between two looks.
	Total       int64
	First, Last time.Time
	Samples     []string
}

Group is one distinct error of one component.

type LineWriter

type LineWriter struct {
	// contains filtered or unexported fields
}

LineWriter wraps the writer of the standard library logger: every line that reads like an error is counted, every line is passed on. The agent logs with the standard logger.

func NewLineWriter

func NewLineWriter(agg *Aggregator, next io.Writer) *LineWriter

NewLineWriter counts the error-looking lines written to it into agg and writes all of them to next.

func (*LineWriter) Write

func (w *LineWriter) Write(p []byte) (int, error)

type Observation

type Observation struct {
	Component, Instance, Fingerprint, Kind, Normalized string
	Total                                              int64
	First, Last                                        time.Time
	Samples                                            []string
}

Observation is one journal event read back.

func ParseEvent

func ParseEvent(ev *corev1.Event) (Observation, bool)

ParseEvent reads a journal event; false when it is not one (or is damaged).

func (Observation) String

func (o Observation) String() string

String is for logs and tests.

type Publisher

type Publisher struct {
	Client    kubernetes.Interface
	Namespace string
	Agg       *Aggregator
	Interval  time.Duration
	Log       logr.Logger
}

Publisher writes the changed groups of an aggregator as Events.

func (*Publisher) Flush

func (p *Publisher) Flush(ctx context.Context)

Flush publishes the groups that changed since the last flush. A failure to publish is not an error of the journal (it would feed itself): the group is marked changed again and tried at the next flush.

func (*Publisher) Run

func (p *Publisher) Run(ctx context.Context)

Run publishes every Interval (30 s by default) until ctx ends, and once more at the end.

type Sink

type Sink struct {
	// contains filtered or unexported fields
}

Sink is a logr sink that counts every error-level message into an Aggregator and passes everything on to the real sink. The error path of any controller-runtime based component is then in the journal without touching its code.

func (*Sink) Enabled

func (s *Sink) Enabled(level int) bool

func (*Sink) Error

func (s *Sink) Error(err error, msg string, kv ...any)

func (*Sink) Info

func (s *Sink) Info(level int, msg string, kv ...any)

func (*Sink) Init

func (s *Sink) Init(info logr.RuntimeInfo)

func (*Sink) WithName

func (s *Sink) WithName(name string) logr.LogSink

func (*Sink) WithValues

func (s *Sink) WithValues(kv ...any) logr.LogSink

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL