syncer

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 1, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package syncer contains the event debouncer, the thread-safe state cache and the single-threaded reconcile worker.

Index

Constants

View Source
const DefaultDeleteGuardMin = 3

DefaultDeleteGuardMin is how many deletions a run needs before the share guard applies. Below it, deleting everything is a two-host homelab doing exactly what it was told.

View Source
const StreamDownGrace = 2 * time.Minute

StreamDownGrace is how long the Docker event stream may be disconnected before the readiness probe fails. Reconnects take seconds; minutes mean the daemon or the socket proxy is gone.

Variables

This section is empty.

Functions

func BuildResource

func BuildResource(t *docker.Target, certificate npm.CertificateID, instance, prefix string) npm.Resource

BuildResource converts a Docker target into the NPM payload of its kind, stamping the ownership marker used for orphan detection.

The certificate is passed in because it is resolved against the live certificate list first (see certificates.go): "auto" only becomes an id once the reconcile loop knows what exists.

Types

type API

type API interface {
	List(ctx context.Context, kind npm.Kind) ([]npm.Resource, error)
	Create(ctx context.Context, resource npm.Resource) (npm.Resource, error)
	Update(ctx context.Context, id int, resource npm.Resource) (npm.Resource, error)
	Delete(ctx context.Context, kind npm.Kind, id int) error
	// SetEnabled toggles a resource through the dedicated /enable and
	// /disable endpoints. Neither API accepts `enabled` in a create or
	// update body.
	SetEnabled(ctx context.Context, kind npm.Kind, id int, enabled bool) error
	// ListCertificates backs the automatic certificate selection.
	ListCertificates(ctx context.Context) ([]npm.Certificate, error)
	// ListAccessLists resolves access lists referenced by name.
	ListAccessLists(ctx context.Context) ([]npm.AccessList, error)
	// Dialect reports what the server accepts: the flavour, which decides
	// whether NPMplus-only settings can be sent at all, and the fields a
	// release has added or retired - the meta object above all.
	Dialect() npm.Dialect
	// Recheck notices a server upgrade and re-probes the dialect if the
	// version changed. Called on the periodic resync.
	Recheck(ctx context.Context) error
}

API is the NPM surface the worker needs. It is kind-agnostic, which is what lets one reconcile implementation serve proxy hosts, redirection hosts, streams and 404 hosts alike. Implemented by *npm.Client.

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

Cache is the in-memory state cache guarding the API against redundant writes. Fingerprints are tracked per resource kind, so proxy hosts, redirections, streams and 404 hosts never collide even when they share a domain name.

It is read by the reconcile loop and the readiness probe while the worker writes to it, hence the explicit RWMutex.

func NewCache

func NewCache() *Cache

NewCache returns an empty cache with one bucket per resource kind.

func (*Cache) Counts

func (c *Cache) Counts() map[npm.Kind]int

Counts returns the number of managed resources per kind.

func (*Cache) Delete

func (c *Cache) Delete(kind npm.Kind, key string)

Delete removes a single entry.

func (*Cache) Get

func (c *Cache) Get(kind npm.Kind, key string) (Entry, bool)

Get returns the cached entry for a resource.

func (*Cache) Len

func (c *Cache) Len() int

Len returns the total number of cached resources across all kinds.

func (*Cache) LenKind

func (c *Cache) LenKind(kind npm.Kind) int

LenKind returns the number of cached resources of one kind.

func (*Cache) ReplaceKind

func (c *Cache) ReplaceKind(kind npm.Kind, entries map[string]Entry)

ReplaceKind swaps the contents of a single kind atomically. Other kinds are left untouched, so a failing collection does not discard known state.

func (*Cache) Set

func (c *Cache) Set(kind npm.Kind, key string, entry Entry)

Set stores or replaces a single entry.

func (*Cache) Snapshot

func (c *Cache) Snapshot() map[npm.Kind]map[string]Entry

Snapshot returns a deep copy of the cache contents.

type Debouncer

type Debouncer[T any] struct {
	// Interval is the quiet period after the last event.
	Interval time.Duration
	// MaxWait caps how long a continuous stream of events can delay a batch.
	// Zero disables the cap.
	MaxWait time.Duration
	// Now is injectable for tests; defaults to time.Now.
	Now func() time.Time
}

Debouncer coalesces bursts of events (think `docker compose up`) into a single batch. A batch is emitted once no new event arrived for Interval, and at the latest MaxWait after the first event of the burst.

func (*Debouncer[T]) Run

func (d *Debouncer[T]) Run(ctx context.Context, in <-chan T, out chan<- []T)

Run pumps events from in to out until ctx is cancelled or in is closed. It never blocks the producer for longer than one batch handover, so the Docker event stream cannot back up while the NPM API is slow.

On shutdown any pending batch is dropped on purpose: the worker performs a full reconcile before exiting, which supersedes individual events.

type Entry

type Entry struct {
	// ID is the id NPM assigned to the resource.
	ID int
	// Hash fingerprints the configuration last written to NPM.
	Hash string
	// Container is the container the resource was derived from.
	Container string
	// Index is the label index inside that container.
	Index int
	// Enabled is the enabled state last observed for the resource. It is
	// tracked separately from Hash because it is not part of the write
	// payload: both APIs toggle it through /enable and /disable.
	Enabled bool
	// Certificate is the certificate id attached to the resource, reported by
	// the /status endpoint.
	Certificate int
	// Running mirrors the container state at the time of the last run.
	Running bool
}

Entry is the cached state of one managed NPM resource.

type Metrics

type Metrics struct {
	Runs             uint64
	RunErrors        uint64
	Created          uint64
	Updated          uint64
	Deleted          uint64
	Failed           uint64
	Skipped          uint64
	DeletionsBlocked uint64
	// CertificateClass counts how often a certificate was chosen by match
	// class ("exact", "wildcard", ...).
	CertificateClass map[string]uint64
	// Managed is the number of resources currently under management.
	Managed int
	// ManagedByKind breaks that down per collection.
	ManagedByKind map[npm.Kind]int
	// LastRun is when the most recent reconcile finished.
	LastRun time.Time
	// LastDuration is how long it took.
	LastDuration time.Duration
}

Metrics is what the /metrics endpoint publishes.

Counters are cumulative for the lifetime of the process; gauges describe the current state. Everything is a plain value, so the HTTP handler stays a formatting exercise and this package keeps its "no dependencies" property.

type Options

type Options struct {
	DeleteOrphans   bool
	AdoptExisting   bool
	DryRun          bool
	ResyncInterval  time.Duration
	ShutdownTimeout time.Duration
	// Kinds limits reconciliation to a subset of resource types. Empty means
	// all four.
	Kinds []npm.Kind
	// CertificatePartial decides what happens when no single certificate
	// covers every domain of a host.
	CertificatePartial certs.Partial
	// CertificateAutoCreate requests a Let's Encrypt certificate when the
	// automatic selection finds nothing.
	CertificateAutoCreate bool
	// MigrateFromRedth takes over hosts created by Redth/npm-docker-sync and
	// re-stamps them as ours.
	MigrateFromRedth bool
	// CertificatePoll is how often the certificate list is checked for
	// changes; a change triggers a reconcile so a certificate created in the
	// UI reaches its hosts without a restart. 0 disables the poll.
	CertificatePoll time.Duration
	// OnStop decides what happens to the resources of a container that is
	// stopped but still exists.
	OnStop docker.OnStop
	// StopGrace is how long a stopped container is ignored, so a restart or a
	// recreate does not produce a disable/enable cycle.
	StopGrace time.Duration
	// InstanceID identifies this sync instance. Resources carrying another
	// instance's id are never touched, which lets several Docker hosts drive
	// one NPM.
	InstanceID string
	// LabelPrefix is the namespace in use. A resource created from a
	// different prefix means the configuration changed under us, and a run
	// that would delete because of it is stopped.
	LabelPrefix string
	// DeleteGuard is the largest share of the managed resources a single run
	// may delete (0.5 = half). 0 disables the guard.
	DeleteGuard float64
	// DeleteGuardMin is how many deletions a run must reach before the guard
	// looks at the share at all, so small setups stay usable.
	DeleteGuardMin int
}

Options configures the worker.

type ResourceStatus

type ResourceStatus struct {
	Kind        npm.Kind `json:"kind"`
	Key         string   `json:"key"`
	ID          int      `json:"id"`
	Container   string   `json:"container"`
	Index       int      `json:"index"`
	Enabled     bool     `json:"enabled"`
	Running     bool     `json:"container_running"`
	Certificate int      `json:"certificate_id"`
	// LastError is why the resource was last rejected, if it was.
	LastError string `json:"last_error,omitempty"`
	// RetryAfter is when the backoff of a failed resource expires.
	RetryAfter *time.Time `json:"retry_after,omitempty"`
}

ResourceStatus describes one managed resource for the /status endpoint.

type Result

type Result struct {
	Created   int `json:"created"`
	Updated   int `json:"updated"`
	Deleted   int `json:"deleted"`
	Unchanged int `json:"unchanged"`
	Skipped   int `json:"skipped"`
	Failed    int `json:"failed"`
}

Result summarises one reconcile run across all resource kinds.

func (Result) Changed

func (r Result) Changed() bool

Changed reports whether the run modified anything.

func (Result) LogValue

func (r Result) LogValue() slog.Value

LogValue implements slog.LogValuer.

type Status

type Status struct {
	LastRun time.Time
	Managed int
	Failed  int
	Err     error
	// Stream is the health of the Docker event subscription. A stream that
	// has been down for a while means changes are only noticed by the
	// periodic resync.
	Stream docker.StreamStatus
	// Blocked names the safety rule that stopped this run from deleting, if
	// one did.
	Blocked string
}

Status reports the outcome of the most recent reconcile run.

Err is reserved for failures that make the whole run meaningless (Docker or the NPM API unreachable). A single rejected resource is counted in Failed instead: the other hosts are still in sync, so the process is not unhealthy.

func (Status) Ready

func (s Status) Ready() bool

Ready reports whether at least one reconcile run finished without an infrastructure failure and the event stream is healthy.

type StreamSource

type StreamSource interface {
	StreamStatus() docker.StreamStatus
}

StreamSource is implemented by a source that also streams Docker events. The readiness probe reports the stream: a silently disconnected listener only reacts on the periodic resync, which must not look healthy.

type TargetSource

type TargetSource interface {
	// Snapshot returns the desired resources plus what the parser learned
	// about the containers - above all which of them are protected from
	// deletion because their labels could not be read.
	Snapshot(ctx context.Context) (docker.Snapshot, error)
}

TargetSource provides the desired state derived from Docker.

type Worker

type Worker struct {
	// contains filtered or unexported fields
}

Worker owns every write to the NPM API. Exactly one goroutine runs the reconcile loop, which keeps API calls sequential and avoids the SQLite "database is locked" errors NPM is prone to under concurrent writes.

func NewWorker

func NewWorker(api API, src TargetSource, opts Options, log *slog.Logger) *Worker

NewWorker creates a worker.

func (*Worker) Cache

func (w *Worker) Cache() *Cache

Cache exposes the state cache (used by tests and the health endpoint).

func (*Worker) CertificateHash

func (w *Worker) CertificateHash() string

CertificateHash returns the fingerprint of the certificate list of the last successful fetch. The poll loop compares it to detect new certificates.

func (*Worker) Metrics

func (w *Worker) Metrics() Metrics

Metrics returns a snapshot of the counters.

func (*Worker) PollCertificates

func (w *Worker) PollCertificates(ctx context.Context) bool

PollCertificates fetches the certificate list and reports whether it changed since the last run, which is what makes a freshly issued certificate reach its hosts without a restart.

func (*Worker) Reconcile

func (w *Worker) Reconcile(ctx context.Context) (Result, error)

Reconcile compares the desired Docker state with the live NPM state for every resource kind and applies the delta. Individual failures are collected so one broken resource cannot block the rest.

func (*Worker) Resources

func (w *Worker) Resources() []ResourceStatus

Resources lists every managed resource with what is known about it. It backs the /status endpoint, which is what a dashboard or an uptime check reads.

func (*Worker) Run

func (w *Worker) Run(ctx context.Context, triggers <-chan []docker.Event) error

Run processes reconcile triggers until ctx is cancelled. It performs an initial reconciliation on startup, a periodic full resync, and a final flush during graceful shutdown.

func (*Worker) Status

func (w *Worker) Status() Status

Status returns the outcome of the most recent reconcile run. It is used by the readiness probe.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL