archive

package
v0.1.0-alpha.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 18, 2026 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Overview

Package archive gives audit entries and events durability beyond their in-state rings (T-92).

Both histories live in capped rings inside replicated state (50k audit entries, 10k events), so on a busy cluster they age out in days — fine for "what just happened", useless for "who deleted this app in April". When backup.archive is on, a leader-gated sweeper continuously copies both streams to the cluster's existing S3 destination as gzipped NDJSON, sealed with the cluster data key exactly like a backup, and the query RPCs can read them back and merge with the ring.

Objects are laid out so a time-range query can skip most of them without a download:

audit/<YYYY-MM-DD>/<startMs>-<endMs>-<ulid>.ndjson.gz.enc
events/<YYYY-MM-DD>/<startMs>-<endMs>-<ulid>.ndjson.gz.enc

Nothing here ever deletes an object: the archive is the durable copy, and how long it is kept is the bucket's lifecycle policy to decide.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Encode

func Encode(sealer secrets.Sealer, records []proto.Message) ([]byte, error)

Encode marshals records to sealed, gzipped NDJSON.

Types

type Archiver

type Archiver struct {
	// contains filtered or unexported fields
}

Archiver copies the audit and event rings out to object storage.

func New

func New(src Source, cursors CursorStore, sealer secrets.Sealer, dest func() (volumes.ObjectStore, bool), clk clock.Clock, log *slog.Logger) *Archiver

New builds an archiver.

func (*Archiver) Sweep

func (a *Archiver) Sweep(ctx context.Context) int

Sweep archives everything settled since the last cursor for both streams. It returns the number of records written.

type Cursor

type Cursor struct {
	Ms  int64    `json:"ms"`
	IDs []string `json:"ids,omitempty"`
}

Cursor is the archiver's resume point for one stream, persisted in the replicated KV.

It is a millisecond watermark plus the ids already written at exactly that millisecond. A bare watermark cannot be both complete and duplicate-free: exclusive (> ms) drops entries sharing the boundary millisecond, inclusive (>= ms) re-writes them. Audit ids are minted before the raft round trip so they are not a strict watermark either, hence time plus an id set.

type CursorStore

type CursorStore interface {
	PutCursor(ctx context.Context, key string, value []byte) error
}

CursorStore persists the resume cursor through raft.

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader fetches archived records back out of object storage.

func NewReader

func NewReader(store volumes.ObjectStore, sealer secrets.Sealer) *Reader

NewReader builds a reader over a destination.

func (*Reader) Audit

func (r *Reader) Audit(ctx context.Context, sinceMs, untilMs int64, limit int, keep func(*zatterav1.AuditEntry) bool) ([]*zatterav1.AuditEntry, error)

Audit returns archived audit entries overlapping [sinceMs, untilMs] that satisfy keep, newest first, at most limit of them. untilMs <= 0 is open-ended. Objects whose key range misses the window are never fetched.

func (*Reader) Events

func (r *Reader) Events(ctx context.Context, sinceMs, untilMs int64, limit int, keep func(*zatterav1.Event) bool) ([]*zatterav1.Event, error)

Events is Audit for the event stream.

type Source

type Source interface {
	AuditSince(sinceMs int64) []*zatterav1.AuditEntry
	EventsSince(sinceMs int64) []*zatterav1.Event
	KV(key string) ([]byte, int64, int64, bool)
}

Source is the state the archiver reads. *state.Store satisfies it.

type Stream

type Stream string

Stream names the two archived histories; it is also the object key prefix.

const (
	StreamAudit  Stream = "audit"
	StreamEvents Stream = "events"
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL