logstore

package
v1.6.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 23, 2026 License: AGPL-3.0 Imports: 10 Imported by: 0

Documentation

Overview

Package logstore is the shared store for execution logs (deployments, pipeline steps, jobs, …). It externalizes the full log of a finished run out of Postgres text columns into a content-addressed object on a shared volume (or, later, object storage), keeping only a bounded tail + a reference in the database.

A producer keeps streaming lines live over the eventbus and appending a bounded tail to its DB row exactly as before; when the run reaches a terminal state it calls Externalize(ref, fullLog) once, which gzip-writes the log to the store and returns the counters to record on the row. Readers replay the store object for a finished run (falling back to the DB tail when the store is unconfigured, the ref is empty, or the object is gone) and the eventbus for the live remainder — the existing SSE contract, sourced from the store.

The store is nil-safe: a nil *Store (backend "off" / unconfigured) reports Enabled() == false and every method is a no-op, so the DB-tail-only behavior is preserved and the store is never a hard boot dependency.

Index

Constants

This section is empty.

Variables

View Source
var ErrNotFound = errors.New("logstore: object not found")

ErrNotFound is returned by a backend when an object does not exist.

Functions

func BackupRef

func BackupRef(workspaceID, backupID uint) string

BackupRef is the key for a managed-database backup run's output.

func DeploymentRef

func DeploymentRef(workspaceID, appID, deploymentID uint) string

DeploymentRef is the key for an application deployment's build/deploy log.

func JobRef

func JobRef(workspaceID, jobID uint) string

JobRef is the key for a one-off / cron job run's output.

func PipelineRunRef

func PipelineRunRef(workspaceID, runID uint) string

PipelineRunRef is the key for a run's aggregate/checkout output.

func PipelineStepRef

func PipelineStepRef(workspaceID, runID uint, ordinal int) string

PipelineStepRef is the key for one step within a pipeline run.

func PlatformBackupRef

func PlatformBackupRef(backupID uint) string

PlatformBackupRef is the key for a platform (disaster-recovery) backup run's output. Platform backups have no workspace — they are admin-only.

func SplitLines

func SplitLines(s string) []string

SplitLines yields the non-empty lines of a stored/tail log, for SSE replay.

func VolumeBackupRef

func VolumeBackupRef(workspaceID, backupID uint) string

VolumeBackupRef is the key for a managed-volume backup run's output.

Types

type Backend

type Backend interface {
	// Create returns a write sink for ref, replacing any existing object.
	Create(ref string) (io.WriteCloser, error)
	// Open returns the raw stored bytes for ref (still gzip-compressed when the
	// store compresses). ErrNotFound when the object is absent.
	Open(ref string) (io.ReadCloser, error)
	// Delete removes the object for ref. Absent is not an error.
	Delete(ref string) error
	// Sweep deletes every object last modified before cutoff and returns the
	// number removed.
	Sweep(cutoff time.Time) (int, error)
	// Name identifies the backend for logging (e.g. "filesystem").
	Name() string
}

Backend is the pluggable object store behind the log store. The filesystem backend is the default; an S3/MinIO backend can be added later without touching producers or readers (they only see *Store).

func NewFSBackend

func NewFSBackend(dir string) (Backend, error)

NewFSBackend roots a filesystem backend at dir, creating it if needed.

type Config

type Config struct {
	// MaxBytes caps a single externalized log; past it the middle is dropped and
	// Truncated is set (0 = no cap).
	MaxBytes int64
	// TailBytes is the size of the bounded tail the store returns for the DB row.
	TailBytes int
	// Compress gzip-compresses objects at rest when true.
	Compress bool
	// RetentionDays is how long objects are kept by Sweep (0 = keep forever).
	RetentionDays int
}

Config bounds and shapes what the store writes. Zero values fall back to the defaults below.

type Result

type Result struct {
	Ref       string // object key (empty when the store is disabled)
	Bytes     int64  // uncompressed size actually stored
	Lines     int    // line count actually stored
	Truncated bool   // middle was dropped to honor MaxBytes
	Tail      string // bounded last slice for the DB LogTail column
}

Result reports what Externalize wrote, to record on the DB row.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is the shared log store. A nil *Store is valid and disabled.

func New

func New(be Backend, cfg Config) *Store

New builds a store over a backend. A nil backend yields a disabled store.

func (*Store) Backend

func (s *Store) Backend() string

Backend returns the backend name (for logging/status); "off" when disabled.

func (*Store) Compressed

func (s *Store) Compressed() bool

Compressed reports whether stored objects are gzip-compressed (for the download endpoint's Content-Encoding).

func (*Store) Delete

func (s *Store) Delete(ref string) error

Delete removes ref's object (used on resource deletion / retention).

func (*Store) Enabled

func (s *Store) Enabled() bool

Enabled reports whether the store has a backend (safe on a nil receiver).

func (*Store) Externalize

func (s *Store) Externalize(ref, content string) (Result, error)

Externalize writes the full log for ref and returns the counters + bounded tail to persist. On a disabled store it is a no-op returning a Result whose Tail is the (bounded) input, so callers can always trust Result.Tail.

func (*Store) Open

func (s *Store) Open(ref string) (io.ReadCloser, error)

Open returns a decompressed reader over ref's full log (history + download).

func (*Store) OpenRaw

func (s *Store) OpenRaw(ref string) (io.ReadCloser, error)

OpenRaw returns the raw stored bytes for ref (still gzipped when Compressed), for a streaming download that avoids server-side decompression.

func (*Store) Sweep

func (s *Store) Sweep(now time.Time) (int, error)

Sweep removes objects older than the configured retention window and returns the count deleted. A zero/absent retention keeps everything (no-op).

func (*Store) Tail

func (s *Store) Tail(ref string, n int) (string, error)

Tail returns the last n bytes of ref's decompressed log, trimmed to a line boundary. Empty string (no error) when the object is absent or the store is disabled — callers fall back to the DB tail.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL