Documentation
¶
Overview ¶
Package logstore is the shared store for execution logs (deployments, pipeline steps, jobs, …). It externalizes the full log of a finished run out of Postgres text columns into a content-addressed object on a shared volume (or, later, object storage), keeping only a bounded tail + a reference in the database.
A producer keeps streaming lines live over the eventbus and appending a bounded tail to its DB row exactly as before; when the run reaches a terminal state it calls Externalize(ref, fullLog) once, which gzip-writes the log to the store and returns the counters to record on the row. Readers replay the store object for a finished run (falling back to the DB tail when the store is unconfigured, the ref is empty, or the object is gone) and the eventbus for the live remainder — the existing SSE contract, sourced from the store.
The store is nil-safe: a nil *Store (backend "off" / unconfigured) reports Enabled() == false and every method is a no-op, so the DB-tail-only behavior is preserved and the store is never a hard boot dependency.
Index ¶
- Variables
- func BackupRef(workspaceID, backupID uint) string
- func DeploymentRef(workspaceID, appID, deploymentID uint) string
- func JobRef(workspaceID, jobID uint) string
- func PipelineRunRef(workspaceID, runID uint) string
- func PipelineStepRef(workspaceID, runID uint, ordinal int) string
- func PlatformBackupRef(backupID uint) string
- func SplitLines(s string) []string
- func VolumeBackupRef(workspaceID, backupID uint) string
- type Backend
- type Config
- type Result
- type Store
- func (s *Store) Backend() string
- func (s *Store) Compressed() bool
- func (s *Store) Delete(ref string) error
- func (s *Store) Enabled() bool
- func (s *Store) Externalize(ref, content string) (Result, error)
- func (s *Store) Open(ref string) (io.ReadCloser, error)
- func (s *Store) OpenRaw(ref string) (io.ReadCloser, error)
- func (s *Store) Sweep(now time.Time) (int, error)
- func (s *Store) Tail(ref string, n int) (string, error)
Constants ¶
This section is empty.
Variables ¶
var ErrNotFound = errors.New("logstore: object not found")
ErrNotFound is returned by a backend when an object does not exist.
Functions ¶
func DeploymentRef ¶
DeploymentRef is the key for an application deployment's build/deploy log.
func PipelineRunRef ¶
PipelineRunRef is the key for a run's aggregate/checkout output.
func PipelineStepRef ¶
PipelineStepRef is the key for one step within a pipeline run.
func PlatformBackupRef ¶
PlatformBackupRef is the key for a platform (disaster-recovery) backup run's output. Platform backups have no workspace — they are admin-only.
func SplitLines ¶
SplitLines yields the non-empty lines of a stored/tail log, for SSE replay.
func VolumeBackupRef ¶
VolumeBackupRef is the key for a managed-volume backup run's output.
Types ¶
type Backend ¶
type Backend interface {
// Create returns a write sink for ref, replacing any existing object.
Create(ref string) (io.WriteCloser, error)
// Open returns the raw stored bytes for ref (still gzip-compressed when the
// store compresses). ErrNotFound when the object is absent.
Open(ref string) (io.ReadCloser, error)
// Delete removes the object for ref. Absent is not an error.
Delete(ref string) error
// Sweep deletes every object last modified before cutoff and returns the
// number removed.
Sweep(cutoff time.Time) (int, error)
// Name identifies the backend for logging (e.g. "filesystem").
Name() string
}
Backend is the pluggable object store behind the log store. The filesystem backend is the default; an S3/MinIO backend can be added later without touching producers or readers (they only see *Store).
func NewFSBackend ¶
NewFSBackend roots a filesystem backend at dir, creating it if needed.
type Config ¶
type Config struct {
// MaxBytes caps a single externalized log; past it the middle is dropped and
// Truncated is set (0 = no cap).
MaxBytes int64
// TailBytes is the size of the bounded tail the store returns for the DB row.
TailBytes int
// Compress gzip-compresses objects at rest when true.
Compress bool
// RetentionDays is how long objects are kept by Sweep (0 = keep forever).
RetentionDays int
}
Config bounds and shapes what the store writes. Zero values fall back to the defaults below.
type Result ¶
type Result struct {
Ref string // object key (empty when the store is disabled)
Bytes int64 // uncompressed size actually stored
Lines int // line count actually stored
Truncated bool // middle was dropped to honor MaxBytes
Tail string // bounded last slice for the DB LogTail column
}
Result reports what Externalize wrote, to record on the DB row.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is the shared log store. A nil *Store is valid and disabled.
func (*Store) Compressed ¶
Compressed reports whether stored objects are gzip-compressed (for the download endpoint's Content-Encoding).
func (*Store) Externalize ¶
Externalize writes the full log for ref and returns the counters + bounded tail to persist. On a disabled store it is a no-op returning a Result whose Tail is the (bounded) input, so callers can always trust Result.Tail.
func (*Store) Open ¶
func (s *Store) Open(ref string) (io.ReadCloser, error)
Open returns a decompressed reader over ref's full log (history + download).
func (*Store) OpenRaw ¶
func (s *Store) OpenRaw(ref string) (io.ReadCloser, error)
OpenRaw returns the raw stored bytes for ref (still gzipped when Compressed), for a streaming download that avoids server-side decompression.