Help us shape the future of Go! Take the 2026 Go Developer Survey

workspacelogs

package
v1.27.0-beta.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 2, 2026 License: MPL-2.0 Imports: 27 Imported by: 0

Documentation

Overview

Package workspacelogs retains bounded, redacted build and lifecycle output for workspace runs. It owns the on-disk record format, per-run manifests, append and read paths, retention, and typed cursors. It does not execute hooks or own a background service.

Index

Constants

View Source
const (
	ContextRunID              = "runId"
	ContextFailedInvocationID = "failedInvocationId"
)

Keys under which a failed command carries run identity in its error context.

View Source
const (
	DefaultReadLimit  = 500
	MaxReadLimit      = 2000
	DefaultIndexLimit = 20
	MaxIndexLimit     = 100
)

Page limits.

View Source
const (
	// MaxSegmentBytes is the size at which a producer rolls to a new segment.
	MaxSegmentBytes = 512 * 1024
	// MaxRunBytes is the retained data per run in one physical store.
	MaxRunBytes = 10 * 1024 * 1024
	// MaxGenerationBytes is the retained data per workspace generation.
	MaxGenerationBytes = 50 * 1024 * 1024
	// MaxCompletedRuns is the completed runs kept per workspace generation.
	MaxCompletedRuns = 20
	// RetentionAge is how long a run is kept after its last update.
	RetentionAge = 30 * 24 * time.Hour
	// MaxStoreBytes is the total retained data per installation.
	MaxStoreBytes = 512 * 1024 * 1024
)

Retention bounds. They count encoded bytes on disk, not text length.

View Source
const (
	ReasonRefreshUnsupported = "refresh_unsupported"
	ReasonSourceOffline      = "source_offline"
	ReasonRefreshFailed      = "refresh_failed"
	ReasonNoSpool            = "no_spool"
)

Availability reasons shared by the readers on both sides of a refresh.

View Source
const (
	// MaxChunkBytes is the largest output text in one record.
	MaxChunkBytes = 8 * 1024
	// MaxPageBytes is the largest encoded read or transport page.
	MaxPageBytes = 256 * 1024
	// MaxMetadataBytes is the largest encoded non-output record.
	MaxMetadataBytes = 64 * 1024
	// MaxProducers is the most producers a run records.
	MaxProducers = 64
	// MaxCursorBytes is the largest encoded cursor.
	MaxCursorBytes = 8 * 1024
	// MaxDetailBytes truncates optional free-text fields.
	MaxDetailBytes = 1024
)

Size bounds. The values are defaults for the first delivery.

View Source
const (
	// AckDirName is the directory beside the runs that holds mirror acknowledgements.
	AckDirName = "mirror"
)
View Source
const SchemaVersion = 1

SchemaVersion is the version written to every manifest, record and response.

View Source
const SpoolDirName = "workspace-logs"

SpoolDirName is the private subdirectory of a container data directory that holds the container-side spool.

Variables

View Source
var (
	ErrInvalidID        = errors.New("invalid identifier")
	ErrInvalidRecord    = errors.New("invalid record")
	ErrNotFound         = errors.New("not found")
	ErrIdentityConflict = errors.New("workspace identity already bound")
	ErrSequenceGap      = errors.New("sequence gap")
	ErrInvalidCursor    = errors.New("invalid cursor")
	ErrInvalidSelector  = errors.New("invalid selector")
	ErrUnsafePath       = errors.New("unsafe storage path")
	ErrCapacity         = errors.New("storage capacity exhausted")
	ErrProducerCap      = errors.New("producer limit reached")
	ErrDiscoverySealed  = errors.New("producer discovery is sealed")
)

Errors returned by the package.

View Source
var ErrSourceGap = errors.New("source records are not retained")

ErrSourceGap means the next records a sender needs are no longer retained at its own store, so mirroring cannot continue past them.

Functions

func DeterministicID

func DeterministicID(parts ...string) string

func FormatUnavailable

func FormatUnavailable(reason string) string

FormatUnavailable returns the stderr line a reader prints before exiting non-zero, so the caller can tell unavailable history from a command error.

func NewID

func NewID() string

NewID returns a new random lower-case UUID for runs, producers and invocations.

func ParseUnavailable

func ParseUnavailable(stderr string) (string, bool)

ParseUnavailable extracts a typed reason from remote stderr. It returns false when stderr carries no marker.

func PositionCursor

func PositionCursor(runID string, pos map[string]uint64) (string, error)

PositionCursor returns an unfiltered run cursor that resumes after pos.

func ResolveSpool

func ResolveSpool() (string, error)

ResolveSpool returns the root of an existing container spool. It checks the primary and the non-root fallback data directories in that order and never creates, chmods or probes anything. ErrNotFound means no spool exists.

func SplitText

func SplitText(text string, maxBytes int) []string

SplitText splits text into chunks of at most max bytes without breaking a UTF-8 sequence. A trailing fragment without a newline is kept.

func ValidHook

func ValidHook(name string) bool

ValidHook reports whether name is an exact lifecycle property name.

func ValidID

func ValidID(id string) bool

ValidID reports whether id is a canonical lower-case UUID. Every identifier that reaches the filesystem passes this check first.

func ValidateReadSelectors

func ValidateReadSelectors(req ReadRequest) error

ValidateReadSelectors checks the values a caller may pass to a reader before any I/O. Every side of a refresh calls it again, so a hand-built command line cannot smuggle a path, a flag or an oversized cursor through.

func WithRecorder

func WithRecorder(ctx context.Context, rec *Recorder) context.Context

WithRecorder returns a context that carries rec so deep call sites can record without new parameters. A nil rec returns ctx unchanged.

Types

type AckKey

type AckKey struct {
	RunID      string
	ProducerID string
	Hop        Hop
}

AckKey identifies one stored position.

type AckState

type AckState struct {
	SchemaVersion int    `json:"schemaVersion"`
	RunID         string `json:"runId"`
	ProducerID    string `json:"producerId"`
	Hop           Hop    `json:"hop"`
	// Acked is the highest sequence the peer persisted or represented as a gap.
	Acked uint64 `json:"acked"`
	// Gapped is set once the peer reported sequences it will never hold.
	Gapped   bool   `json:"gapped,omitempty"`
	GapFirst uint64 `json:"gapFirst,omitempty"`
	GapLast  uint64 `json:"gapLast,omitempty"`
}

AckState is the durable position of one producer on one hop.

type AckStore

type AckStore struct {
	// contains filtered or unexported fields
}

AckStore keeps acknowledged positions in small files beneath one private directory. Positions only move forward.

func NewAckStore

func NewAckStore(dir string) (*AckStore, error)

NewAckStore creates dir (absolute) and returns the store.

func (*AckStore) Advance

func (a *AckStore) Advance(key AckKey, acked uint64) error

Advance moves the acknowledged position forward. A lower value is ignored.

func (*AckStore) Load

func (a *AckStore) Load(key AckKey) (AckState, error)

Load returns the stored position, or a zero position when none exists.

func (*AckStore) RecordGap

func (a *AckStore) RecordGap(key AckKey, first, last uint64) error

RecordGap stores that the peer will never hold first..last and moves the position past the gap so the sender stops retrying those bytes.

type AppendResult

type AppendResult struct {
	// Accepted is the highest contiguous sequence persisted for the producer.
	// Acknowledge this value, never a higher one.
	Accepted   uint64
	Written    int
	Duplicates int
	Elided     int
}

AppendResult reports what an Append call persisted.

type Backend

type Backend interface {
	RegisterProducer(runID string, ref ProducerRef) error
	Append(records []Record) (AppendResult, error)
	AddCondition(runID string, c Condition) error
}

Backend is the persistence a Recorder writes to. *Store implements it.

type BatchRequest

type BatchRequest struct {
	SchemaVersion int
	RunID         string
	ProducerID    string
	FirstSequence uint64
	LastSequence  uint64
	// Records holds JSON encoded Record values.
	Records [][]byte
}

BatchRequest is one contiguous batch of a producer's encoded records.

type BatchResult

type BatchResult struct {
	Result MirrorResult
	// Accepted is the highest contiguous sequence persisted for the producer.
	Accepted uint64
	GapFirst uint64
	GapLast  uint64
	Detail   string
}

BatchResult is the receiver's answer.

type BoundedBuffer

type BoundedBuffer struct {
	Max int
	// contains filtered or unexported fields
}

BoundedBuffer keeps at most Max bytes of what is written and drops the rest. Writes never fail, so it is safe as a diagnostics sink.

func (*BoundedBuffer) String

func (b *BoundedBuffer) String() string

String returns the kept bytes.

func (*BoundedBuffer) Write

func (b *BoundedBuffer) Write(p []byte) (int, error)

Write keeps the head of the stream.

type CaptureState

type CaptureState string

CaptureState describes how much of a run's output is retained.

const (
	CapturePending     CaptureState = "pending"
	CaptureRecording   CaptureState = "recording"
	CaptureComplete    CaptureState = "complete"
	CapturePartial     CaptureState = "partial"
	CaptureUnavailable CaptureState = "unavailable"
)

Capture states.

type CommandState

type CommandState string

CommandState is the outcome of the command that started a run. It is separate from hook and capture completion.

const (
	CommandPending   CommandState = "pending"
	CommandRunning   CommandState = "running"
	CommandSucceeded CommandState = "succeeded"
	CommandFailed    CommandState = "failed"
	CommandCanceled  CommandState = "canceled"
	CommandUnknown   CommandState = "unknown"
)

Command states.

type Condition

type Condition string

Condition explains why capture is not complete. It never replaces the command outcome.

const (
	ConditionTruncated         Condition = "truncated"
	ConditionRetentionGap      Condition = "retention_gap"
	ConditionWriteError        Condition = "write_error"
	ConditionUnsupportedSource Condition = "unsupported_source"
	ConditionSourceUnavailable Condition = "source_unavailable"
	ConditionInterrupted       Condition = "interrupted"
	ConditionCapacity          Condition = "capacity"
	ConditionProducerCap       Condition = "producer_cap"
	ConditionMalformedRecord   Condition = "malformed_record"
)

Capture conditions.

type DiagnosticWriter

type DiagnosticWriter struct {
	// contains filtered or unexported fields
}

DiagnosticWriter adapts a Recorder to io.WriteCloser for one invocation stream. Write always reports success so a failing store cannot change the behavior of the process producing the output.

func (*DiagnosticWriter) Close

func (w *DiagnosticWriter) Close() error

Close flushes buffered text, including a final fragment without a newline.

func (*DiagnosticWriter) Write

func (w *DiagnosticWriter) Write(p []byte) (int, error)

Write redacts p and records it.

type Hop

type Hop string

Hop names one acknowledged position. A host that both receives and forwards keeps both, so durability at the next host is never inferred from durability in its own store.

const (
	// HopInbound is the position persisted in this host's store for a producer
	// that mirrors into it.
	HopInbound Hop = "inbound"
	// HopUpstream is the position the next host acknowledged for records this
	// host sends on.
	HopUpstream Hop = "upstream"
)

Acknowledgement hops.

type ImportRequest

type ImportRequest struct {
	// RunID, Context and WorkspaceUID are the identity the caller expects. The
	// response must match all of them.
	RunID        string
	Context      string
	WorkspaceUID string
	Response     *ReadResponse
}

ImportRequest is one validated remote page to merge into the local store.

type ImportResult

type ImportResult struct {
	Written    int
	Duplicates int
	// Stalled lists producers whose records could not be merged past a gap or
	// a malformed remote record. Their remaining records are not imported.
	Stalled []string
}

ImportResult reports what a merge changed.

type IndexRequest

type IndexRequest struct {
	Context string
	// WorkspaceUID narrows the index to one workspace generation.
	WorkspaceUID string
	After        string
	// Limit is the most runs in the page. Zero means DefaultIndexLimit.
	Limit int
}

IndexRequest selects one page of the run index.

type IndexResponse

type IndexResponse struct {
	SchemaVersion int          `json:"schemaVersion"`
	Runs          []RunSummary `json:"runs"`
	NextCursor    string       `json:"nextCursor,omitempty"`
	HasMore       bool         `json:"hasMore"`
}

IndexResponse is one page of runs, newest first by (createdAt, runId).

type Invocation

type Invocation struct {
	ID       string
	Phase    string
	Hook     string
	Index    *int
	Key      string
	State    InvocationState
	Reason   string
	ExitCode *int
	Signal   string
	Duration time.Duration
}

Invocation describes one hook or build step transition.

type InvocationState

type InvocationState string

InvocationState is the lifecycle state of one hook or build invocation.

const (
	InvocationPending   InvocationState = "pending"
	InvocationDeferred  InvocationState = "deferred"
	InvocationRunning   InvocationState = "running"
	InvocationSucceeded InvocationState = "succeeded"
	InvocationFailed    InvocationState = "failed"
	InvocationSkipped   InvocationState = "skipped"
	InvocationCanceled  InvocationState = "canceled"
	InvocationUnknown   InvocationState = "completion_unknown"
)

Invocation states.

type Kind

type Kind string

Kind identifies the shape of a Record.

const (
	KindProducer   Kind = "producer"
	KindInvocation Kind = "invocation"
	KindOutput     Kind = "output"
	KindCapture    Kind = "capture"
)

Record kinds.

type Location

type Location string

Location is where an invocation executed.

const (
	LocationHost         Location = "host"
	LocationProviderHost Location = "provider_host"
	LocationContainer    Location = "container"
)

Execution locations.

type MirrorResult

type MirrorResult string

MirrorResult is the typed outcome of one batch.

const (
	// MirrorAccepted means records up to Accepted are persisted. Accepted can be
	// lower than the batch's last sequence when the receiver was behind.
	MirrorAccepted MirrorResult = "accepted"
	// MirrorRetentionGap means the receiver will never hold the gap range.
	MirrorRetentionGap MirrorResult = "retention_gap"
	// MirrorRejected means the batch is invalid and must not be retried.
	MirrorRejected MirrorResult = "rejected_invalid"
)

Batch outcomes.

type Options

type Options struct {
	// Root is the absolute storage directory.
	Root string
	// Now overrides the clock in tests.
	Now func() time.Time
}

Options configures a Store.

type ProducerRef

type ProducerRef struct {
	ProducerID       string        `json:"producerId"`
	ParentProducerID string        `json:"parentProducerId,omitempty"`
	State            ProducerState `json:"state"`
	Location         Location      `json:"location,omitempty"`
	Sealed           bool          `json:"discoverySealed,omitempty"`
}

ProducerRef is a run manifest entry for one producer.

type ProducerState

type ProducerState string

ProducerState is the lifecycle state of one producer (a process or worker).

const (
	ProducerReserved    ProducerState = "reserved"
	ProducerRunning     ProducerState = "running"
	ProducerTerminal    ProducerState = "terminal"
	ProducerUnavailable ProducerState = "unavailable"
)

Producer states.

type ReadRequest

type ReadRequest struct {
	RunID        string
	Hook         string
	InvocationID string
	// After is the NextCursor of the previous page.
	After string
	// Limit is the most records in the page. Zero means DefaultReadLimit.
	Limit int
	// MaxBytes caps the encoded page. Zero or less means MaxPageBytes.
	MaxBytes int
}

ReadRequest selects one page of a run.

type ReadResponse

type ReadResponse struct {
	SchemaVersion int                  `json:"schemaVersion"`
	Run           RunSummary           `json:"run"`
	Records       []Record             `json:"records"`
	NextCursor    string               `json:"nextCursor,omitempty"`
	HasMore       bool                 `json:"hasMore"`
	Availability  []SourceAvailability `json:"availability,omitempty"`
	Complete      bool                 `json:"complete"`
}

ReadResponse is one bounded page.

func DecodeReadResponse

func DecodeReadResponse(r io.Reader) (*ReadResponse, error)

DecodeReadResponse decodes one remote page, refusing input larger than MaxPageBytes.

func ReadSpoolPage

func ReadSpoolPage(sel SpoolSelector) (*ReadResponse, error)

ReadSpoolPage returns one bounded page of a spooled run. It validates the selectors again, so a caller cannot read a run of another context or workspace generation.

type Record

type Record struct {
	SchemaVersion int       `json:"schemaVersion"`
	Kind          Kind      `json:"kind"`
	RunID         string    `json:"runId"`
	ProducerID    string    `json:"producerId"`
	Sequence      uint64    `json:"sequence"`
	Time          time.Time `json:"time"`

	// Producer and invocation transitions.
	ParentProducerID string `json:"parentProducerId,omitempty"`
	// ChildProducerID, on a producer record, reserves (or finalizes) a child
	// producer the writer is about to launch. Mirrored stores learn about the
	// child from this record, before any of its own records arrive.
	ChildProducerID string          `json:"childProducerId,omitempty"`
	ProducerState   ProducerState   `json:"producerState,omitempty"`
	DiscoverySealed bool            `json:"discoverySealed,omitempty"`
	InvocationID    string          `json:"invocationId,omitempty"`
	Phase           string          `json:"phase,omitempty"`
	Hook            string          `json:"hook,omitempty"`
	HookIndex       *int            `json:"hookIndex,omitempty"`
	HookKey         string          `json:"hookKey,omitempty"`
	Location        Location        `json:"location,omitempty"`
	State           InvocationState `json:"state,omitempty"`
	Reason          string          `json:"reason,omitempty"`
	DurationMillis  *int64          `json:"durationMillis,omitempty"`
	ExitCode        *int            `json:"exitCode,omitempty"`
	Signal          string          `json:"signal,omitempty"`

	// Output.
	Stream Stream `json:"stream,omitempty"`
	Text   string `json:"text,omitempty"`
	Bytes  int    `json:"bytes,omitempty"`
	// Elided marks output whose text was dropped for capacity. Sequence and
	// Bytes are kept so producers and readers stay contiguous.
	Elided bool `json:"elided,omitempty"`

	// Capture conditions.
	Condition    Condition `json:"condition,omitempty"`
	Detail       string    `json:"detail,omitempty"`
	DroppedBytes int64     `json:"droppedBytes,omitempty"`
}

Record is one immutable entry in a producer's stream. Fields outside the record's Kind are omitted.

func (*Record) Validate

func (r *Record) Validate() error

Validate checks structural bounds that do not depend on stored state.

type Recorder

type Recorder struct {
	// contains filtered or unexported fields
}

Recorder writes one producer's records. It assigns the producer sequence and never fails the caller: the first storage error disables the recorder, is kept for Err, and later calls become no-ops, except that Close still makes one best-effort attempt, so the recorded command's own error and exit semantics are untouched.

func NewRecorder

func NewRecorder(opts RecorderOptions) (*Recorder, error)

NewRecorder registers a new producer in the run and records that it is running. A failure to register returns a recorder that is already disabled together with the error, so callers may ignore the recorder's state.

func RecorderFrom

func RecorderFrom(ctx context.Context) *Recorder

RecorderFrom returns the recorder carried by ctx, or nil. Every Recorder method is safe to skip when this is nil.

func (*Recorder) Capture

func (r *Recorder) Capture(c Condition, detail string, droppedBytes int64)

Capture records a condition that makes the run's capture partial.

func (*Recorder) Close

func (r *Recorder) Close(state ProducerState, sealDiscovery bool)

Close records the producer's final state. Close any DiagnosticWriter first. A disabled recorder still makes one best-effort attempt so the producer does not stay running in the run.

func (*Recorder) Err

func (r *Recorder) Err() error

Err returns the first storage error, if any.

func (*Recorder) Invocation

func (r *Recorder) Invocation(inv Invocation)

Invocation records a transition of one invocation, including silent ones.

func (*Recorder) MarkChild

func (r *Recorder) MarkChild(id string, state ProducerState, reason string)

MarkChild records a factual outcome for a reserved child, for example ProducerUnavailable when its launch was not observed. Reason is shown to readers.

func (*Recorder) ProducerID

func (r *Recorder) ProducerID() string

ProducerID returns the identity of this producer.

func (*Recorder) ReserveChild

func (r *Recorder) ReserveChild(location Location) string

ReserveChild records that a child producer will be launched and returns its ID. Pass the ID to the child as RecorderOptions.ProducerID. Reserve every child before sealing discovery.

func (*Recorder) RunID

func (r *Recorder) RunID() string

RunID returns the run this recorder writes to.

func (*Recorder) SealDiscovery

func (r *Recorder) SealDiscovery()

SealDiscovery records that this producer has reserved every child it will launch.

func (*Recorder) Writer

func (r *Recorder) Writer(invocationID string, stream Stream) *DiagnosticWriter

Writer returns a writer for one invocation's stream. Each writer redacts on its own, so a secret split across writes is masked and streams do not mix.

func (*Recorder) WriterRedacting

func (r *Recorder) WriterRedacting(
	invocationID string,
	stream Stream,
	extra *secrets.Redactor,
) *DiagnosticWriter

WriterRedacting is Writer with extra values masked on top of the recorder's own redactor.

type RecorderOptions

type RecorderOptions struct {
	Backend  Backend
	RunID    string
	Location Location
	// ProducerID uses a producer ID its parent reserved earlier. Empty
	// generates one.
	ProducerID string
	// Resumed marks a worker that continues a run whose launching process
	// already recorded the plan. Plan-time transitions are not repeated.
	Resumed bool
	// ParentProducerID links this producer to the one that launched it.
	ParentProducerID string
	// Redactor masks known secrets before anything is persisted. Nil keeps text
	// as written.
	Redactor *secrets.Redactor
	// Now overrides the clock in tests.
	Now func() time.Time
}

RecorderOptions configures a Recorder.

type RunManifest

type RunManifest struct {
	SchemaVersion      int           `json:"schemaVersion"`
	RunID              string        `json:"runId"`
	Context            string        `json:"context"`
	WorkspaceUID       *string       `json:"workspaceUid"`
	WorkspaceIDAtStart string        `json:"workspaceIdAtStart,omitempty"`
	Command            string        `json:"command"`
	TaskID             string        `json:"taskId,omitempty"`
	CreatedAt          time.Time     `json:"createdAt"`
	UpdatedAt          time.Time     `json:"updatedAt"`
	CommandState       CommandState  `json:"commandState"`
	CaptureState       CaptureState  `json:"captureState"`
	Conditions         []Condition   `json:"conditions,omitempty"`
	Producers          []ProducerRef `json:"producers,omitempty"`
	DiscoverySealed    bool          `json:"discoverySealed,omitempty"`
}

RunManifest is the per-run metadata file.

func (*RunManifest) Active

func (m *RunManifest) Active() bool

Active reports whether a writer may still append to the run: discovery is open or a producer has not reached a final state.

func (*RunManifest) Complete

func (m *RunManifest) Complete() bool

Complete reports whether every selected record has been captured: discovery is sealed, every producer is terminal and capture has no gaps.

func (*RunManifest) Summary

func (m *RunManifest) Summary() RunSummary

Summary returns the bounded view of m.

type RunSpec

type RunSpec struct {
	// RunID joins an existing run created by another process. Empty allocates
	// a new run.
	RunID              string
	Context            string
	WorkspaceUID       string
	WorkspaceIDAtStart string
	Command            string
	TaskID             string
}

RunSpec describes a run at creation. WorkspaceUID may stay empty until the workspace is resolved and bound.

type RunSummary

type RunSummary struct {
	RunID              string       `json:"runId"`
	Context            string       `json:"context"`
	WorkspaceUID       *string      `json:"workspaceUid"`
	WorkspaceIDAtStart string       `json:"workspaceIdAtStart,omitempty"`
	Command            string       `json:"command"`
	TaskID             string       `json:"taskId,omitempty"`
	CreatedAt          time.Time    `json:"createdAt"`
	CommandState       CommandState `json:"commandState"`
	CaptureState       CaptureState `json:"captureState"`
	Conditions         []Condition  `json:"conditions,omitempty"`
}

RunSummary is the bounded run description returned by readers.

type Session

type Session struct {
	Store    *Store
	Recorder *Recorder
	RunID    string
	// contains filtered or unexported fields
}

Session is one process's handle on a run: the local store, the run, and a recorder for this process. A nil Session is valid and does nothing, so storage problems never change the command being recorded.

func StartSession

func StartSession(opts SessionOptions) *Session

StartSession opens the default store, creates or joins the run and starts a recorder. It returns nil when history cannot be recorded.

func StartSessionIn

func StartSessionIn(store *Store, opts SessionOptions) *Session

StartSessionIn is StartSession against an explicit store, for processes that keep their spool outside the default state directory.

func (*Session) BindWorkspace

func (s *Session) BindWorkspace(uid, idAtStart string)

BindWorkspace binds the resolved workspace UID to the run.

func (*Session) Context

func (s *Session) Context(ctx context.Context) context.Context

Context returns ctx carrying this session's recorder.

func (*Session) FailureContext

func (s *Session) FailureContext() map[string]string

FailureContext is the Session form of Store.FailureContext. A nil Session has no history to point at.

func (*Session) Finish

func (s *Session) Finish(err error, sealDiscovery bool)

Finish records the command outcome and closes this process's producer. Pass sealDiscovery only from the process that schedules no further producers. That process alone writes the command state.

func (*Session) ProducerID

func (s *Session) ProducerID() string

ProducerID returns this process's producer ID, or empty without a session.

type SessionOptions

type SessionOptions struct {
	// RunID joins a run allocated earlier (a detached submission or a parent
	// process). Empty allocates a new run.
	RunID string
	// ParentProducerID links this process's producer to the one that
	// launched it.
	ParentProducerID string
	// ProducerID uses a producer ID reserved by the parent process.
	ProducerID string
	// Resumed marks a worker continuing a run whose plan the parent recorded.
	Resumed  bool
	Run      RunSpec
	Location Location
	Redactor *secrets.Redactor
}

SessionOptions describes the run an entry point records into.

type Sink

type Sink struct {
	// contains filtered or unexported fields
}

Sink validates incoming batches against the expected run and persists them idempotently. It acknowledges only what reached its store or was recorded as a gap.

func NewSink

func NewSink(opts SinkOptions) (*Sink, error)

NewSink creates or joins the expected run and returns its sink.

func (*Sink) Append

func (s *Sink) Append(req BatchRequest) (BatchResult, error)

Append validates and persists one batch. A non-nil error is a transient storage failure the sender may retry.

func (*Sink) RunID

func (s *Sink) RunID() string

RunID returns the run this sink accepts.

type SinkOptions

type SinkOptions struct {
	Store *Store
	// Spec identifies the run this host expects. RunID is required; the run is
	// created or joined at construction.
	Spec RunSpec
	// Acks persists the inbound position and retention gaps. Optional.
	Acks *AckStore
	// OnAccepted is called after records are persisted, so a forwarding host can
	// queue them for the next hop. It must not block.
	OnAccepted func(producerID string, accepted uint64)
}

SinkOptions configures a Sink.

type SourceAvailability

type SourceAvailability struct {
	ProducerID string `json:"producerId"`
	Available  bool   `json:"available"`
	Reason     string `json:"reason,omitempty"`
}

SourceAvailability tells whether a producer's records can be read.

type SpoolSelector

type SpoolSelector struct {
	// Root is the validated spool root from ResolveSpool.
	Root string
	// Context and WorkspaceUID must match the stored run. An empty UID skips
	// the UID check only when the run is not bound to one.
	Context      string
	WorkspaceUID string
	Read         ReadRequest
}

SpoolSelector selects one page of a container-side read.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is a file-backed archive of workspace run output. It is safe for use by several goroutines and several processes sharing one Root.

func Open

func Open(opts Options) (*Store, error)

Open creates the storage root if needed and returns a Store. It also removes expired runs.

func OpenDefault

func OpenDefault() (*Store, error)

OpenDefault opens the store at the PathManager workspace logs directory.

func (*Store) AddCondition

func (s *Store) AddCondition(runID string, c Condition) error

AddCondition records a capture condition and marks capture partial.

func (*Store) Append

func (s *Store) Append(records []Record) (AppendResult, error)

Append persists a batch of records from one producer. Records at or below the producer's last persisted sequence are duplicates and are skipped, so a retransmitted batch never repeats output. A batch that skips ahead of the next expected sequence returns ErrSequenceGap with Accepted unchanged.

func (*Store) BindWorkspace

func (s *Store) BindWorkspace(runID, uid, idAtStart string) error

BindWorkspace binds the resolved workspace UID to a provisional run. Binding the same UID again is a no-op; a different UID is rejected.

func (*Store) CreateRun

func (s *Store) CreateRun(spec RunSpec) (*RunManifest, error)

CreateRun creates a run, or joins the run named by spec.RunID. Joining requires the stored context (and workspace UID, when both are set) to match.

func (*Store) FailedInvocations

func (s *Store) FailedInvocations(runID string) ([]string, error)

FailedInvocations returns the IDs of invocations whose latest recorded state is failed, in order of first appearance. At most MaxProducers IDs are returned; callers only use an ID when exactly one invocation failed.

func (*Store) FailureContext

func (s *Store) FailureContext(runID string) map[string]string

FailureContext returns the error context that identifies a failed run: the run ID, plus the failed invocation when exactly one invocation failed. Lookup problems yield whatever identity is known.

func (*Store) ImportRemote

func (s *Store) ImportRemote(req ImportRequest) (ImportResult, error)

ImportRemote merges a remote page into the local run. Records are deduplicated by producer and sequence. It rejects a response for another run, context or workspace generation, oversized pages and invalid records. Nothing is written when validation fails.

func (*Store) ListRuns

func (s *Store) ListRuns(req IndexRequest) (*IndexResponse, error)

ListRuns returns retained runs for a context, including runs whose workspace no longer exists.

func (*Store) LocalPositions

func (s *Store) LocalPositions(runID string) (map[string]uint64, error)

LocalPositions returns the last committed sequence of every producer of the run. A refresh asks the remote side only for records after these positions.

func (*Store) Manifest

func (s *Store) Manifest(runID string) (*RunManifest, error)

Manifest returns a copy of the run manifest.

func (*Store) Prune

func (s *Store) Prune() error

Prune removes expired and surplus completed runs. It never removes a run with an active writer, unless the run has not been updated within RetentionAge (a crashed writer). Another process pruning at the same time makes this a no-op.

func (*Store) Read

func (s *Store) Read(req ReadRequest) (*ReadResponse, error)

Read returns the next page of a run. Output is read from disk in pages; the archive is never loaded whole.

func (*Store) RegisterProducer

func (s *Store) RegisterProducer(runID string, ref ProducerRef) error

RegisterProducer reserves a producer in the run. Registering a known producer is a no-op. Overflowing MaxProducers marks capture partial and returns ErrProducerCap; execution is not affected.

func (*Store) Root

func (s *Store) Root() string

Root returns the storage directory.

func (*Store) SealDiscovery

func (s *Store) SealDiscovery(runID string) error

SealDiscovery closes producer discovery. Call it after every scheduler has recorded its child reservations or exclusions.

func (*Store) SetCommandState

func (s *Store) SetCommandState(runID string, state CommandState) error

SetCommandState records the originating command's outcome.

func (*Store) SetTaskID

func (s *Store) SetTaskID(runID, taskID string) error

SetTaskID associates a background task with a run created before the task existed.

type StoreSource

type StoreSource struct {
	// contains filtered or unexported fields
}

StoreSource reads a producer's committed records back from a Store in bounded batches for mirroring.

func NewStoreSource

func NewStoreSource(store *Store) *StoreSource

NewStoreSource returns a source over store.

func (*StoreSource) ReadAfter

func (s *StoreSource) ReadAfter(
	runID, producerID string, after uint64, maxBytes int,
) ([]Record, error)

ReadAfter returns the contiguous records of one producer after sequence after, at most maxBytes when JSON encoded. It returns ErrSourceGap when the first record needed is missing or unreadable.

type Stream

type Stream string

Stream names the origin of output text.

const (
	StreamStdout   Stream = "stdout"
	StreamStderr   Stream = "stderr"
	StreamCombined Stream = "combined"
)

Output streams. StreamCombined is only for upstream producers that already merge stdout and stderr.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL