overwatch

package
v0.5.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 2, 2026 License: Apache-2.0 Imports: 7 Imported by: 0

Documentation

Overview

Package overwatch provides real-time execution monitoring capabilities for the Superbrain system. It implements the Overwatch Layer which wraps CLI executions with monitoring, heartbeat detection, and silence threshold detection to identify hung processes.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DefaultOverwatchConfig

func DefaultOverwatchConfig() *config.OverwatchConfig

DefaultOverwatchConfig returns the default Overwatch configuration.

Types

type Monitor

type Monitor struct {
	// contains filtered or unexported fields
}

Monitor provides real-time monitoring of CLI process execution. It wraps stdout/stderr streams, detects silence, and emits diagnostic snapshots.

func NewMonitor

func NewMonitor(cfg *config.OverwatchConfig, onSnapshot SnapshotHandler) *Monitor

NewMonitor creates a new Monitor with the given configuration.

func (*Monitor) ActiveContextCount

func (m *Monitor) ActiveContextCount() int

ActiveContextCount returns the number of currently monitored executions.

func (*Monitor) GetContext

func (m *Monitor) GetContext(requestID string) *OverwatchContext

GetContext returns the monitoring context for a request ID.

func (*Monitor) StartMonitoring

func (m *Monitor) StartMonitoring(processID int, provider, model, requestID string) *OverwatchContext

StartMonitoring begins monitoring a process execution. It returns an OverwatchContext that tracks the execution state.

func (*Monitor) Stop

func (m *Monitor) Stop()

Stop stops all monitoring and cleans up resources.

func (*Monitor) StopMonitoring

func (m *Monitor) StopMonitoring(requestID string)

StopMonitoring stops monitoring for a specific request.

func (*Monitor) WrapReader

func (m *Monitor) WrapReader(reader io.Reader, ctx *OverwatchContext) *MonitoredReader

WrapReader creates a MonitoredReader that tracks output from the given reader.

func (*Monitor) WrapWriter

func (m *Monitor) WrapWriter(writer io.Writer, ctx *OverwatchContext) *MonitoredWriter

WrapWriter creates a MonitoredWriter that tracks output written to the given writer.

type MonitoredReader

type MonitoredReader struct {
	// contains filtered or unexported fields
}

MonitoredReader wraps an io.Reader to track output and update the monitoring context.

func (*MonitoredReader) Read

func (mr *MonitoredReader) Read(p []byte) (n int, err error)

Read implements io.Reader, recording output to the monitoring context.

func (*MonitoredReader) ReadLine

func (mr *MonitoredReader) ReadLine() (string, bool, error)

ReadLine reads a single line from the reader and records it. Returns the line, whether more data is available, and any error.

type MonitoredWriter

type MonitoredWriter struct {
	// contains filtered or unexported fields
}

MonitoredWriter wraps an io.Writer to track output and update the monitoring context.

func (*MonitoredWriter) Write

func (mw *MonitoredWriter) Write(p []byte) (n int, err error)

Write implements io.Writer, recording output to the monitoring context.

type Overwatch

type Overwatch interface {
	// StartMonitoring begins watching an execution.
	StartMonitoring(processID int, provider, model, requestID string) *OverwatchContext

	// StopMonitoring stops watching an execution.
	StopMonitoring(requestID string)

	// GetContext returns the monitoring context for a request.
	GetContext(requestID string) *OverwatchContext

	// WrapReader creates a monitored reader.
	WrapReader(reader io.Reader, ctx *OverwatchContext) *MonitoredReader

	// WrapWriter creates a monitored writer.
	WrapWriter(writer io.Writer, ctx *OverwatchContext) *MonitoredWriter

	// Stop stops all monitoring.
	Stop()
}

Overwatch is the main interface for the Overwatch Layer. It provides methods for starting monitoring, checking health, and capturing snapshots.

type OverwatchContext

type OverwatchContext struct {

	// ProcessID is the OS process ID of the monitored process.
	ProcessID int

	// StartTime is when the process was spawned.
	StartTime time.Time

	// LastOutputTime is the timestamp of the most recent stdout/stderr output.
	// Used for heartbeat detection and silence threshold calculation.
	LastOutputTime time.Time

	// LogBuffer stores recent log lines for diagnostic snapshots.
	LogBuffer *RingBuffer

	// RestartCount tracks how many times this execution has been restarted.
	RestartCount int

	// DiagnosticCount tracks how many diagnostic snapshots have been captured.
	DiagnosticCount int

	// HealingActions records all autonomous interventions taken during this execution.
	HealingActions []types.HealingAction

	// Config holds the Overwatch configuration parameters.
	Config *config.OverwatchConfig

	// Provider is the provider being executed (e.g., "claudecli", "geminicli").
	Provider string

	// Model is the model being used for this execution.
	Model string

	// RequestID uniquely identifies the request that triggered this execution.
	RequestID string

	// IsSilent indicates whether the process is currently in a silent state.
	IsSilent bool

	// SilenceStartTime is when the current silence period began (if IsSilent is true).
	SilenceStartTime time.Time
	// contains filtered or unexported fields
}

OverwatchContext holds the monitoring state for a single CLI execution. It tracks process health, output timing, and healing actions taken during the execution lifecycle.

func NewOverwatchContext

func NewOverwatchContext(processID int, cfg *config.OverwatchConfig, provider, model, requestID string) *OverwatchContext

NewOverwatchContext creates a new monitoring context for a process execution. It initializes the log buffer and sets the start time.

func (*OverwatchContext) CanRestart

func (ctx *OverwatchContext) CanRestart() bool

CanRestart returns true if another restart attempt is allowed.

func (*OverwatchContext) CaptureSnapshot

func (ctx *OverwatchContext) CaptureSnapshot(processState string) *types.DiagnosticSnapshot

CaptureSnapshot creates a diagnostic snapshot of the current execution state. This is called when the silence threshold is exceeded to gather diagnostic information.

func (*OverwatchContext) CheckHeartbeat

func (ctx *OverwatchContext) CheckHeartbeat() bool

CheckHeartbeat checks if the process is healthy based on output timing. Returns true if output was received within the silence threshold, false otherwise.

func (*OverwatchContext) GetElapsedTime

func (ctx *OverwatchContext) GetElapsedTime() time.Duration

GetElapsedTime returns the total time elapsed since process start.

func (*OverwatchContext) GetElapsedTimeMs

func (ctx *OverwatchContext) GetElapsedTimeMs() int64

GetElapsedTimeMs returns the total time elapsed since process start in milliseconds.

func (*OverwatchContext) GetHealingActions

func (ctx *OverwatchContext) GetHealingActions() []types.HealingAction

GetHealingActions returns a copy of all healing actions recorded.

func (*OverwatchContext) GetHealingMetadata

func (ctx *OverwatchContext) GetHealingMetadata() *types.HealingMetadata

GetHealingMetadata creates a HealingMetadata struct from the context's recorded actions.

func (*OverwatchContext) GetLastLogLines

func (ctx *OverwatchContext) GetLastLogLines(n int) []string

GetLastLogLines returns the last n log lines from the buffer.

func (*OverwatchContext) GetSilenceDuration

func (ctx *OverwatchContext) GetSilenceDuration() time.Duration

GetSilenceDuration returns how long the process has been silent.

func (*OverwatchContext) IncrementDiagnosticCount

func (ctx *OverwatchContext) IncrementDiagnosticCount() int

IncrementDiagnosticCount increments the diagnostic counter and returns the new count.

func (*OverwatchContext) IncrementRestartCount

func (ctx *OverwatchContext) IncrementRestartCount() int

IncrementRestartCount increments the restart counter and returns the new count.

func (*OverwatchContext) MarkSilent

func (ctx *OverwatchContext) MarkSilent()

MarkSilent marks the process as being in a silent state. This is called when the silence threshold is first exceeded.

func (*OverwatchContext) RecordHealingAction

func (ctx *OverwatchContext) RecordHealingAction(action types.HealingAction)

RecordHealingAction adds a healing action to the context's history.

func (*OverwatchContext) RecordOutput

func (ctx *OverwatchContext) RecordOutput(line string)

RecordOutput records that output was received from the process. This updates the LastOutputTime for heartbeat tracking and clears the silent state.

func (*OverwatchContext) ResetForRestart

func (ctx *OverwatchContext) ResetForRestart(newPID int)

ResetForRestart resets timing state for a new restart attempt. Preserves restart count, healing actions, and configuration.

func (*OverwatchContext) UpdateProcessID

func (ctx *OverwatchContext) UpdateProcessID(pid int)

UpdateProcessID updates the process ID (used after restart).

type RingBuffer

type RingBuffer struct {
	// contains filtered or unexported fields
}

RingBuffer is a thread-safe circular buffer for storing log lines. It maintains a fixed-size buffer that overwrites the oldest entries when full, ensuring memory-bounded log retention for diagnostic purposes.

func NewRingBuffer

func NewRingBuffer(size int) *RingBuffer

NewRingBuffer creates a new RingBuffer with the specified capacity. If size is <= 0, a default size of 50 is used.

func (*RingBuffer) Cap

func (rb *RingBuffer) Cap() int

Cap returns the maximum capacity of the buffer.

func (*RingBuffer) Clear

func (rb *RingBuffer) Clear()

Clear removes all lines from the buffer. This operation is thread-safe.

func (*RingBuffer) GetAll

func (rb *RingBuffer) GetAll() []string

GetAll returns all lines currently in the buffer in chronological order. This operation is thread-safe.

func (*RingBuffer) GetLast

func (rb *RingBuffer) GetLast(n int) []string

GetLast returns the last n lines from the buffer in chronological order. If n is greater than the number of stored lines, all stored lines are returned. If n is <= 0, an empty slice is returned. This operation is thread-safe.

func (*RingBuffer) IsFull

func (rb *RingBuffer) IsFull() bool

IsFull returns true if the buffer has reached capacity and is overwriting old entries. This operation is thread-safe.

func (*RingBuffer) Len

func (rb *RingBuffer) Len() int

Len returns the current number of lines in the buffer. This operation is thread-safe.

func (*RingBuffer) Write

func (rb *RingBuffer) Write(line string)

Write adds a line to the buffer. If the buffer is full, the oldest line is overwritten. This operation is thread-safe.

type SnapshotHandler

type SnapshotHandler func(ctx *OverwatchContext, snapshot *types.DiagnosticSnapshot)

SnapshotHandler is called when a diagnostic snapshot is captured due to silence detection.

type StreamMonitor

type StreamMonitor struct {
	// contains filtered or unexported fields
}

StreamMonitor provides line-by-line monitoring of stdout and stderr streams.

func NewStreamMonitor

func NewStreamMonitor(ctx *OverwatchContext) *StreamMonitor

NewStreamMonitor creates a StreamMonitor for the given context.

func (*StreamMonitor) MonitorStreams

func (sm *StreamMonitor) MonitorStreams(
	bgCtx context.Context,
	stdout, stderr io.Reader,
	onStdout, onStderr func(line string),
)

MonitorStreams starts monitoring stdout and stderr streams concurrently. It reads lines from each stream and records them in the context. The provided callbacks are called for each line read.

func (*StreamMonitor) Stop

func (sm *StreamMonitor) Stop()

Stop cancels stream monitoring.

func (*StreamMonitor) Wait

func (sm *StreamMonitor) Wait()

Wait blocks until both stdout and stderr monitoring complete.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL