watchdog

type Alert ¶

type Alert struct {
	Signal   string
	Severity Severity
	Reason   string
	Guidance string
}

Alert is what a triggered signal returns. Signal is the stable string ID the rest of the system can dispatch on (future "auto" mode picks behavior per signal).

Reason and Guidance are the same observation written for two different readers, and the split matters because the readers can do different things about it. Reason is operator-facing — the agent's wiring logs it verbatim and the attach guardrail endpoint serves it — so it names operator affordances: the interrupt endpoint, the budget ceiling, the reset. Guidance is model-facing: it is what FormatFeedback injects into the session's next prompt under ModeFeedback, so it says what the model itself can do about the pattern and names no affordance the model does not have. Telling a looping model to "POST /sessions/{id}/interrupt" is at best noise and at worst an invitation to hallucinate a tool call for it.

Guidance is optional. A signal that leaves it empty still feeds — FormatFeedback falls back to Reason — because a third-party signal silently producing no feedback would be a worse failure than one producing operator-flavored feedback.

func (a Alert) String() string

type AlternatingCycleSignal ¶ added in v0.4.0

type AlternatingCycleSignal struct {
	// MaxPeriod is the longest cycle considered. Periods start at 2 —
	// period 1 is a consecutive repeat, which RepeatedToolCallSignal
	// owns.
	MaxPeriod int
	// Cycles is how many consecutive repetitions of the block are
	// required before the signal trips.
	Cycles int
	// contains filtered or unexported fields
}

AlternatingCycleSignal trips when the agent repeats the same short *sequence* of tool calls over and over — a → b → a → b → a → b (#649).

This is the evasion the consecutive-repeat detector named in its own docstring and could not catch: "consecutive" means a run of one call, so any loop with a second call wedged in it reads as varied activity. The shape is not hypothetical. The live UAT that motivated upstream's enforce-mode backstop (#623) was an agent cycling list_agents → check_agent, and it survived both an operator "stop" and an interrupt.

Detection is a period scan over the recent call history: for each period p in 2..MaxPeriod, the last p*Cycles observations trip the signal when they consist of Cycles byte-identical blocks of length p. Keys are canonicalized (canonicalArgs), so a cycle that alternates "./main.go" with "main.go" still reads as one call — but unlike the repeat detector this cannot use the pairwise path-suffix relation, which is not transitive and therefore cannot key a history buffer.

Two deliberate limits on false positives, because this alert is meant to be actionable rather than ambient:

• A block whose entries are all the same call is skipped. It is a plain repeat, already the other signal's job, and alerting twice for one pattern doubles the noise.
• Cycles defaults to 3, i.e. six calls for the a→b→a→b→a→b shape. Two passes through a sequence is normal work — read a file, list pods, read another, list again. Three passes with byte-identical arguments each time is not: nothing in the inputs changed, so nothing in the results can have.

A polling loop written as alternating tool calls is the known false positive, and it is a shape mast's own workloads produce — a scheduler-driven daemon watching a rollout settle looks exactly like this. An operator who wants the pattern anyway drops the signal by constructing DefaultWatchdog with their own signal list — which is the right answer for that workload, because the signal is Critical and the cost of the false positive therefore scales with the posture: one log line under warn, a paragraph of unwanted steering under feedback, a halted rollout watch under enforce.

func NewAlternatingCycleSignal(maxPeriod, cycles int) *AlternatingCycleSignal

func (s *AlternatingCycleSignal) Name() string

func (s *AlternatingCycleSignal) ObserveToolCall(tc ToolCall) *Alert

func (s *AlternatingCycleSignal) Reset()

type DefaultWatchdog ¶

type DefaultWatchdog struct {
	// contains filtered or unexported fields
}

DefaultWatchdog is the package-default implementation. Fans observations across the configured signals; Check collects alerts from each.

func NewDefaultWatchdog() *DefaultWatchdog

func (w *DefaultWatchdog) Check() []Alert

func (w *DefaultWatchdog) ObserveToolCall(tc ToolCall)

func (w *DefaultWatchdog) ObserveToolResult(tr ToolResult)

func (w *DefaultWatchdog) Reset()

type Enforcer ¶ added in v0.4.0

type Enforcer struct {
	// contains filtered or unexported fields
}

Enforcer holds one session's halt state.

One per session, alongside the Watchdog itself: the signals count across turns, and so does the trip. A daemon-global enforcer would let one runaway session refuse turns for every other one.

Safe for concurrent use. The alert path runs from the event tap while an attach handler can be reading Tripped or calling Reset.

func NewEnforcer(mode Mode, remedy string) *Enforcer

func (e *Enforcer) Adopt(signal, reason string) bool

func (e *Enforcer) Mode() Mode

func (e *Enforcer) Observe(a Alert) bool

func (e *Enforcer) Preflight() error

func (e *Enforcer) Reset()

func (e *Enforcer) Tripped() (bool, string)

type Feedback ¶ added in v0.4.0

type Feedback struct {
	// contains filtered or unexported fields
}

Feedback holds one session's queue of observations awaiting delivery to the model.

One per session, alongside the Watchdog and the Enforcer: an observation about one session's loop is meaningless prepended to another's prompt.

Safe for concurrent use. Alerts arrive from the event tap while the turn path may be draining the queue.

func NewFeedback(mode Mode) *Feedback

func (f *Feedback) Drain() []Alert

func (f *Feedback) Pending() int

func (f *Feedback) Queue(alerts []Alert)

type Mode ¶ added in v0.4.0

type Mode string

Mode is the watchdog's posture: what the deployment does when a signal trips.

The postures are a ladder, and each rung includes the one below it: warn logs, feedback also tells the model, enforce also stops it.

Detection is identical in all three. Every signal observes, tallies, and alerts the same way — the mode only decides the reaction, which is why a workload can be switched between them without changing what gets found.

const (
	// ModeWarn logs the alert and lets the turn run. The bottom rung, and
	// what an empty value parses to — but not what a host picks when
	// nobody chose; see DefaultMode for why those are different
	// questions.
	ModeWarn Mode = "warn"

	// ModeFeedback warns, and additionally routes each alert's Guidance
	// into the session's next prompt. The party that can stop making the
	// looping call is the model making it, and under warn alone it is
	// the one party never told.
	ModeFeedback Mode = "feedback"

	// ModeEnforce feeds back, and additionally halts on a Critical
	// alert: the turn in flight is cancelled and every subsequent turn
	// is refused until an operator resets, the same contract the budget
	// ceiling keeps.
	//
	// Enforce implies feedback on purpose. Without it, the turn after a
	// reset starts with the model knowing nothing about why it was
	// stopped, which is a treadmill: loop, halt, reset, loop.
	ModeEnforce Mode = "enforce"
)

func ParseMode(s string) (Mode, error)

func (m Mode) Enforces() bool

func (m Mode) Feeds() bool

type RepeatedToolCallSignal ¶

type RepeatedToolCallSignal struct {
	Threshold int
	// contains filtered or unexported fields
}

RepeatedToolCallSignal trips when the same (name, args) tool call appears Threshold times consecutively. Catches the read_file loop pattern from issue #144 and similar runaway-tool-call shapes.

"Consecutive" is the key word: a → b → a → b → a doesn't trip (no run of identical calls), but a → a → a → a → a does. This matches operator intuition ("the agent is stuck on the same thing") without flagging legitimate patterns like alternating-tool exploration loops.

Args comparison is path-canonicalized (#649, see canonical.go), not literal-string as in v1: "main.go", "./main.go" and "/workspace/main.go" are one call, because an agent re-reading one file under three spellings is as stuck as one re-reading it under the same spelling. Everything else still compares exactly — a detector that generalizes too eagerly flags legitimate work.

func NewRepeatedToolCallSignal(threshold int) *RepeatedToolCallSignal

func (s *RepeatedToolCallSignal) Name() string

func (s *RepeatedToolCallSignal) ObserveToolCall(tc ToolCall) *Alert

func (s *RepeatedToolCallSignal) Reset()

type Severity ¶

type Severity string

Severity classifies the urgency of an alert. Warn is operator- visible but never action-blocking. Critical marks a runaway the deployment should act on: under ModeEnforce a Critical alert halts the turn in flight and refuses the next one until an operator resets (see enforce.go), while under ModeWarn it is logged like any other alert.

Severity is an intrinsic property of the pattern, not of the wiring. A tool loop is a runaway whether or not anything is configured to stop it, so the signal reports Critical either way and the mode decides the reaction.

const (
	SeverityWarn     Severity = "warn"
	SeverityCritical Severity = "critical"
)

type Signal ¶

type Signal interface {
	// Name returns the stable signal ID used in Alert.Signal.
	Name() string

	// ObserveToolCall updates the signal's internal state with one
	// tool invocation. Returning a non-nil Alert means the signal
	// tripped on this observation; DefaultWatchdog appends it to
	// the pending-alerts buffer.
	ObserveToolCall(ToolCall) *Alert

	// Reset clears the signal's state. Called from
	// DefaultWatchdog.Reset.
	Reset()
}

Signal is the per-detector interface inside DefaultWatchdog. Each signal owns its own state and decides when to emit an alert. Implementations must be safe to call serially from DefaultWatchdog (which holds a mutex across observations); they do NOT need to be concurrency-safe themselves.

Adding a new signal: implement Signal, append to NewDefaultWatchdog's signal list (or to a constructor variant). No changes to DefaultWatchdog itself.

type SignalResultObserver ¶ added in v0.4.0

type SignalResultObserver interface {
	ObserveToolResult(ToolResult) *Alert
}

SignalResultObserver is the same extension one level down, for signals inside DefaultWatchdog. A Signal that doesn't implement it simply never sees results.

type ToolCall ¶

type ToolCall struct {
	Name string
	Args string
}

ToolCall is the per-tool-call observation the watchdog needs. Name is the canonical tool name (e.g. "read_file", "mcp.gke.list_clusters"). Args is the JSON-serialized argument blob, passed through as the caller produced it: the detectors canonicalize path-shaped values themselves (#649, canonical.go) rather than requiring every caller to agree on a normal form, and a third-party Watchdog implementation still sees the raw args.

type ToolFailureStreakSignal ¶ added in v0.4.0

type ToolFailureStreakSignal struct {
	Threshold int
	// contains filtered or unexported fields
}

ToolFailureStreakSignal trips when Threshold tool calls in a row all return errors. Any successful call resets the run — the point is "this agent currently has no verified evidence," and one success is evidence.

Severity is Warn, and stays Warn when mast grows a posture that acts on Critical. Halting a daemon three denials into a legitimate RBAC probe would make the backstop the outage. A failure streak is an evidence problem, so it goes where evidence problems belong — the operator log, and the session's guardrail surface. Runaway *behavior* is the loop detectors' business.

func NewToolFailureStreakSignal(threshold int) *ToolFailureStreakSignal

func (s *ToolFailureStreakSignal) Name() string

func (s *ToolFailureStreakSignal) ObserveToolCall(ToolCall) *Alert

func (s *ToolFailureStreakSignal) ObserveToolResult(tr ToolResult) *Alert

func (s *ToolFailureStreakSignal) Reset()

type ToolResult ¶ added in v0.4.0

type ToolResult struct {
	Name  string
	Error string
}

ToolResult is the outcome half of a tool invocation. Error is the tool's error text, empty for success — the ADK convention is a reserved "error" key inside FunctionResponse.Response, and the session-event bridge flattens that to this field so the watchdog never has to know the provider's response shape.

func (r ToolResult) Failed() bool

type ToolResultObserver ¶ added in v0.4.0

type ToolResultObserver interface {
	ObserveToolResult(ToolResult)
}

ToolResultObserver is the optional half of Watchdog: an implementation that also wants to see tool *outcomes* implements it, and the bridge feeds results through a type assertion.

Deliberately not folded into Watchdog. That interface is documented as a plug-in point ("consumers can plug in their own implementation"), so widening it would break every third-party watchdog at a minor version to add one signal — and a custom watchdog that only counts calls stays perfectly valid.

type TrippedError ¶ added in v0.4.0

type TrippedError struct {
	// Signal is the alert that halted the session.
	Signal string
	// Reason is the operator-facing text, alert included.
	Reason string
}

TrippedError is what a caller returns from a turn the watchdog halted, and what Preflight returns for every turn after it. A distinct type so a host can tell "an operator must reset this" apart from a failure worth retrying: retrying a watchdog trip re-drives the loop that caused it, which is the exact failure enforce mode exists to break.

func (e *TrippedError) Error() string

type Watchdog ¶

type Watchdog interface {
	// ObserveToolCall records one tool invocation. Called by the
	// agent's event-tap as tool calls stream by; safe to call from
	// any goroutine.
	ObserveToolCall(ToolCall)

	// Check returns alerts triggered since the last Check call and
	// resets the per-call alert buffer. Returns nil when no signal
	// has tripped. Typically called from the agent's post-turn
	// hook; an alert returned here is "for the turn just ended."
	Check() []Alert

	// Reset clears all accumulated state. Called when the agent
	// resets (e.g. via a hypothetical /clear that clears history)
	// so signals don't carry across a logical session boundary.
	Reset()
}

Watchdog observes per-turn telemetry and returns any alerts that triggered during observation. Implementations must be safe for concurrent use — the agent calls Observe* methods from the streaming event handler and Check from the post-turn hook; concurrency is bounded but real.

The interface is intentionally narrow for v1: just tool-call observation + alert reporting. Richer telemetry (turn timing, per-turn cost delta, files-touched diff) can be added as additional Observe* methods as new signals need them.