hooks

package
v0.10.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 1, 2026 License: MIT Imports: 10 Imported by: 0

Documentation

Overview

Package hooks provides a middleware system for intercepting agent execution events.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func TenantFromContext added in v0.4.0

func TenantFromContext(ctx context.Context) string

TenantFromContext returns the tenant id from context, or "" if unset.

func WithTenant added in v0.4.0

func WithTenant(ctx context.Context, tenant string) context.Context

WithTenant returns a context carrying the tenant id used for metric labels.

Types

type CacheHook

type CacheHook struct {

	// TTL is how long cached responses remain valid. Default: 5 minutes.
	TTL time.Duration
	// MaxEntries caps the cache size. When exceeded, the least-recently-used
	// entry is evicted. 0 means unlimited.
	MaxEntries int
	// Hits and Misses track cache statistics.
	Hits   int
	Misses int
	// contains filtered or unexported fields
}

CacheHook caches LLM responses for identical requests. It intercepts EventModelCallBefore to check the cache and EventModelCallAfter to store results. Caching is skipped for streaming requests and tool-call responses.

The cache is a bounded LRU with per-entry TTL: lookups and insertions are O(1), and eviction removes the least-recently-used entry rather than scanning the whole map.

func NewCacheHook

func NewCacheHook(ttl time.Duration) *CacheHook

NewCacheHook creates a response cache with the given TTL.

func (*CacheHook) After

func (h *CacheHook) After(_ context.Context, evt *Event) error

func (*CacheHook) Before

func (h *CacheHook) Before(_ context.Context, evt *Event) error

func (*CacheHook) Clear

func (h *CacheHook) Clear()

Clear removes all cached entries.

func (*CacheHook) Stats

func (h *CacheHook) Stats() (hits, misses int)

Stats returns cache hit/miss counts.

type CallMetric

type CallMetric struct {
	Type             EventType     `json:"type"`
	Name             string        `json:"name"`
	StartedAt        time.Time     `json:"started_at"`
	Duration         time.Duration `json:"duration"`
	PromptTokens     int           `json:"prompt_tokens,omitempty"`
	CompletionTokens int           `json:"completion_tokens,omitempty"`
	Error            bool          `json:"error,omitempty"`
}

CallMetric records timing and usage data for a single model or tool call.

type Chain

type Chain []Hook

Chain runs multiple hooks in sequence.

func (Chain) After

func (c Chain) After(ctx context.Context, evt *Event) error

func (Chain) Before

func (c Chain) Before(ctx context.Context, evt *Event) error

type CostReport

type CostReport struct {
	PromptTokens     int     `json:"prompt_tokens"`
	CompletionTokens int     `json:"completion_tokens"`
	TotalTokens      int     `json:"total_tokens"`
	TotalCost        float64 `json:"total_cost"`
	Currency         string  `json:"currency"`
}

CostReport is a snapshot of accumulated costs.

type CostTracker

type CostTracker struct {

	// Budget is the maximum total spend (in currency units) before calls are
	// blocked. 0 means unlimited.
	Budget float64
	// RejectUnknownModels controls handling of models absent from the price
	// table. When true, After returns an error for such models instead of
	// silently costing $0; when false (default) their token usage is still
	// tracked and the event is flagged, but cost is not accrued.
	RejectUnknownModels bool
	// contains filtered or unexported fields
}

CostTracker is a hook that tracks LLM API costs per session and enforces optional budget limits. It intercepts model call events to accumulate token usage and compute costs using a configurable price table.

func NewCostTracker

func NewCostTracker(priceTable map[string]ModelPrice) *CostTracker

NewCostTracker creates a cost tracker with the given price table. The priceTable maps model names (e.g. "gpt-4o") to their per-token prices.

func (*CostTracker) After

func (ct *CostTracker) After(_ context.Context, evt *Event) error

func (*CostTracker) Before

func (ct *CostTracker) Before(_ context.Context, evt *Event) error

func (*CostTracker) GetGlobalCost

func (ct *CostTracker) GetGlobalCost() CostReport

GetGlobalCost returns the accumulated cost across all sessions.

func (*CostTracker) GetSessionCost

func (ct *CostTracker) GetSessionCost(sessionID string) CostReport

GetSessionCost returns the accumulated cost for a specific session.

func (*CostTracker) UnknownModels added in v0.4.0

func (ct *CostTracker) UnknownModels() map[string]int

UnknownModels returns a snapshot of models seen with no configured price and the number of calls recorded for each. It is safe for concurrent use.

type Event

type Event struct {
	Type     EventType      `json:"type"`
	Name     string         `json:"name"` // tool name, model name, or node ID
	Input    any            `json:"input,omitempty"`
	Output   any            `json:"output,omitempty"`
	Error    error          `json:"-"`
	Metadata map[string]any `json:"metadata,omitempty"`
}

Event carries data about an execution event.

type EventType

type EventType string

EventType identifies the kind of execution event.

const (
	EventToolCallBefore  EventType = "tool_call.before"
	EventToolCallAfter   EventType = "tool_call.after"
	EventModelCallBefore EventType = "model_call.before"
	EventModelCallAfter  EventType = "model_call.after"
	EventNodeBefore      EventType = "node.before"
	EventNodeAfter       EventType = "node.after"

	EventContextOverflow EventType = "context.overflow"
	EventSummarization   EventType = "context.summarize"
	EventSessionStart    EventType = "session.start"
	EventSessionEnd      EventType = "session.end"
)

type Hook

type Hook interface {
	// Before is called before the event executes. Return an error to abort.
	Before(ctx context.Context, evt *Event) error
	// After is called after the event executes.
	After(ctx context.Context, evt *Event) error
}

Hook intercepts execution events for logging, metrics, or modification.

type LoggingHook

type LoggingHook struct {
	Events []Event
}

LoggingHook is a simple hook that records events for observability.

func (*LoggingHook) After

func (h *LoggingHook) After(_ context.Context, evt *Event) error

func (*LoggingHook) Before

func (h *LoggingHook) Before(_ context.Context, evt *Event) error

type MetricsHook

type MetricsHook struct {
	// contains filtered or unexported fields
}

MetricsHook provides structured observability for model and tool calls. It tracks latency, token usage, and error rates with thread-safe counters.

func NewMetricsHook

func NewMetricsHook() *MetricsHook

NewMetricsHook creates a new metrics hook.

func (*MetricsHook) After

func (h *MetricsHook) After(_ context.Context, evt *Event) error

func (*MetricsHook) Before

func (h *MetricsHook) Before(_ context.Context, evt *Event) error

func (*MetricsHook) GetMetrics

func (h *MetricsHook) GetMetrics() []CallMetric

GetMetrics returns all recorded call metrics.

func (*MetricsHook) GetSummary

func (h *MetricsHook) GetSummary() MetricsSummary

GetSummary computes an aggregated summary of all recorded metrics.

func (*MetricsHook) Reset

func (h *MetricsHook) Reset()

Reset clears all recorded metrics.

type MetricsSink added in v0.4.0

type MetricsSink interface {
	RecordToolCall(tenant, tool string, d time.Duration, isErr bool)
	RecordModelCall(tenant, provider string, d time.Duration, promptTokens, completionTokens int64, isErr bool)
}

MetricsSink is the minimal metrics interface that PrometheusHook feeds during execution. It is defined here (rather than importing os/metrics) so the engine layer stays free of a dependency on the control plane; the integrator wires a concrete sink such as *metrics.Registry, which satisfies this interface. Tenant is plumbed from context so cost/latency/tokens can be attributed per tenant.

type MetricsSummary

type MetricsSummary struct {
	TotalModelCalls   int           `json:"total_model_calls"`
	TotalToolCalls    int           `json:"total_tool_calls"`
	TotalErrors       int           `json:"total_errors"`
	TotalPromptTokens int           `json:"total_prompt_tokens"`
	TotalCompTokens   int           `json:"total_completion_tokens"`
	AvgModelLatency   time.Duration `json:"avg_model_latency"`
	AvgToolLatency    time.Duration `json:"avg_tool_latency"`
	MaxModelLatency   time.Duration `json:"max_model_latency"`
	MaxToolLatency    time.Duration `json:"max_tool_latency"`
}

MetricsSummary aggregates metrics across all calls.

type ModelPrice

type ModelPrice struct {
	PromptPricePerToken     float64 // cost per prompt token (e.g. $0.000003)
	CompletionPricePerToken float64 // cost per completion token
}

ModelPrice defines the per-token cost for a model.

type PrometheusHook added in v0.4.0

type PrometheusHook struct {
	// contains filtered or unexported fields
}

PrometheusHook bridges execution events to a MetricsSink (e.g. the ChronosOS Prometheus registry) so tool calls, model calls, tokens, errors, and latency are recorded as execution happens. It tracks per-call start times to compute durations across the Before/After boundary.

func NewPrometheusHook added in v0.4.0

func NewPrometheusHook(sink MetricsSink) *PrometheusHook

NewPrometheusHook creates a hook that feeds sink. A nil sink makes the hook a no-op (safe to install unconditionally).

func (*PrometheusHook) After added in v0.4.0

func (h *PrometheusHook) After(ctx context.Context, evt *Event) error

func (*PrometheusHook) Before added in v0.4.0

func (h *PrometheusHook) Before(_ context.Context, evt *Event) error

type RateLimitHook

type RateLimitHook struct {
	// RequestsPerMinute caps the number of model calls per minute. 0 = unlimited.
	RequestsPerMinute int
	// TokensPerMinute caps the estimated prompt tokens per minute. 0 = unlimited.
	TokensPerMinute int
	// WaitOnLimit controls whether the hook blocks until capacity is available
	// (true) or returns an error immediately (false). Default: true.
	WaitOnLimit bool
	// contains filtered or unexported fields
}

RateLimitHook enforces per-provider rate limits using a token-bucket algorithm. It blocks (or returns an error) on EventModelCallBefore if the rate limit would be exceeded.

The token buckets are individually synchronized, so the hook never holds a single global mutex while waiting for capacity — concurrent callers wait in parallel rather than being serialized behind one another.

func NewRateLimitHook

func NewRateLimitHook(requestsPerMinute, tokensPerMinute int) *RateLimitHook

NewRateLimitHook creates a rate limit hook.

func (*RateLimitHook) After

func (h *RateLimitHook) After(_ context.Context, evt *Event) error

func (*RateLimitHook) Before

func (h *RateLimitHook) Before(ctx context.Context, evt *Event) error

type RetryHook

type RetryHook struct {
	MaxRetries int
	BaseDelay  time.Duration
	MaxDelay   time.Duration

	// RetryableError optionally classifies errors. If nil, all errors trigger retry.
	RetryableError func(err error) bool

	// OnRetry is called when a retry is about to be attempted. It receives the
	// attempt number (1-based) and the delay that will be applied.
	OnRetry func(attempt int, delay time.Duration)

	// Retries tracks the total number of retries performed (for observability).
	// Access it concurrently via RetriesCount / addRetries, which guard it with
	// mu; the exported field remains for backward-compatible reads after a hook
	// has finished executing.
	Retries int

	// SleepFn is the function used to sleep between retries.
	// Defaults to time.Sleep. Override in tests for instant execution.
	SleepFn func(time.Duration)
	// contains filtered or unexported fields
}

RetryHook retries failed model calls with exponential backoff and jitter. Unlike a metadata-only hook, this hook performs actual retries by re-invoking the model provider when a model call fails.

Usage:

hook := hooks.NewRetryHook(3)
agent.New("id", "name").AddHook(hook).Build()

func NewRetryHook

func NewRetryHook(maxRetries int) *RetryHook

NewRetryHook creates a retry hook with sensible defaults.

func (*RetryHook) After

func (h *RetryHook) After(ctx context.Context, evt *Event) error

After inspects model call errors and performs actual retries. When a model call fails with a retryable error, this hook:

  1. Waits with exponential backoff + jitter
  2. Re-invokes the model provider using the original request
  3. Updates the event output/error with the retry result
  4. Repeats up to MaxRetries times

func (*RetryHook) Before

func (h *RetryHook) Before(_ context.Context, _ *Event) error

func (*RetryHook) RetriesCount added in v0.4.0

func (h *RetryHook) RetriesCount() int

RetriesCount returns the total number of retries performed so far. It is safe for concurrent use.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL