Documentation
¶
Overview ¶
Package hooks provides a middleware system for intercepting agent execution events.
Index ¶
- func TenantFromContext(ctx context.Context) string
- func WithTenant(ctx context.Context, tenant string) context.Context
- type CacheHook
- type CallMetric
- type Chain
- type CostReport
- type CostTracker
- func (ct *CostTracker) After(_ context.Context, evt *Event) error
- func (ct *CostTracker) Before(_ context.Context, evt *Event) error
- func (ct *CostTracker) GetGlobalCost() CostReport
- func (ct *CostTracker) GetSessionCost(sessionID string) CostReport
- func (ct *CostTracker) UnknownModels() map[string]int
- type Event
- type EventType
- type Hook
- type LoggingHook
- type MetricsHook
- type MetricsSink
- type MetricsSummary
- type ModelPrice
- type PrometheusHook
- type RateLimitHook
- type RetryHook
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func TenantFromContext ¶ added in v0.4.0
TenantFromContext returns the tenant id from context, or "" if unset.
Types ¶
type CacheHook ¶
type CacheHook struct {
// TTL is how long cached responses remain valid. Default: 5 minutes.
TTL time.Duration
// MaxEntries caps the cache size. When exceeded, the least-recently-used
// entry is evicted. 0 means unlimited.
MaxEntries int
// Hits and Misses track cache statistics.
Hits int
Misses int
// contains filtered or unexported fields
}
CacheHook caches LLM responses for identical requests. It intercepts EventModelCallBefore to check the cache and EventModelCallAfter to store results. Caching is skipped for streaming requests and tool-call responses.
The cache is a bounded LRU with per-entry TTL: lookups and insertions are O(1), and eviction removes the least-recently-used entry rather than scanning the whole map.
func NewCacheHook ¶
NewCacheHook creates a response cache with the given TTL.
type CallMetric ¶
type CallMetric struct {
Type EventType `json:"type"`
Name string `json:"name"`
StartedAt time.Time `json:"started_at"`
Duration time.Duration `json:"duration"`
PromptTokens int `json:"prompt_tokens,omitempty"`
CompletionTokens int `json:"completion_tokens,omitempty"`
Error bool `json:"error,omitempty"`
}
CallMetric records timing and usage data for a single model or tool call.
type CostReport ¶
type CostReport struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
TotalCost float64 `json:"total_cost"`
Currency string `json:"currency"`
}
CostReport is a snapshot of accumulated costs.
type CostTracker ¶
type CostTracker struct {
// Budget is the maximum total spend (in currency units) before calls are
// blocked. 0 means unlimited.
Budget float64
// RejectUnknownModels controls handling of models absent from the price
// table. When true, After returns an error for such models instead of
// silently costing $0; when false (default) their token usage is still
// tracked and the event is flagged, but cost is not accrued.
RejectUnknownModels bool
// contains filtered or unexported fields
}
CostTracker is a hook that tracks LLM API costs per session and enforces optional budget limits. It intercepts model call events to accumulate token usage and compute costs using a configurable price table.
func NewCostTracker ¶
func NewCostTracker(priceTable map[string]ModelPrice) *CostTracker
NewCostTracker creates a cost tracker with the given price table. The priceTable maps model names (e.g. "gpt-4o") to their per-token prices.
func (*CostTracker) GetGlobalCost ¶
func (ct *CostTracker) GetGlobalCost() CostReport
GetGlobalCost returns the accumulated cost across all sessions.
func (*CostTracker) GetSessionCost ¶
func (ct *CostTracker) GetSessionCost(sessionID string) CostReport
GetSessionCost returns the accumulated cost for a specific session.
func (*CostTracker) UnknownModels ¶ added in v0.4.0
func (ct *CostTracker) UnknownModels() map[string]int
UnknownModels returns a snapshot of models seen with no configured price and the number of calls recorded for each. It is safe for concurrent use.
type Event ¶
type Event struct {
Type EventType `json:"type"`
Name string `json:"name"` // tool name, model name, or node ID
Input any `json:"input,omitempty"`
Output any `json:"output,omitempty"`
Error error `json:"-"`
Metadata map[string]any `json:"metadata,omitempty"`
}
Event carries data about an execution event.
type EventType ¶
type EventType string
EventType identifies the kind of execution event.
const ( EventToolCallBefore EventType = "tool_call.before" EventToolCallAfter EventType = "tool_call.after" EventModelCallBefore EventType = "model_call.before" EventModelCallAfter EventType = "model_call.after" EventNodeBefore EventType = "node.before" EventNodeAfter EventType = "node.after" EventContextOverflow EventType = "context.overflow" EventSummarization EventType = "context.summarize" EventSessionStart EventType = "session.start" EventSessionEnd EventType = "session.end" )
type Hook ¶
type Hook interface {
// Before is called before the event executes. Return an error to abort.
Before(ctx context.Context, evt *Event) error
// After is called after the event executes.
After(ctx context.Context, evt *Event) error
}
Hook intercepts execution events for logging, metrics, or modification.
type LoggingHook ¶
type LoggingHook struct {
Events []Event
}
LoggingHook is a simple hook that records events for observability.
type MetricsHook ¶
type MetricsHook struct {
// contains filtered or unexported fields
}
MetricsHook provides structured observability for model and tool calls. It tracks latency, token usage, and error rates with thread-safe counters.
func (*MetricsHook) GetMetrics ¶
func (h *MetricsHook) GetMetrics() []CallMetric
GetMetrics returns all recorded call metrics.
func (*MetricsHook) GetSummary ¶
func (h *MetricsHook) GetSummary() MetricsSummary
GetSummary computes an aggregated summary of all recorded metrics.
type MetricsSink ¶ added in v0.4.0
type MetricsSink interface {
RecordToolCall(tenant, tool string, d time.Duration, isErr bool)
RecordModelCall(tenant, provider string, d time.Duration, promptTokens, completionTokens int64, isErr bool)
}
MetricsSink is the minimal metrics interface that PrometheusHook feeds during execution. It is defined here (rather than importing os/metrics) so the engine layer stays free of a dependency on the control plane; the integrator wires a concrete sink such as *metrics.Registry, which satisfies this interface. Tenant is plumbed from context so cost/latency/tokens can be attributed per tenant.
type MetricsSummary ¶
type MetricsSummary struct {
TotalModelCalls int `json:"total_model_calls"`
TotalToolCalls int `json:"total_tool_calls"`
TotalErrors int `json:"total_errors"`
TotalPromptTokens int `json:"total_prompt_tokens"`
TotalCompTokens int `json:"total_completion_tokens"`
AvgModelLatency time.Duration `json:"avg_model_latency"`
AvgToolLatency time.Duration `json:"avg_tool_latency"`
MaxModelLatency time.Duration `json:"max_model_latency"`
MaxToolLatency time.Duration `json:"max_tool_latency"`
}
MetricsSummary aggregates metrics across all calls.
type ModelPrice ¶
type ModelPrice struct {
PromptPricePerToken float64 // cost per prompt token (e.g. $0.000003)
CompletionPricePerToken float64 // cost per completion token
}
ModelPrice defines the per-token cost for a model.
type PrometheusHook ¶ added in v0.4.0
type PrometheusHook struct {
// contains filtered or unexported fields
}
PrometheusHook bridges execution events to a MetricsSink (e.g. the ChronosOS Prometheus registry) so tool calls, model calls, tokens, errors, and latency are recorded as execution happens. It tracks per-call start times to compute durations across the Before/After boundary.
func NewPrometheusHook ¶ added in v0.4.0
func NewPrometheusHook(sink MetricsSink) *PrometheusHook
NewPrometheusHook creates a hook that feeds sink. A nil sink makes the hook a no-op (safe to install unconditionally).
type RateLimitHook ¶
type RateLimitHook struct {
// RequestsPerMinute caps the number of model calls per minute. 0 = unlimited.
RequestsPerMinute int
// TokensPerMinute caps the estimated prompt tokens per minute. 0 = unlimited.
TokensPerMinute int
// WaitOnLimit controls whether the hook blocks until capacity is available
// (true) or returns an error immediately (false). Default: true.
WaitOnLimit bool
// contains filtered or unexported fields
}
RateLimitHook enforces per-provider rate limits using a token-bucket algorithm. It blocks (or returns an error) on EventModelCallBefore if the rate limit would be exceeded.
The token buckets are individually synchronized, so the hook never holds a single global mutex while waiting for capacity — concurrent callers wait in parallel rather than being serialized behind one another.
func NewRateLimitHook ¶
func NewRateLimitHook(requestsPerMinute, tokensPerMinute int) *RateLimitHook
NewRateLimitHook creates a rate limit hook.
type RetryHook ¶
type RetryHook struct {
MaxRetries int
BaseDelay time.Duration
MaxDelay time.Duration
// RetryableError optionally classifies errors. If nil, all errors trigger retry.
RetryableError func(err error) bool
// OnRetry is called when a retry is about to be attempted. It receives the
// attempt number (1-based) and the delay that will be applied.
OnRetry func(attempt int, delay time.Duration)
// Retries tracks the total number of retries performed (for observability).
// Access it concurrently via RetriesCount / addRetries, which guard it with
// mu; the exported field remains for backward-compatible reads after a hook
// has finished executing.
Retries int
// SleepFn is the function used to sleep between retries.
// Defaults to time.Sleep. Override in tests for instant execution.
SleepFn func(time.Duration)
// contains filtered or unexported fields
}
RetryHook retries failed model calls with exponential backoff and jitter. Unlike a metadata-only hook, this hook performs actual retries by re-invoking the model provider when a model call fails.
Usage:
hook := hooks.NewRetryHook(3)
agent.New("id", "name").AddHook(hook).Build()
func NewRetryHook ¶
NewRetryHook creates a retry hook with sensible defaults.
func (*RetryHook) After ¶
After inspects model call errors and performs actual retries. When a model call fails with a retryable error, this hook:
- Waits with exponential backoff + jitter
- Re-invokes the model provider using the original request
- Updates the event output/error with the retry result
- Repeats up to MaxRetries times
func (*RetryHook) RetriesCount ¶ added in v0.4.0
RetriesCount returns the total number of retries performed so far. It is safe for concurrent use.