Documentation
¶
Overview ¶
Package hook provides the Hook interface for observing LLM generation lifecycle events: start, end, per-frame, tool calls, and errors.
Index ¶
- func WrapStream(ctx context.Context, src *niro.Stream, h Hook, model string, start time.Time) *niro.Stream
- type GenerateEndInfo
- type GenerateStartInfo
- type Hook
- type NoOpHook
- func (NoOpHook) OnError(context.Context, error)
- func (NoOpHook) OnFrame(context.Context, niro.Frame, time.Duration) error
- func (NoOpHook) OnGenerateEnd(context.Context, GenerateEndInfo)
- func (NoOpHook) OnGenerateStart(ctx context.Context, _ GenerateStartInfo) context.Context
- func (NoOpHook) OnToolCall(context.Context, niro.ToolCall)
- func (NoOpHook) OnToolResult(context.Context, niro.ToolResult, time.Duration)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func WrapStream ¶
func WrapStream(ctx context.Context, src *niro.Stream, h Hook, model string, start time.Time) *niro.Stream
WrapStream interposes a Hook between a provider stream and the consumer. It creates a goroutine that reads from src, fires OnFrame (and OnToolCall / OnToolResult for the corresponding frame kinds) for each frame, and emits the frame to the returned stream. When the source is exhausted it fires OnGenerateEnd with usage and duration. On any error it fires OnError.
Both runtime.Runtime and dsl.Runner use this to get consistent hook coverage from a single implementation.
Types ¶
type GenerateEndInfo ¶
type GenerateEndInfo struct {
Provider string // Provider name
Model string // Model actually used
RequestID string // Unique request ID for tracing
Usage niro.Usage // Token usage
FinishReason string // Why generation stopped
Duration time.Duration // Wall-clock duration
ResponseID string // Provider-assigned response ID
Error error // Non-nil if generation failed
}
GenerateEndInfo contains metadata about a completed generation.
type GenerateStartInfo ¶
type GenerateStartInfo struct {
Provider string // Provider name (e.g. "openai", "anthropic")
Model string // Requested model
Messages int // Number of messages
Tools int // Number of tools
RequestID string // Unique request ID for tracing
FunctionID string // Caller-assigned ID for tracing
Metadata map[string]string // Arbitrary metadata from the caller
}
GenerateStartInfo contains metadata about an incoming generation request.
type Hook ¶
type Hook interface {
// OnGenerateStart is called before a Provider.Generate request.
// The returned context is passed to the provider — use it to
// inject trace IDs, span contexts, or request-scoped values.
OnGenerateStart(ctx context.Context, info GenerateStartInfo) context.Context
// OnGenerateEnd is called after a generation stream is fully consumed.
// It receives the final usage, response metadata, and any error.
OnGenerateEnd(ctx context.Context, info GenerateEndInfo)
// OnFrame is called for each frame emitted by the provider.
// elapsed is the wall-clock duration since OnGenerateStart. Use it to
// measure time-to-first-token (TTFT) and inter-token jitter — critical
// metrics for real-time and telephony applications.
// This is the per-token hook — keep it extremely fast.
// A nil return is fine; return a non-nil error to abort the stream.
OnFrame(ctx context.Context, f niro.Frame, elapsed time.Duration) error
// OnToolCall is called when a tool call is about to be executed.
OnToolCall(ctx context.Context, call niro.ToolCall)
// OnToolResult is called when a tool call completes.
OnToolResult(ctx context.Context, result niro.ToolResult, elapsed time.Duration)
// OnError is called when an error occurs at any stage.
OnError(ctx context.Context, err error)
}
Hook provides observability into Niro operations. Implement this interface for telemetry, logging, or integration with platforms like Langfuse, Datadog, or OpenTelemetry.
All methods are called synchronously in the hot path. Implementations MUST be fast and non-blocking. Heavy work (network I/O, persistence) should be dispatched to a background goroutine.
A zero-value Hook (nil) is safe — the runtime checks before calling.
type NoOpHook ¶
type NoOpHook struct{}
NoOpHook is a Hook that does nothing. Embed it in your hook struct to only override the methods you need:
type MyHook struct { hook.NoOpHook }
func (h *MyHook) OnGenerateEnd(ctx context.Context, info hook.GenerateEndInfo) {
log.Printf("model=%s tokens=%d", info.Model, info.Usage.TotalTokens)
}
func (NoOpHook) OnGenerateEnd ¶
func (NoOpHook) OnGenerateEnd(context.Context, GenerateEndInfo)
Source Files