Documentation
¶
Overview ¶
Audit log sink for LLM calls.
AuditEvent captures the privacy-preserving metadata of a single provider call: model/provider, session and key identifiers, sha256 hashes of the prompt and response (never the raw text), token counts, cost, latency, and status.
Sinks are pluggable via the AuditSink interface. The default NoopSink discards events; JSONLFileSink appends one JSON object per line to a configurable file. Everything here is stdlib-only.
OpenTelemetry GenAI semantic convention attribute keys for AI agent spans.
These exported constants are the canonical, ecosystem-wide attribute keys for describing LLM / AI agent operations. They follow the OpenTelemetry GenAI semantic conventions (gen_ai.*) and are shared as the reference set that the other hawk-eco repos (hawk, yaad, tok, trace) should mirror when emitting spans, so dashboards and exporters can correlate cost/usage/identity across the whole ecosystem.
The legacy llm.* attribute keys in observability.go remain for backwards compatibility; new instrumentation should prefer the gen_ai.* keys below. This file is stdlib-only and purely additive (constant declarations only).
Spec: https://opentelemetry.io/docs/specs/semconv/gen-ai/
Package eyrie observability provides OpenTelemetry-compatible structured tracing and metrics collection for all LLM provider calls.
Design follows OpenTelemetry Go SDK patterns (TraceID/SpanID, span lifecycle, metric instruments) while remaining zero-dependency (Go stdlib only).
Usage is opt-in: a nil *Telemetry adds zero overhead.
Index ¶
- Constants
- func HashContent(s string) string
- type AuditEvent
- type AuditSink
- type JSONLFileSink
- type MetricsCollector
- func (mc *MetricsCollector) CacheHitRate() float64
- func (mc *MetricsCollector) CostAccumulator(key string) float64
- func (mc *MetricsCollector) ErrorRate(provider string) float64
- func (mc *MetricsCollector) ExportJSON() string
- func (mc *MetricsCollector) ExportPrometheus() string
- func (mc *MetricsCollector) LatencyHistogram(key string) (p50, p95, p99 float64)
- func (mc *MetricsCollector) RecordCustom(name string, value float64, attrs map[string]string)
- func (mc *MetricsCollector) RecordRequest(provider, model string, inputTok, outputTok int, latency time.Duration, ...)
- func (mc *MetricsCollector) RequestCount(key string) int64
- func (mc *MetricsCollector) TokensUsed(key string) (int64, int64)
- func (mc *MetricsCollector) TotalCost() float64
- func (mc *MetricsCollector) TotalRequests() int64
- type NoopSink
- type Span
- type SpanEvent
- type SpanStatus
- type Telemetry
Constants ¶
const ( // AttrGenAISystem identifies the GenAI provider/system handling the request // (e.g. "openai", "anthropic", "gemini"). OTel: gen_ai.system. AttrGenAISystem = "gen_ai.system" // AttrGenAIRequestModel is the model name requested by the caller // (e.g. "gpt-4o", "claude-sonnet-4-5"). OTel: gen_ai.request.model. AttrGenAIRequestModel = "gen_ai.request.model" // AttrGenAIResponseModel is the model that actually served the response, when // the provider reports it. OTel: gen_ai.response.model. AttrGenAIResponseModel = "gen_ai.response.model" // AttrGenAIUsageInputTokens is the number of prompt/input tokens consumed. // OTel: gen_ai.usage.input_tokens. AttrGenAIUsageInputTokens = "gen_ai.usage.input_tokens" // AttrGenAIUsageOutputTokens is the number of completion/output tokens // generated. OTel: gen_ai.usage.output_tokens. AttrGenAIUsageOutputTokens = "gen_ai.usage.output_tokens" // AttrGenAIOperationName names the GenAI operation (e.g. "chat", "embeddings", // "tool_use"). OTel: gen_ai.operation.name. AttrGenAIOperationName = "gen_ai.operation.name" // AttrCostUSD is the computed monetary cost of the operation in US dollars. // Ecosystem extension to the OTel gen_ai.* set; key: cost.usd. AttrCostUSD = "cost.usd" // AttrToolName is the name of the tool/function invoked during an agent step. // OTel: gen_ai.tool.name (exposed here as the stable key "tool.name"). AttrToolName = "tool.name" // AttrSessionID correlates spans belonging to the same logical session or // conversation. Ecosystem key: session.id. AttrSessionID = "session.id" // AttrAgentID identifies the agent instance producing the span. Ecosystem // key: agent.id (aligns with OTel gen_ai.agent.id). AttrAgentID = "agent.id" )
GenAI semantic convention attribute keys. These mirror the OpenTelemetry gen_ai.* namespace plus the small set of ecosystem extensions (cost.usd, session.id, agent.id) documented in docs/OTEL-CONVENTIONS.md.
const ( SpanLLMChat = "llm.chat" SpanLLMStream = "llm.stream" SpanLLMRetry = "llm.retry" SpanLLMCacheHit = "llm.cache_hit" )
const ( AttrLLMProvider = "llm.provider" AttrLLMModel = "llm.model" AttrLLMInputTokens = "llm.input_tokens" AttrLLMOutputTokens = "llm.output_tokens" AttrLLMCostUSD = "llm.cost_usd" AttrLLMLatencyMs = "llm.latency_ms" AttrLLMStatus = "llm.status" )
Variables ¶
This section is empty.
Functions ¶
func HashContent ¶
HashContent returns the lowercase hex-encoded sha256 of s. Use it to derive PromptHash / ResponseHash without persisting raw prompt or response text.
Types ¶
type AuditEvent ¶
type AuditEvent struct {
Timestamp time.Time `json:"timestamp"`
Model string `json:"model"`
Provider string `json:"provider"`
SessionID string `json:"session_id,omitempty"`
VirtualKeyID string `json:"virtual_key_id,omitempty"`
PromptHash string `json:"prompt_hash,omitempty"`
ResponseHash string `json:"response_hash,omitempty"`
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
CostUSD float64 `json:"cost_usd"`
LatencyMS int64 `json:"latency_ms"`
Status string `json:"status"`
}
AuditEvent is a single audit record. It deliberately stores only hashes of prompt/response content so that raw text is never persisted to the audit log.
type AuditSink ¶
type AuditSink interface {
Record(event AuditEvent) error
}
AuditSink records audit events. Implementations must be safe for concurrent use.
type JSONLFileSink ¶
type JSONLFileSink struct {
// contains filtered or unexported fields
}
JSONLFileSink appends audit events as newline-delimited JSON to a file. Writes are serialized with a mutex so concurrent callers do not interleave lines.
func NewJSONLFileSink ¶
func NewJSONLFileSink(path string) (*JSONLFileSink, error)
NewJSONLFileSink opens (creating if needed) the file at path for append-only writes and returns a sink that writes to it. The caller owns the sink and should call Close when done.
func (*JSONLFileSink) Close ¶
func (s *JSONLFileSink) Close() error
Close closes the underlying file.
func (*JSONLFileSink) Record ¶
func (s *JSONLFileSink) Record(event AuditEvent) error
Record implements AuditSink, appending the event as one JSON line.
type MetricsCollector ¶
type MetricsCollector struct {
// contains filtered or unexported fields
}
MetricsCollector aggregates metrics for LLM operations. It uses atomic operations and mutexes for thread safety with minimal contention.
func NewMetricsCollector ¶
func NewMetricsCollector() *MetricsCollector
NewMetricsCollector creates a new MetricsCollector.
func (*MetricsCollector) CacheHitRate ¶
func (mc *MetricsCollector) CacheHitRate() float64
CacheHitRate returns the ratio of cache hits to total cache-eligible requests.
func (*MetricsCollector) CostAccumulator ¶
func (mc *MetricsCollector) CostAccumulator(key string) float64
CostAccumulator returns the total cost in USD for a provider/model key.
func (*MetricsCollector) ErrorRate ¶
func (mc *MetricsCollector) ErrorRate(provider string) float64
ErrorRate returns the error rate for a provider (errors / total requests involving that provider).
func (*MetricsCollector) ExportJSON ¶
func (mc *MetricsCollector) ExportJSON() string
ExportJSON dumps all metrics as a JSON string.
func (*MetricsCollector) ExportPrometheus ¶
func (mc *MetricsCollector) ExportPrometheus() string
ExportPrometheus dumps metrics in Prometheus exposition format.
func (*MetricsCollector) LatencyHistogram ¶
func (mc *MetricsCollector) LatencyHistogram(key string) (p50, p95, p99 float64)
LatencyHistogram returns P50, P95, P99 latencies in milliseconds for a key.
func (*MetricsCollector) RecordCustom ¶
func (mc *MetricsCollector) RecordCustom(name string, value float64, attrs map[string]string)
RecordCustom records a custom named metric.
func (*MetricsCollector) RecordRequest ¶
func (mc *MetricsCollector) RecordRequest(provider, model string, inputTok, outputTok int, latency time.Duration, costUSD float64, isError bool)
RecordRequest records a completed LLM request with all its metrics.
func (*MetricsCollector) RequestCount ¶
func (mc *MetricsCollector) RequestCount(key string) int64
RequestCount returns total requests for a provider/model key.
func (*MetricsCollector) TokensUsed ¶
func (mc *MetricsCollector) TokensUsed(key string) (int64, int64)
TokensUsed returns (inputTokens, outputTokens) for a provider/model key.
func (*MetricsCollector) TotalCost ¶
func (mc *MetricsCollector) TotalCost() float64
TotalCost returns the sum of all accumulated costs in USD.
func (*MetricsCollector) TotalRequests ¶
func (mc *MetricsCollector) TotalRequests() int64
TotalRequests returns the sum of all request counts.
type NoopSink ¶
type NoopSink struct{}
NoopSink is an AuditSink that discards all events. It is the safe default when auditing is not configured.
func (NoopSink) Record ¶
func (NoopSink) Record(AuditEvent) error
Record implements AuditSink and does nothing.
type Span ¶
type Span struct {
TraceID string `json:"trace_id"`
SpanID string `json:"span_id"`
Name string `json:"name"`
StartTime time.Time `json:"start_time"`
EndTime time.Time `json:"end_time,omitempty"`
Provider string `json:"provider,omitempty"`
Model string `json:"model,omitempty"`
Status SpanStatus `json:"status"`
Attributes map[string]string `json:"attributes,omitempty"`
Events []SpanEvent `json:"events,omitempty"`
}
Span represents a single timed operation in a trace, modeled after OpenTelemetry's Span concept. It carries a TraceID, SpanID, timing information, and arbitrary string attributes.
func (*Span) SetAttribute ¶
SetAttribute sets a single attribute on the span.
type SpanEvent ¶
type SpanEvent struct {
Name string `json:"name"`
Timestamp time.Time `json:"timestamp"`
Attributes map[string]string `json:"attributes,omitempty"`
}
SpanEvent records a timestamped event within a span.
type SpanStatus ¶
type SpanStatus string
SpanStatus represents the outcome of a span.
const ( StatusOK SpanStatus = "ok" StatusError SpanStatus = "error" StatusUnset SpanStatus = "unset" )
type Telemetry ¶
type Telemetry struct {
// OnSpanEnd is an optional callback invoked when a span ends.
// Useful for exporting spans to external systems.
OnSpanEnd func(*Span)
// contains filtered or unexported fields
}
Telemetry wraps observability for all LLM calls. A nil *Telemetry is safe to use and adds zero overhead (all methods are no-ops on nil receiver).
func NewTelemetry ¶
func NewTelemetry() *Telemetry
NewTelemetry creates a new Telemetry instance with an initialized MetricsCollector.
func (*Telemetry) EndSpan ¶
EndSpan completes a span, recording its status and duration. If err is non-nil, the span status is set to error and the error message is recorded. No-op if either t or span is nil.
func (*Telemetry) Metrics ¶
func (t *Telemetry) Metrics() *MetricsCollector
Metrics returns the underlying MetricsCollector, or nil if Telemetry is nil.
func (*Telemetry) RecordMetric ¶
RecordMetric records a named metric value with attributes. This is a general-purpose method for recording custom metrics. No-op if t is nil.