Documentation
¶
Overview ¶
Package metrics provides the unified PromptKit metrics collector.
The Collector records both pipeline operational metrics (from EventBus events) and eval result metrics (from pack-defined MetricDefs) into a Prometheus registry. It replaces the dead-code runtime/metrics/prometheus package and the hand-rolled Prometheus text writer in runtime/evals/metrics.go.
Usage:
reg := prometheus.NewRegistry()
collector := metrics.NewCollector(metrics.CollectorOpts{
Registerer: reg,
Namespace: "myapp",
ConstLabels: prometheus.Labels{"env": "prod"},
InstanceLabels: []string{"tenant"},
})
// Per-conversation binding:
ctx := collector.Bind(prometheus.Labels{"tenant": "acme"})
bus.SubscribeAll(ctx.OnEvent) // pipeline metrics
recorder := ctx // evals.MetricRecorder
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Collector ¶
type Collector struct {
// contains filtered or unexported fields
}
Collector records both pipeline and eval metrics into a Prometheus registry. Created once per process via NewCollector(). Shared across conversations.
Pipeline events are recorded via MetricContext.OnEvent (implements events.Listener). Eval results are recorded via MetricContext.Record (implements evals.MetricRecorder).
Instance labels (varying per conversation) are passed via a MetricContext that the SDK creates per conversation using Bind().
func NewCollector ¶
func NewCollector(opts CollectorOpts) *Collector
NewCollector creates a unified metrics collector and registers pipeline metrics into the provided Registerer (or prometheus.DefaultRegisterer).
func NewEvalOnlyCollector ¶
func NewEvalOnlyCollector(opts CollectorOpts) *Collector
NewEvalOnlyCollector creates a Collector that only records eval result metrics, with pipeline operational metrics disabled. This is a convenience wrapper for:
metrics.NewCollector(metrics.CollectorOpts{
Registerer: reg,
DisablePipelineMetrics: true,
...
})
Use this for standalone eval workers (e.g. sdk.Evaluate()) that don't run a live pipeline and therefore don't need provider, tool, or pipeline metrics.
func (*Collector) Bind ¶
func (c *Collector) Bind(instanceLabels map[string]string) *MetricContext
Bind creates a MetricContext for a specific conversation, binding instance label values. The returned context implements both events.Listener (via OnEvent) and evals.MetricRecorder (via Record).
Label key ordering in the map does not matter — the Collector sorts InstanceLabels internally for deterministic Prometheus label ordering. Only the keys need to match; values are looked up by key, not position.
If the Collector has no InstanceLabels, pass nil.
func (*Collector) Registry ¶
func (c *Collector) Registry() *prometheus.Registry
Registry returns the underlying *prometheus.Registry if one was provided, or nil if a non-Registry Registerer was used.
type CollectorOpts ¶
type CollectorOpts struct {
// Registerer is the Prometheus registerer to register metrics into.
// If nil, prometheus.DefaultRegisterer is used.
// If the value is a *prometheus.Registry, it is accessible via
// Collector.Registry() for use with promhttp.HandlerFor().
Registerer prometheus.Registerer
// Namespace overrides the metric name prefix (default: "promptkit").
Namespace string
// ConstLabels are added to every metric as constant label pairs.
// These must be truly constant for the lifetime of the process —
// they are baked into the metric descriptor at registration time.
// Use for process-level dimensions: env, region, service_name.
ConstLabels prometheus.Labels
// InstanceLabels are label NAMES that vary per conversation.
// They become additional dimensions on every Vec metric, allowing
// multiple conversations to share one Collector without registration
// conflicts. Values are provided per-conversation via Bind().
// Use for: tenant, prompt_name, conversation_type.
InstanceLabels []string
// DisablePipelineMetrics disables operational metrics (provider, tool, pipeline,
// validation). Set this to true when you only need eval result metrics — for
// example, in standalone eval workers that use sdk.Evaluate() without a live
// pipeline. See also NewEvalOnlyCollector() for a convenience constructor.
DisablePipelineMetrics bool
// DisableEvalMetrics disables eval result metrics.
DisableEvalMetrics bool
}
CollectorOpts configures a Collector.
type MetricContext ¶
type MetricContext struct {
// contains filtered or unexported fields
}
MetricContext is a per-conversation handle that carries instance label values. It implements events.Listener (via OnEvent) and evals.MetricRecorder (via Record), forwarding observations to the shared Collector with bound labels.
func (*MetricContext) OnEvent ¶
func (mc *MetricContext) OnEvent(event *events.Event)
OnEvent processes a pipeline event and records relevant metrics. This method is designed to be used with EventBus.SubscribeAll.
func (*MetricContext) Record ¶
func (mc *MetricContext) Record(result evals.EvalResult, metric *evals.MetricDef) error
Record implements evals.MetricRecorder. It records an eval result into the Prometheus registry using the metric definition from the pack.
Source Files