telemetry

package
v1.9.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 4, 2026 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package telemetry holds the OpenTelemetry plumbing the engine and the embedders share: the tracer, the span attribute keys, how an error is named, the bounded component labels, and the metric families recorded on a go-utils metrics sink.

Spans go to the global tracer provider, which forge.New installs. Without one they cost a no-op tracer call. Metrics are recorded only on a sink the caller hands over (engine.WithMetrics), never on otel.Meter: a Forge app installs no global MeterProvider, so nothing recorded there is exported.

Index

Constants

View Source 
const (
	MetricIngestDuration   = "weave_ingest_duration_seconds"
	MetricRetrieveDuration = "weave_retrieve_duration_seconds"
	MetricVectorDuration   = "weave_vector_operation_duration_seconds"
	MetricOrphanedHits     = "weave_retrieve_hits_orphaned_total"
	MetricGenAIDuration    = "gen_ai.client.operation.duration"
	MetricGenAITokens      = "gen_ai.client.token.usage" // #nosec G101 -- a metric name, not a credential
)

Metric names. Forge exports a name outside its semconv namespaces with its own prefix and no unit suffix, so the weave ones carry their unit in the name and scrape as forge_<name>. The gen_ai ones are the OpenTelemetry GenAI client metrics; gen_ai is not a forge semconv namespace, so they scrape as forge_gen_ai_client_operation_duration and forge_gen_ai_client_token_usage.

View Source 
const (
	LabelChunker     = "chunker"
	LabelRetriever   = "retriever"
	LabelVectorStore = "vectorstore"
	LabelOperation   = "operation"
	LabelResult      = "result"

	LabelGenAIOperation = "gen_ai.operation.name"
	LabelGenAIProvider  = "gen_ai.provider.name"
	LabelEmbedOperation = "weave.embed.operation"
	LabelErrorType      = "error.type"
	LabelTokenType      = "gen_ai.token.type" // #nosec G101 -- a label key, not a credential
)

Metric label keys. Forge turns the dots into underscores on the scrape.

View Source 
const (
	VectorSearch = "search"
	VectorUpsert = "upsert"
)

Vector store operations, the values of the operation label.

View Source 
const (
	GenAIOperationName    = attribute.Key("gen_ai.operation.name")
	GenAIProviderName     = attribute.Key("gen_ai.provider.name")
	GenAIRequestModel     = attribute.Key("gen_ai.request.model")
	GenAIUsageInputTokens = attribute.Key("gen_ai.usage.input_tokens")
	ErrorTypeKey          = attribute.Key("error.type")

	HTTPRequestMethod      = attribute.Key("http.request.method")
	HTTPResponseStatusCode = attribute.Key("http.response.status_code")
	ServerAddress          = attribute.Key("server.address")
	ServerPort             = attribute.Key("server.port")

	CollectionID     = attribute.Key("weave.collection.id")
	DocumentID       = attribute.Key("weave.document.id")
	Result           = attribute.Key("weave.result")
	ChunkerKind      = attribute.Key("weave.chunker.kind")
	VectorStoreKind  = attribute.Key("weave.vectorstore.kind")
	RetrieverKind    = attribute.Key("weave.retriever.kind")
	ContentBytes     = attribute.Key("weave.content.bytes")
	ChunkCount       = attribute.Key("weave.chunk.count")
	EmbedOperation   = attribute.Key("weave.embed.operation")
	EmbedInputs      = attribute.Key("weave.embed.inputs")
	VectorCount      = attribute.Key("weave.vector.count")
	SearchTopK       = attribute.Key("weave.search.top_k")
	SearchResults    = attribute.Key("weave.search.results")
	RetrieveTopK     = attribute.Key("weave.retrieve.top_k")
	RetrieveResults  = attribute.Key("weave.retrieve.results")
	RetrieveHydrated = attribute.Key("weave.retrieve.hydrated")
	RetrieveOrphaned = attribute.Key("weave.retrieve.orphaned")
	ReindexDocuments = attribute.Key("weave.reindex.documents")
	ReindexChunks    = attribute.Key("weave.reindex.chunks")
	RecoverListed    = attribute.Key("weave.recover.listed")
	RecoverFailed    = attribute.Key("weave.recover.failed")
	RecoverPurged    = attribute.Key("weave.recover.purged")
	RecoverSkipped   = attribute.Key("weave.recover.skipped_no_tenant")
	IngestUnchanged  = attribute.Key("weave.ingest.unchanged")
	IngestReplaced   = attribute.Key("weave.ingest.replaced")
)

Span attribute keys. The gen_ai, http, server and error ones follow the OpenTelemetry semantic conventions; the weave ones are this module's own. Values are counts, booleans, bounded enum strings and opaque resource IDs. Query text, document and chunk content, titles, sources, metadata, tenant IDs and keys never go on a span.

View Source 
const (
	EmbedIngest  = "ingest"
	EmbedQuery   = "query"
	EmbedReindex = "reindex"
)

Embed operations: what an embedding call was for. They are the values of weave.embed.operation on spans and metrics.

View Source 
const (
	ResultOK       = "ok"
	ResultError    = "error"
	ResultNotFound = "not_found"
)

Result values for spans and metric labels.

View Source 
const (
	None   = "none"
	Custom = "custom"
)

None is the component kind for a stage that is not configured, and Custom the kind for a component whose kind is not one Weave ships.

View Source 
const HookErrorInterval = time.Minute

HookErrorInterval is how often one hook of one extension may log an error. Errors in between are counted and reported on the next line, so an extension that fails on every event during an outage logs once a minute, not once per document.

View Source 
const InstrumentationName = "github.com/xraph/weave"

InstrumentationName is the instrumentation scope on every span Weave starts.

View Source 
const OperationEmbeddings = "embeddings"

OperationEmbeddings is gen_ai.operation.name for an embedding call.

View Source 
const TokenInput = "input"

TokenInput is gen_ai.token.type for input tokens. Embeddings have no output tokens.

Variables

View Source 
var (
	// IngestBuckets span a short note on a fast local embedder to a long
	// document embedded over the network, in seconds.
	IngestBuckets = []float64{0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10, 30, 60}
	// RetrieveBuckets span an in-memory search to a reranked hosted one, in
	// seconds.
	RetrieveBuckets = []float64{0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10}
	// VectorBuckets span an in-memory store to a remote one under load, in
	// seconds.
	VectorBuckets = []float64{0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5}
	// GenAIDurationBuckets are the bounds the GenAI conventions advise for
	// gen_ai.client.operation.duration, in seconds.
	GenAIDurationBuckets = []float64{0.01, 0.02, 0.04, 0.08, 0.16, 0.32, 0.64, 1.28, 2.56, 5.12, 10.24, 20.48, 40.96, 81.92}
	// GenAITokenBuckets are the bounds the GenAI conventions advise for
	// gen_ai.client.token.usage, in tokens.
	GenAITokenBuckets = []float64{1, 4, 16, 64, 256, 1024, 4096, 16384, 65536, 262144, 1048576, 4194304, 16777216, 67108864}
)

Histogram bounds, in the metric's unit.

View Source 
var (
	ChunkerKinds     = kindSet("fixed", "recursive", "semantic", "sliding", "code")
	EmbedderKinds    = kindSet("openai", "local", "nexus")
	VectorStoreKinds = kindSet("memory", "pgvector", "fabriq")
	RetrieverKinds   = kindSet("similarity", "mmr", "hybrid", "rerank")
)

Component kinds Weave ships. A component reports its kind through weave.Describer; any kind outside these sets is reported as "custom", so a metric label never carries a string a caller chose.

Functions

func ErrorType 

func ErrorType(err error) string

ErrorType names err for error.type, the exception event and log lines: the context error it is, the Weave sentinel it wraps, or the Go type of the innermost error in its chain. It never returns the error's text, which can quote a URL, a tenant key, a DSN or a fragment of a document. A nil err is "".

func Fail 

func Fail(span trace.Span, err error) string

Fail marks span as failed: error.type, an exception event that carries only exception.type, and an error status whose description is that type. It returns the type. The error's text never reaches the span.

func Kind 

func Kind(x any, allowed KindSet) string

Kind returns x's kind when it describes itself with one of the kinds in allowed, "custom" for any other component, and "none" for a nil x.

func ResultOf 

func ResultOf(err error) string

ResultOf is the result label for err: "ok" for nil, "error" otherwise.

func Tracer 

func Tracer() trace.Tracer

Tracer returns the tracer Weave starts its spans with. It reads the global tracer provider on every call, so whichever provider is installed when the span starts is the one that records it.

Types

type Counter 

type Counter struct {
	// contains filtered or unexported fields
}

Counter is a counter declared with label keys. A nil *Counter records nothing.

func NewCounter 

func NewCounter(m gometrics.Metrics, name, help string, keys ...string) *Counter

NewCounter declares a counter on m. A nil m returns nil. A counter with no labels is created right away, so it is exported as 0 before its first increment and a rate over it never starts from a missing series.

func (*Counter) Add 

func (c *Counter) Add(n float64, values ...string)

Add adds n to the series for values, given in the declared key order.

type Histogram 

type Histogram struct {
	// contains filtered or unexported fields
}

Histogram is a histogram declared with label keys. A nil *Histogram records nothing.

func NewHistogram 

func NewHistogram(m gometrics.Metrics, name, help, unit string, buckets []float64, keys ...string) *Histogram

NewHistogram declares a histogram on m. A nil m returns nil.

func (*Histogram) Observe 

func (h *Histogram) Observe(v float64, values ...string)

Observe records v on the series for values, given in the declared key order.

type HookErrors 

type HookErrors struct {
	// contains filtered or unexported fields
}

HookErrors throttles hook error log lines per hook and extension. The zero value is ready to use and safe for concurrent use.

func (*HookErrors) Allow 

func (h *HookErrors) Allow(hook, ext string, now time.Time) (ok bool, suppressed int)

Allow reports whether an error from hook in ext may be logged at now, and how many errors from the same pair it stood for since the last line.

type KindSet 

type KindSet map[string]struct{}

KindSet is a fixed set of component kinds.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.