llm

type CallStat ¶ added in v0.7.1

type CallStat struct {
	// Purpose names what the call was for. Chat calls carry a pkg/llm Purpose
	// constant; embedding calls carry their own op names, which is why the
	// field is a plain string.
	Purpose  string
	Identity pkgllm.Identity
	Usage    pkgllm.Usage
	// Items is the number of inputs in this call. The embedding path sets it
	// (one batch = N texts) so throughput (items or tokens per second) is
	// derivable from DurationMS; chat calls are single-prompt and leave it 0.
	Items      int
	DurationMS int64
	// Error is the failure text when the call did not return a result, empty
	// on success. Failures are recorded because a call that times out or comes
	// back unparseable is exactly what the sink exists to make countable —
	// dropping it hides the brittleness it is evidence of. Such a row carries
	// no tokens, and provider and model only when the failure happened past
	// the point they were known.
	Error string
}

CallStat is one LLM call's metrics, handed to a StatsSink for durable collection. The timestamp is added by the sink implementation. The sink owns the wire shape; this is the in-process form.

type Embedder ¶ added in v0.4.0

type Embedder interface {
	// EmbedDocuments embeds texts as index-side passages, applying the
	// configured DocumentTemplate. Used by the indexer at build /
	// lazy-fill time.
	EmbedDocuments(ctx context.Context, texts []string) ([][]float32, error)
	// EmbedQueries embeds texts as retrieval-side queries, applying the
	// configured QueryTemplate. Used by the search finder at query time.
	EmbedQueries(ctx context.Context, texts []string) ([][]float32, error)
	Dimensions() int
	Fingerprint() string
	// BatchSize is the per-call input cap the embedder targets. Drivers
	// the indexer's outer bucketing so each Embed call corresponds to
	// exactly one HTTP round-trip — progress callbacks fire per batch
	// instead of "all-at-once at the end" of a giant cross-entry call.
	// Larger inputs still embed correctly; the embedder splits
	// internally and the indexer just sees a slower individual call.
	BatchSize() int
}

Embedder turns a batch of texts into dense vectors. Implementations are thin transport adapters around an embedding service (OpenAI-compatible `/v1/embeddings`, Ollama `/api/embed`); construction is via embed.New.

The interface splits embedding into a document-side and a query-side because instruction-tuned encoders (Qwen3, E5, Nomic, BGE) want asymmetric prefixes — `passage:` on documents, `query:` on queries, or `Instruct: …\nQuery:…` only on queries. The split makes the call-site intent explicit so the wrong template can't silently slip onto the wrong side. Untemplated models (OpenAI text-embedding-3) treat both methods as equivalent.

Dimensions returns the vector length the implementation produces — must be stable across calls so the index can validate row shape.

Fingerprint is an opaque identifier for the (provider + model + truncation + document template) tuple. Used by the search index to detect rows whose embedding is stale after a configuration change. The format is implementation-defined, but implementations must guarantee that two embedders that produce distribution-comparable document vectors share a fingerprint, and two that don't, don't. The query template deliberately does NOT factor into the fingerprint — query template changes affect retrieval quality but never invalidate indexed embeddings, so they're a free-tweak knob (see EmbeddingConfig).

type StatsSink ¶ added in v0.7.1

type StatsSink interface {
	RecordCall(CallStat)
}

StatsSink durably records per-call LLM metrics (e.g. to a local JSONL file). Implementations must be safe for concurrent use — batch operations like `sdd summarize --all` call them from multiple goroutines.