app

type Answer ¶

type Answer struct {
	Text      string
	Citations []domain.Citation
	// Grounded reports whether the answer had any grounding input — at least one
	// retrieved chunk or attachment. False means the model answered from its own
	// knowledge alone (only possible in non-strict mode).
	Grounded bool
	// Provenance records the generation identity and determinism settings the
	// generator used, when it captured them. The openai adapter fills it; in-
	// memory fakes leave it nil. It backs an ask manifest's generation block and
	// is nil-safe everywhere.
	Provenance *Provenance
}

Answer is a grounded synthesis result with citations back to chunks, each carrying the source provenance needed to display it.

type AnswerCache ¶

type AnswerCache interface {
	Get(ctx context.Context, key string, notBefore time.Time) (Answer, bool, error)
	Put(ctx context.Context, key string, answer Answer, storedAt time.Time) error
	Prune(ctx context.Context, cutoff time.Time) error
}

AnswerCache stores synthesized answers keyed by an opaque content hash, for reuse across runs (each lore invocation is a fresh process, so this must be persistent to pay off). It is deliberately time-explicit: the TTL policy lives in the caller, not the store, which keeps the store a dumb time-indexed KV and the conformance suite deterministic.

Semantics:

• Get returns a hit only when an entry exists AND was stored at or after notBefore; an older entry is a miss (expired), as is an unknown key.
• Put replaces any existing entry for key.
• Prune deletes every entry stored before cutoff.

type Asker ¶

type Asker struct {
	// contains filtered or unexported fields
}

Asker answers a question with a synthesis grounded in retrieved chunks.

func NewAsker(querier *Querier, generator Generator) *Asker

func (a *Asker) Ask(ctx context.Context, collection, question string, k int, attachments []domain.Attachment, strict bool, source string, filter domain.Predicate) (Answer, error)

func (a *Asker) AskExplain(ctx context.Context, collection, question string, k int, attachments []domain.Attachment, strict bool, source string, filter domain.Predicate) (Answer, Retrieval, error)

func (a *Asker) Synthesize(ctx context.Context, question string, hits []domain.ChunkHit, attachments []domain.Attachment) (Answer, error)

func (a *Asker) SynthesizeReproducible(ctx context.Context, question string, hits []domain.ChunkHit, attachments []domain.Attachment) (Answer, error)

func (a *Asker) SynthesizeStream(ctx context.Context, question string, hits []domain.ChunkHit, attachments []domain.Attachment, onDelta func(string)) (Answer, error)

type CaseResult ¶ added in v1.0.0

type CaseResult struct {
	Question     string
	HasRetrieval bool // the case declared expected_sources/chunks
	Recall       float64
	Precision    float64
	MRR          float64
	NDCG         float64
	HitRate      float64
	HasVerify    bool
	SupportRate  float64
	Claims       []domain.Claim
}

CaseResult is the per-question outcome: the retrieval metrics (when the case declared expectations) and, when verification ran, the answer's claims and support rate.

type Catalog ¶

type Catalog struct {
	// contains filtered or unexported fields
}

Catalog is the collection-management use case: create, list, and inspect the corpora lore knows about — including the documents ingested into a collection.

func NewCatalog(collections CollectionRepository, docs DocumentRepository, embedder Embedder, chunkers domain.Registry) *Catalog

func (c *Catalog) ChunksByIDs(ctx context.Context, collection string, ids []string) ([]domain.Chunk, error)

func (c *Catalog) Diff(ctx context.Context, from, to string) (CollectionDiff, error)

func (c *Catalog) DocumentChunks(ctx context.Context, collection, sourceURI string) ([]domain.Chunk, error)

func (c *Catalog) Get(ctx context.Context, name string) (*domain.Collection, error)

func (c *Catalog) Init(ctx context.Context, name string) (*domain.Collection, error)

func (c *Catalog) List(ctx context.Context) ([]*domain.Collection, error)

func (c *Catalog) ListDocuments(ctx context.Context, collection string) ([]*domain.Document, error)

type Checker ¶ added in v1.0.0

type Checker struct {
	// contains filtered or unexported fields
}

Checker verifies an answer's faithfulness: it segments the answer into claims and judges whether each claim is entailed by the chunks it cites, via the Verifier port. It is the use case behind `ask --verify`.

func NewChecker(verifier Verifier, catalog *Catalog) *Checker

func (c *Checker) Verify(ctx context.Context, collection string, ans Answer) ([]domain.Claim, error)

func (c *Checker) VerifyWithEvidence(ctx context.Context, ans Answer, evidenceByID map[domain.ChunkID]string) ([]domain.Claim, error)

type CollectionDiff ¶ added in v1.0.0

type CollectionDiff struct {
	Added   []DocRef    // present in `to`, absent from `from`
	Removed []DocRef    // present in `from`, absent from `to`
	Changed []DocChange // same source URI, different content hash
}

CollectionDiff is the document-level difference between two collections, compared by source URI and content hash. It is space-independent: it compares what was ingested, not how it was embedded, so two collections pinned to different embedding spaces (e.g. a collection and a re-imported snapshot) can still be diffed. Each slice is ordered by source URI.

func (d CollectionDiff) Empty() bool

type CollectionRepository ¶

type CollectionRepository interface {
	// Create fails with ErrAlreadyExists if the name is taken.
	Create(ctx context.Context, c *domain.Collection) error
	// Get fails with ErrNotFound if no such collection exists.
	Get(ctx context.Context, name string) (*domain.Collection, error)
	List(ctx context.Context) ([]*domain.Collection, error)
	// Delete removes the collection record, failing with ErrNotFound if no
	// such collection exists. Cascading removal of its documents and vectors
	// is orchestrated by the use case, which holds the
	// DocumentRepository and VectorIndex; this port cannot reach them.
	Delete(ctx context.Context, name string) error
	// RecordSource adds source to the collection's remembered Sources,
	// idempotently (recording an existing source is a no-op). It fails with
	// ErrNotFound if no such collection exists. Get reflects recorded sources.
	RecordSource(ctx context.Context, name, source string) error
}

CollectionRepository persists Collection aggregates.

type CorpusDrift ¶ added in v1.0.0

type CorpusDrift struct {
	Collection string
	Was        domain.ContentHash
	Now        domain.ContentHash
}

CorpusDrift names a collection whose content digest changed since the manifest was recorded — the corpus the exhibit was grounded in no longer exists.

type CorpusRef ¶ added in v1.0.0

type CorpusRef struct {
	Collection string             `json:"collection"`
	Digest     domain.ContentHash `json:"digest"`
	Model      string             `json:"embedding_model"`
	Dimensions int                `json:"dimensions"`
}

CorpusRef pins one collection's content identity (CorpusSnapshot digest) and embedding space at ask time. Replay compares each against the live collection; a changed digest means the corpus drifted and the exhibit cannot be reproduced.

func CorpusRefsOf(ctx context.Context, cat *Catalog, collections []string) ([]CorpusRef, error)

type DeterministicGenerator ¶ added in v1.0.0

type DeterministicGenerator interface {
	SynthesizeDeterministic(ctx context.Context, question string, hits []domain.ChunkHit, attachments []domain.Attachment) (Answer, error)
}

DeterministicGenerator is an optional capability a Generator may implement: it synthesizes with the generation pinned for reproducibility (temperature 0 and a fixed seed) and records the pinned values in the returned Answer's Provenance. The audited ask --reproducible path requires it; callers detect support with a type assertion (see Asker.SynthesizeReproducible) and fail with ErrReproducibleUnsupported when it is absent rather than silently producing a non-deterministic answer.

type DocChange ¶ added in v1.0.0

type DocChange struct {
	SourceURI string
	From      string
	To        string
}

DocChange is one document whose content hash differs between the two collections (From in `from`, To in `to`).

type DocRef ¶ added in v1.0.0

type DocRef struct {
	SourceURI string
	Hash      string
}

DocRef identifies one document by its source URI and content hash.

type DocumentRepository ¶

type DocumentRepository interface {
	// Upsert stores the document and replaces its chunks atomically.
	Upsert(ctx context.Context, doc *domain.Document, chunks []domain.Chunk) error
	// GetBySource fails with ErrNotFound if the source was never ingested.
	GetBySource(ctx context.Context, collection, sourceURI string) (*domain.Document, error)
	// GetChunks hydrates chunks by ID, preserving input order. IDs with no
	// stored chunk are skipped, so the result may be shorter than the input.
	GetChunks(ctx context.Context, ids []domain.ChunkID) ([]domain.Chunk, error)
	// GetChunksByDocument returns all chunks of one document in seq order. An
	// unknown document (or collection) yields no chunks and no error; existence
	// is the caller's concern (resolve the document via GetBySource first).
	GetChunksByDocument(ctx context.Context, collection string, id domain.DocumentID) ([]domain.Chunk, error)
	// GetChunksByIDs returns the chunks with the given IDs that belong to the
	// collection, in input order. IDs with no stored chunk in the collection are
	// omitted (the caller diffs requested vs returned). An unknown collection
	// yields no chunks and no error; collection existence is the caller's concern.
	GetChunksByIDs(ctx context.Context, collection string, ids []string) ([]domain.Chunk, error)
	// GetDocuments hydrates documents by ID, preserving input order. IDs with no
	// stored document are skipped, so the result may be shorter than the input.
	// Used to attach source provenance to retrieval results.
	GetDocuments(ctx context.Context, ids []domain.DocumentID) ([]*domain.Document, error)
	// ListDocuments returns every document in the collection. An unknown or empty
	// collection yields no documents and no error; collection existence is the
	// CollectionRepository's concern. Order is unspecified.
	ListDocuments(ctx context.Context, collection string) ([]*domain.Document, error)
	// Delete removes the document and its chunks, returning the removed chunk
	// IDs so the use case can delete their vectors via the VectorIndex
	// (this port cannot reach it). Fails with ErrNotFound if no
	// such document exists in the collection.
	Delete(ctx context.Context, collection string, id domain.DocumentID) ([]domain.ChunkID, error)
	// DeleteChunks removes the chunks with the given IDs that belong to the
	// collection, returning the IDs actually removed so the use case can delete
	// their vectors. The owning documents are left in place — a
	// document keeps its record even after losing chunks (sub-document
	// redaction, not document deletion). IDs absent from the collection
	// (unknown, or owned by another collection) are skipped, so the caller can
	// diff requested vs removed; an unknown collection removes nothing. No error
	// for skips — a malformed-ID guard is the caller's concern.
	DeleteChunks(ctx context.Context, collection string, ids []domain.ChunkID) ([]domain.ChunkID, error)
	// DeleteCollection removes every document and its chunks in the collection,
	// returning all removed chunk IDs for the same cascade. A collection with no
	// documents is a no-op (no IDs, no error); collection existence is the
	// CollectionRepository's concern.
	DeleteCollection(ctx context.Context, collection string) ([]domain.ChunkID, error)
}

DocumentRepository persists Documents and their Chunks.

type Embedder ¶

type Embedder interface {
	Space(ctx context.Context) (domain.EmbeddingSpace, error)
	// Embed returns one vector per input text, in input order.
	Embed(ctx context.Context, texts []string) ([][]float32, error)
}

Embedder turns texts into vectors and reports the space it produces, so use cases can enforce space coherence against the target collection.

type EvalCase ¶ added in v1.0.0

type EvalCase struct {
	Question        string   `json:"question"`
	ExpectedSources []string `json:"expected_sources,omitempty"`
	ExpectedChunks  []string `json:"expected_chunks,omitempty"`
	ExpectedAnswer  string   `json:"expected_answer,omitempty"`
}

EvalCase is one question in an eval set with its optional expectations. At least one of ExpectedSources/ExpectedChunks enables retrieval metrics for the case; faithfulness (support rate) is computed when verification is requested.

func ParseEvalSet(r io.Reader) ([]EvalCase, error)

type EvalReport ¶ added in v1.0.0

type EvalReport struct {
	K          int
	Cases      []CaseResult
	Aggregates map[string]float64 // metric name → mean (see metric name constants)
}

EvalReport is the whole-set outcome: per-case results and the aggregate means over the applicable cases.

type Evaluator ¶ added in v1.0.0

type Evaluator struct {
	// contains filtered or unexported fields
}

Evaluator runs an eval set against a collection, computing retrieval metrics and — when verify is set — answer faithfulness, via the existing use cases.

func NewEvaluator(asker *Asker, checker *Checker) *Evaluator

func (e *Evaluator) Evaluate(ctx context.Context, collection string, cases []EvalCase, k int, verify bool, retrieve func(context.Context, string) ([]domain.ChunkHit, error)) (EvalReport, error)

type Exporter ¶

type Exporter struct {
	// contains filtered or unexported fields
}

Exporter serializes a collection into a portable artifact — its embedding-space and chunker pins, metadata, documents, chunks, and vectors — gathered through the persistence ports so the format stays independent of the storage backend.

func NewExporter(collections CollectionRepository, docs DocumentRepository, index VectorIndex) *Exporter

func (e *Exporter) Export(ctx context.Context, collection string, w io.Writer) (TransferSummary, error)

type Extractor ¶

type Extractor interface {
	Supports(contentType string) bool
	Extract(contentType string, raw []byte) (string, error)
}

Extractor converts raw content of a supported type into plain text.

type FromQuery ¶

type FromQuery struct {
	From domain.ChunkHit   // the source chunk's identity + source URI (Score unused)
	Hits []domain.ChunkHit // its target hits, best first
}

FromQuery is one source-collection chunk used directly as a query vector and the target-collection hits it retrieved. It backs `query --from-collection`, which feeds a collection's own stored vectors back as queries (never re-embedding) to find each chunk's nearest neighbors in another collection.

type GenerationManifest ¶ added in v1.0.0

type GenerationManifest struct {
	ChatModel         string   `json:"chat_model"`
	ResolvedModel     string   `json:"resolved_model,omitempty"`
	SystemFingerprint string   `json:"system_fingerprint,omitempty"`
	Deterministic     bool     `json:"deterministic"`
	Temperature       *float64 `json:"temperature,omitempty"`
	Seed              *int     `json:"seed,omitempty"`
}

GenerationManifest records the generation identity. ChatModel is the configured model name (always present). ResolvedModel and SystemFingerprint are the provider's response-reported identity, present when the adapter captured them. Deterministic, Temperature, and Seed are set only for a --reproducible run that pinned them; nil Temperature/Seed means the run used the provider default (non-deterministic).

type Generator ¶

type Generator interface {
	Synthesize(ctx context.Context, question string, hits []domain.ChunkHit, attachments []domain.Attachment) (Answer, error)
}

Generator synthesizes an answer grounded in retrieved chunks, optionally with raw attachments (images/documents) the model reads directly. Attachments are ephemeral context, not part of the collection.

type Importer ¶

type Importer struct {
	// contains filtered or unexported fields
}

Importer reconstructs a collection from a portable artifact, writing it into the local store through the persistence ports. The chunk and document IDs are re-derived from the (target collection, source URI, seq) rather than carried, so importing under a new name produces a coherent collection.

func NewImporter(collections CollectionRepository, docs DocumentRepository, index VectorIndex, remover *Remover, lexical LexicalIndex, embedder Embedder) *Importer

func (im *Importer) Import(ctx context.Context, r io.Reader, name string, force, reEmbed bool) (TransferSummary, error)

type IngestCallOption ¶ added in v1.0.0

type IngestCallOption func(*ingestCall)

IngestCallOption configures a single Ingest or IngestContent invocation (as distinct from IngestOption, which configures the Ingestor at construction).

func WithExclude(globs ...string) IngestCallOption

func WithMeta(meta domain.Metadata) IngestCallOption

type IngestOption ¶

type IngestOption func(*Ingestor)

IngestOption configures an Ingestor at construction.

func WithConcurrency(n int) IngestOption

type IngestSummary ¶

type IngestSummary struct {
	Added       int // documents ingested (new or changed)
	Skipped     int // unchanged or empty documents (idempotent no-ops)
	Unsupported int // documents whose content type no extractor handles
	Excluded    int // documents skipped by an --exclude glob (never read)
	Chunks      int // chunks embedded and stored
}

IngestSummary reports the outcome of an ingestion run.

type Ingestor ¶

type Ingestor struct {
	// contains filtered or unexported fields
}

Ingestor walks a Source, extracts and chunks each document, embeds the chunks, and stores chunks and their vectors. Ingestion is idempotent: a source whose extracted content is unchanged is a no-op.

func NewIngestor(collections CollectionRepository, docs DocumentRepository, index VectorIndex, embedder Embedder, extractor Extractor, source Source, chunkers domain.Registry, lexical LexicalIndex, opts ...IngestOption) *Ingestor

func (i *Ingestor) Ingest(ctx context.Context, collection, root string, opts ...IngestCallOption) (IngestSummary, error)

func (i *Ingestor) IngestContent(ctx context.Context, collection, uri, contentType string, content []byte, opts ...IngestCallOption) (IngestSummary, error)

type LexicalDoc ¶ added in v1.0.0

type LexicalDoc struct {
	ChunkID  domain.ChunkID
	Text     string
	Metadata domain.Metadata
}

LexicalDoc is one chunk's lexical content for the LexicalIndex: its identity, the text to index, and the document metadata a --where filter applies to. It is the lexical-side sibling of VectorEntry.

type LexicalIndex ¶ added in v1.0.0

type LexicalIndex interface {
	Upsert(ctx context.Context, collection string, docs []LexicalDoc) error
	Search(ctx context.Context, collection string, query string, k int, filter domain.Predicate) ([]domain.ChunkID, error)
	Delete(ctx context.Context, collection string, ids []domain.ChunkID) error
}

LexicalIndex is a keyword (BM25) index over chunk text — the lexical half of hybrid retrieval, a sibling of VectorIndex kept separate so each port stays small and every adapter implements it honestly (memstore in-memory BM25, sqlite FTS5). Results feed rank-based fusion (domain.FuseRRF) with the vector results, so Search returns only the ranked identities, not scores.

Semantics:

• Upsert replaces entries with the same ChunkID.
• Search returns up to k chunk IDs ranked by lexical relevance, best first, considering only entries whose Metadata satisfies filter. An empty query, unknown collection, or k <= 0 yields no matches, no error. A chunk is a candidate when it contains at least one query term.
• Delete of absent IDs is a no-op.

type Manifest ¶ added in v1.0.0

type Manifest struct {
	Question     string             `json:"question"`
	AskedAt      time.Time          `json:"asked_at"`
	Corpus       []CorpusRef        `json:"corpus"`
	Retrieval    RetrievalManifest  `json:"retrieval"`
	Generation   GenerationManifest `json:"generation"`
	CitedChunks  []string           `json:"cited_chunks"`
	AnswerDigest domain.ContentHash `json:"answer_digest"`
}

Manifest is the reproducible provenance record of one `ask`: the corpus state, retrieval configuration, and generation identity that produced an answer, the chunks it actually cited, and a digest of the answer text. It turns "trust me, it was grounded" into an auditable, re-runnable exhibit: `lore replay` checks the corpus has not drifted, re-runs the recorded retrieval, and re-synthesizes from the cited chunks to prove the answer reproduces.

type Provenance ¶ added in v1.0.0

type Provenance struct {
	ResolvedModel     string
	SystemFingerprint string
	Deterministic     bool
	Temperature       float64
	Seed              int
}

Provenance records how an answer was generated: the model the provider actually served (ResolvedModel) and its SystemFingerprint, plus whether the request was pinned for reproducibility (Deterministic, with the Temperature and Seed used). It is the generation half of a reproducible ask manifest.

type Querier ¶

type Querier struct {
	// contains filtered or unexported fields
}

Querier retrieves the chunks most similar to a query within a Collection.

func NewQuerier(collections CollectionRepository, index VectorIndex, docs DocumentRepository, embedder Embedder, lexical LexicalIndex) *Querier

func (q *Querier) Explain(ctx context.Context, collection, query string, k int, source string, filter domain.Predicate) (Retrieval, error)

func (q *Querier) ExplainAcross(ctx context.Context, collections []string, query string, k int, source string, filter domain.Predicate) (Retrieval, error)

func (q *Querier) ExplainHybrid(ctx context.Context, collection, query string, k int, source string, filter domain.Predicate) (Retrieval, error)

func (q *Querier) ExplainLexical(ctx context.Context, collection, query string, k int, source string, filter domain.Predicate) (Retrieval, error)

func (q *Querier) Query(ctx context.Context, collection, query string, k int, source string, filter domain.Predicate) ([]domain.ChunkHit, error)

func (q *Querier) QueryAcross(ctx context.Context, collections []string, query string, k int, source string, filter domain.Predicate) ([]domain.ChunkHit, error)

func (q *Querier) QueryFrom(ctx context.Context, target, sourceCollection string, k int, source string, filter domain.Predicate) ([]FromQuery, error)

func (q *Querier) QueryHybrid(ctx context.Context, collection, query string, k int, source string, filter domain.Predicate) ([]domain.ChunkHit, error)

func (q *Querier) QueryLexical(ctx context.Context, collection, query string, k int, source string, filter domain.Predicate) ([]domain.ChunkHit, error)

type RankResult ¶

type RankResult struct {
	Index int
	Score float64
}

RankResult is one scored document from a RerankProvider: Index points into the documents slice the provider was given; Score is its relevance (higher = more relevant).

type Remover ¶

type Remover struct {
	// contains filtered or unexported fields
}

Remover owns the cross-port deletion cascade: a document's chunks live in the DocumentRepository, their vectors in the VectorIndex, and the collection record in the CollectionRepository. No single port can reach the others, so this use case orchestrates them.

func NewRemover(collections CollectionRepository, docs DocumentRepository, index VectorIndex, lexical LexicalIndex) *Remover

func (r *Remover) RemoveChunks(ctx context.Context, collection string, ids []domain.ChunkID) ([]domain.ChunkID, error)

func (r *Remover) RemoveCollection(ctx context.Context, name string) error

func (r *Remover) RemoveDocument(ctx context.Context, collection, sourceURI string) error

type ReplayReport ¶ added in v1.0.0

type ReplayReport struct {
	Drift          []CorpusDrift
	Attempted      bool
	RetrievalMatch bool
	MissingCited   []string
	AnswerMatch    bool
	AnswerDigest   domain.ContentHash
	ExpectedAnswer domain.ContentHash
}

ReplayReport is the outcome of re-running a manifest. Attempted is false when corpus drift aborted reproduction (the fail-closed default); RetrievalMatch is true when every cited chunk still surfaced in re-retrieval; AnswerMatch is true when the re-synthesized answer's digest equals the manifest's.

func (r ReplayReport) Reproduced(allowDrift bool) bool

type Replayer ¶ added in v1.0.0

type Replayer struct {
	// contains filtered or unexported fields
}

Replayer re-runs an ask Manifest to prove its answer reproduces: it checks the corpus has not drifted (each collection's CorpusSnapshot digest still matches), re-runs the recorded retrieval to confirm the cited evidence still surfaces, and re-synthesizes from the pinned cited chunks to confirm the answer text reproduces. It is the verification half of the provenance exhibit.

func NewReplayer(catalog *Catalog, retriever *Retriever, asker *Asker, docs DocumentRepository) *Replayer

func (r *Replayer) Replay(ctx context.Context, m Manifest, allowDrift bool) (ReplayReport, error)

type RerankProvider ¶

type RerankProvider interface {
	// Rerank scores documents against query and returns results best-first. topN
	// is a hint passed to providers that support it; the Reranker use case is the
	// source of truth for final ordering and truncation.
	Rerank(ctx context.Context, query string, documents []string, topN int) ([]RankResult, error)
}

RerankProvider scores (query, document) pairs jointly with a cross-encoder — the rerank half of two-stage retrieval. It is a distinct port from Embedder/ Generator because rerank APIs are their own thing (Cohere-style /rerank), not the OpenAI chat/embeddings shape, and are usually a separate provider.

type Reranker ¶

type Reranker struct {
	// contains filtered or unexported fields
}

Reranker is the two-stage-retrieval use case: it reranks already-retrieved hits against the query via a RerankProvider, reordering them by relevance and attaching each hit's rerank score (the original similarity score is kept).

func NewReranker(provider RerankProvider) *Reranker

func (r *Reranker) Rerank(ctx context.Context, query string, hits []domain.ChunkHit, topN int) ([]domain.ChunkHit, error)

type Retrieval ¶

type Retrieval struct {
	Hits      []domain.ChunkHit
	NextScore float64
	HasNext   bool
}

Retrieval is what Explain returns: the top-k Hits plus the best candidate just outside them (the runner-up), for --explain diagnostics. HasNext is false when there is no further candidate (fewer than k+1 matched after filtering).

type RetrievalManifest ¶ added in v1.0.0

type RetrievalManifest struct {
	K            int      `json:"k"`
	Candidates   int      `json:"candidates,omitempty"`
	Source       string   `json:"source,omitempty"`
	Where        []string `json:"where,omitempty"`
	Rerank       bool     `json:"rerank,omitempty"`
	Hybrid       bool     `json:"hybrid,omitempty"`
	MMR          bool     `json:"mmr,omitempty"`
	MMRLambda    float64  `json:"mmr_lambda,omitempty"`
	Recency      bool     `json:"recency,omitempty"`
	HalfLifeDays float64  `json:"half_life_days,omitempty"`
	MaxPerSource int      `json:"max_per_source,omitempty"`
	Budget       int      `json:"budget,omitempty"`
}

RetrievalManifest records the retrieval levers exactly as resolved, so replay reconstructs the same RetrieveOptions and re-runs an identical pipeline.

type RetrieveOptions ¶ added in v1.0.0

type RetrieveOptions struct {
	Collections []string
	Query       string
	K           int
	Candidates  int // rerank candidate pool (must be >= K when Rerank)
	Source      string
	Filter      domain.Predicate
	Rerank      bool
	Explain     bool // surface the runner-up just outside the top-k
	Hybrid      bool
	// Lexical selects BM25-only retrieval (no query embedding), for querying with
	// no usable embedder. It is a distinct mode, mutually exclusive with the
	// vector-based reorderings (Hybrid/Rerank/MMR/Recency) and single-collection.
	Lexical      bool
	MMR          bool
	MMRLambda    float64
	Recency      bool
	HalfLife     time.Duration
	MaxPerSource int // 0 = no cap
}

RetrieveOptions configures Retriever.Resolve — the single source of truth for retrieval composition shared by the CLI, the eval harness, and the MCP server. Rerank, MMR, and Recency each reorder the candidate pool and are mutually exclusive. Hybrid, Source, Filter, and MaxPerSource compose with any mode.

type Retriever ¶ added in v1.0.0

type Retriever struct {
	// contains filtered or unexported fields
}

Retriever resolves a query into ranked hits, composing vector search with optional hybrid fusion, cross-encoder rerank, MMR diversification, recency decay, and a per-source cap. It is the one place that composition lives; drivers (CLI/eval/MCP) translate their flags into RetrieveOptions and call it.

func NewRetriever(querier *Querier, rerank *Reranker, index VectorIndex) *Retriever

func (r *Retriever) Resolve(ctx context.Context, opts RetrieveOptions) ([]domain.ChunkHit, *float64, error)

type Source ¶

type Source interface {
	Walk(ctx context.Context, root string, fn func(SourceItem) error) error
}

Source yields raw documents from somewhere (filesystem walk, stdin, URL). Walk stops and returns the first error from fn, or ctx.Err on cancellation.

type SourceItem ¶

type SourceItem struct {
	URI         string
	ContentType string
	Fingerprint string
	// ModTime is the source's last-modified time, recorded as recency provenance
	// (the reserved `mtime` metadata key). Zero when the source has no meaningful
	// timestamp (e.g. stdin).
	ModTime time.Time
	Open    func() ([]byte, error)
}

SourceItem is one raw document yielded by a Source. Content is read lazily via Open so a consumer can decide — from the cheap Fingerprint — not to read at all. Fingerprint is a source-side signature (size + sampled-content hash) that changes when the file changes; an empty Fingerprint disables fast-skip.

type StreamingGenerator ¶

type StreamingGenerator interface {
	SynthesizeStream(ctx context.Context, question string, hits []domain.ChunkHit, attachments []domain.Attachment, onDelta func(string)) (Answer, error)
}

StreamingGenerator is an optional capability a Generator may implement: it emits the answer's prose incrementally via onDelta (called in order, once per token or chunk) while still returning the complete Answer — the full text and citations a caller needs for the post-answer sources, --json, and caching. A generator that cannot stream a given request (structured-output mode, or a cache hit) may call onDelta once with the whole text. Callers detect support with a type assertion; the CLI uses it only for interactive streaming.

type SyncSummary ¶

type SyncSummary struct {
	Added       int      // documents ingested (new or changed)
	Skipped     int      // unchanged or empty documents (idempotent no-ops)
	Unsupported int      // documents whose content type no extractor handles
	Chunks      int      // chunks embedded and stored
	Pruned      int      // documents removed (or, in a dry run, that would be)
	PrunedURIs  []string // source URIs of the pruned documents, sorted
}

SyncSummary reports the outcome of a sync run.

type Syncer ¶

type Syncer struct {
	// contains filtered or unexported fields
}

Syncer brings a collection into line with its source(s). It re-ingests (add/update, idempotently) and, when prune is set, removes documents whose source files no longer exist. With no paths it replays the sources the collection remembers from prior add/sync runs.

func NewSyncer(catalog *Catalog, ingestor *Ingestor, remover *Remover, source Source) *Syncer

func (s *Syncer) Sync(ctx context.Context, collection string, paths []string, prune, dryRun bool) (SyncSummary, error)

type TokenCounter ¶

type TokenCounter interface {
	Count(text string) int
}

TokenCounter approximates how many tokens a piece of text occupies in an embedding/generation model's context window. It backs the entire token-awareness story: chunk sizing (the domain chunkers take its Count as an injected func, keeping the tokenizer dependency out of stdlib-only domain) and budget-bounded retrieval. Counts are approximate-but-stable across models, not exact per-model — enough to size windows predictably.

type TransferSummary ¶

type TransferSummary struct {
	Collection string
	Model      string
	Dimensions int
	Documents  int
	Chunks     int
}

TransferSummary describes a collection that was exported or imported.

type VectorEntry ¶

type VectorEntry struct {
	ChunkID  domain.ChunkID
	Vector   []float32
	Metadata domain.Metadata
}

VectorEntry pairs a chunk identity with its vector for indexing. Metadata is the chunk's document-level attributes, carried on the entry so the index can apply a --where filter during Search without reaching into the DocumentRepository (the two are separate ports, possibly separate engines).

type VectorIndex ¶

type VectorIndex interface {
	Upsert(ctx context.Context, collection string, entries []VectorEntry) error
	Search(ctx context.Context, collection string, query []float32, k int, filter domain.Predicate) ([]domain.VectorMatch, error)
	// Entries returns every stored (ChunkID, Vector) for the collection, in
	// unspecified order, as copies the caller may retain. An unknown collection
	// yields no entries and no error (mirrors Search). It feeds a collection's
	// own vectors back as queries (query --from-collection) without re-embedding.
	Entries(ctx context.Context, collection string) ([]VectorEntry, error)
	Delete(ctx context.Context, collection string, ids []domain.ChunkID) error
}

VectorIndex stores and searches vectors. It is deliberately dumb: space coherence is enforced by the use cases via Collection.AcceptsSpace before anything reaches the index.

Semantics:

• Upsert replaces entries with the same ChunkID.
• Search returns up to k matches, best first (higher score = more similar), considering only entries whose Metadata satisfies filter; the filter is applied before the top-k cut, so the result is exact (no over-fetch). The zero Predicate matches every entry, i.e. no filtering. Unknown collection or k <= 0 yields no matches, no error.
• Delete of absent IDs is a no-op.

type Verdict ¶ added in v1.0.0

type Verdict struct {
	Supported bool
	Rationale string
}

Verdict is the entailment judgment for one claim against candidate evidence: whether the evidence supports the claim, with an optional short rationale.

type Verifier ¶ added in v1.0.0

type Verifier interface {
	Verify(ctx context.Context, claim, evidence string) (Verdict, error)
}

Verifier judges whether a claim is entailed by evidence text — the model call behind faithfulness verification (ask --verify). The default implementation reuses the chat model via a structured entailment prompt, so it needs no new dependency and inherits the chat endpoint configuration.