app

package
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 14, 2026 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Overview

Package app contains the use cases and defines the ports (interfaces) they consume. Adapters implement these ports; the conformance suites in internal/conformance are their executable contracts.

Index

Constants

View Source 
const DefaultIngestConcurrency = 8

DefaultIngestConcurrency bounds how many documents are processed at once.

Variables

View Source 
var (
	ErrNotFound      = errors.New("not found")
	ErrAlreadyExists = errors.New("already exists")
	// ErrNoGrounding is returned by Ask in strict mode when retrieval yields no
	// chunks and no attachments were supplied — there is nothing to ground an
	// answer in, so the LLM is not called.
	ErrNoGrounding = errors.New("no grounding")
)

Sentinel errors shared across ports. Adapters wrap these with %w so callers can use errors.Is; the CLI maps them to exit codes.

Functions

This section is empty.

Types

type Answer 

type Answer struct {
	Text      string
	Citations []domain.Citation
	// Grounded reports whether the answer had any grounding input — at least one
	// retrieved chunk or attachment. False means the model answered from its own
	// knowledge alone (only possible in non-strict mode).
	Grounded bool
}

Answer is a grounded synthesis result with citations back to chunks, each carrying the source provenance needed to display it.

type AnswerCache 

type AnswerCache interface {
	Get(ctx context.Context, key string, notBefore time.Time) (Answer, bool, error)
	Put(ctx context.Context, key string, answer Answer, storedAt time.Time) error
	Prune(ctx context.Context, cutoff time.Time) error
}

AnswerCache stores synthesized answers keyed by an opaque content hash, for reuse across runs (each lore invocation is a fresh process, so this must be persistent to pay off). It is deliberately time-explicit: the TTL policy lives in the caller, not the store, which keeps the store a dumb time-indexed KV and the conformance suite deterministic.

Semantics:

  • Get returns a hit only when an entry exists AND was stored at or after notBefore; an older entry is a miss (expired), as is an unknown key.
  • Put replaces any existing entry for key.
  • Prune deletes every entry stored before cutoff.

type Asker 

type Asker struct {
	// contains filtered or unexported fields
}

Asker answers a question with a synthesis grounded in retrieved chunks.

func NewAsker 

func NewAsker(querier *Querier, generator Generator) *Asker

NewAsker wires an Asker over a Querier and a Generator.

func (*Asker) Ask 

func (a *Asker) Ask(ctx context.Context, collection, question string, k int, attachments []domain.Attachment, strict bool, source string) (Answer, error)

Ask retrieves the top-k chunks for the question and synthesizes an answer grounded in them, plus any attachments passed straight to the generator. Retrieval errors short-circuit before the generator is called; generator errors are wrapped. With k <= 0 no chunks are retrieved, so attachments alone ground the answer.

When retrieval yields no chunks and no attachments were supplied there is nothing to ground on: in strict mode this is ErrNoGrounding (the generator is not called, saving the request); otherwise the answer is still produced but marked Grounded=false so the caller can warn that it rests on model knowledge alone. A non-empty source restricts retrieval to documents matching that glob (see Querier.Query).

func (*Asker) AskExplain 

func (a *Asker) AskExplain(ctx context.Context, collection, question string, k int, attachments []domain.Attachment, strict bool, source string) (Answer, Retrieval, error)

AskExplain is Ask, additionally returning the retrieval (the grounding hits plus the runner-up just outside them) so a caller can show which chunks fed the answer, at what scores, and whether the cutoff was arbitrary — distinguishing retrieval starvation (every score low) from a synthesis miss (a high-scoring chunk the model left uncited). Strict/source/attachment semantics match Ask. The hits are the post-filter top-k, best first; the Retrieval is zero when retrieval found nothing or strict mode returns ErrNoGrounding.

func (*Asker) Synthesize 

func (a *Asker) Synthesize(ctx context.Context, question string, hits []domain.ChunkHit, attachments []domain.Attachment) (Answer, error)

Synthesize generates an answer from already-retrieved hits and optional attachments, without performing retrieval — the generation half of Ask, exposed so callers can interpose between retrieval and synthesis (filter, re-rank, or merge hits). Grounded reflects whether any hits or attachments were given. Unlike Ask it has no strict mode: the caller chose the hits.

func (*Asker) SynthesizeStream 

func (a *Asker) SynthesizeStream(ctx context.Context, question string, hits []domain.ChunkHit, attachments []domain.Attachment, onDelta func(string)) (Answer, error)

SynthesizeStream is Synthesize with incremental delivery: when the underlying generator implements StreamingGenerator it emits prose via onDelta as it arrives; otherwise it falls back to a single Synthesize and emits the whole text in one onDelta call. Either way it returns the complete Answer (with Grounded set as Synthesize does), so the caller still gets citations for the post-answer sources, --json, and caching.

type Catalog 

type Catalog struct {
	// contains filtered or unexported fields
}

Catalog is the collection-management use case: create, list, and inspect the corpora lore knows about — including the documents ingested into a collection.

func NewCatalog 

func NewCatalog(collections CollectionRepository, docs DocumentRepository, embedder Embedder, chunkers domain.Registry) *Catalog

NewCatalog wires a Catalog over the collection and document repositories, the embedder whose space new collections are pinned to, and the chunker registry whose spec they are pinned to.

func (*Catalog) ChunksByIDs 

func (c *Catalog) ChunksByIDs(ctx context.Context, collection string, ids []string) ([]domain.Chunk, error)

ChunksByIDs returns the collection's chunks with the given IDs, in input order, for inspecting specific or cited chunks. IDs not present are omitted (the caller diffs requested vs returned). It fails with ErrNotFound if the collection does not exist, so an unknown collection is distinguished from merely-absent chunk IDs.

func (*Catalog) DocumentChunks 

func (c *Catalog) DocumentChunks(ctx context.Context, collection, sourceURI string) ([]domain.Chunk, error)

DocumentChunks returns the stored chunks of one document (by source URI) in seq order — the extracted, chunked text as it was actually indexed, for inspecting what the extractor and chunker produced. It fails with ErrNotFound if no document with that source URI exists in the collection.

func (*Catalog) Get 

func (c *Catalog) Get(ctx context.Context, name string) (*domain.Collection, error)

Get returns the named collection, or ErrNotFound.

func (*Catalog) Init 

func (c *Catalog) Init(ctx context.Context, name string) (*domain.Collection, error)

Init creates a collection pinned to the embedder's current EmbeddingSpace and the active chunker spec (re-ingest under a different chunker is then refused). It fails with ErrAlreadyExists if the name is taken and ErrInvalidArgument if the name is invalid.

func (*Catalog) List 

func (c *Catalog) List(ctx context.Context) ([]*domain.Collection, error)

List returns every collection.

func (*Catalog) ListDocuments 

func (c *Catalog) ListDocuments(ctx context.Context, collection string) ([]*domain.Document, error)

ListDocuments returns the documents ingested into the named collection. It fails with ErrNotFound if the collection does not exist, so callers can distinguish an empty collection from a missing one.

type CollectionRepository 

type CollectionRepository interface {
	// Create fails with ErrAlreadyExists if the name is taken.
	Create(ctx context.Context, c *domain.Collection) error
	// Get fails with ErrNotFound if no such collection exists.
	Get(ctx context.Context, name string) (*domain.Collection, error)
	List(ctx context.Context) ([]*domain.Collection, error)
	// Delete removes the collection record, failing with ErrNotFound if no
	// such collection exists. Cascading removal of its documents and vectors
	// is orchestrated by the use case, which holds the
	// DocumentRepository and VectorIndex; this port cannot reach them.
	Delete(ctx context.Context, name string) error
	// RecordSource adds source to the collection's remembered Sources,
	// idempotently (recording an existing source is a no-op). It fails with
	// ErrNotFound if no such collection exists. Get reflects recorded sources.
	RecordSource(ctx context.Context, name, source string) error
}

CollectionRepository persists Collection aggregates.

type DocumentRepository 

type DocumentRepository interface {
	// Upsert stores the document and replaces its chunks atomically.
	Upsert(ctx context.Context, doc *domain.Document, chunks []domain.Chunk) error
	// GetBySource fails with ErrNotFound if the source was never ingested.
	GetBySource(ctx context.Context, collection, sourceURI string) (*domain.Document, error)
	// GetChunks hydrates chunks by ID, preserving input order. IDs with no
	// stored chunk are skipped, so the result may be shorter than the input.
	GetChunks(ctx context.Context, ids []domain.ChunkID) ([]domain.Chunk, error)
	// GetChunksByDocument returns all chunks of one document in seq order. An
	// unknown document (or collection) yields no chunks and no error; existence
	// is the caller's concern (resolve the document via GetBySource first).
	GetChunksByDocument(ctx context.Context, collection string, id domain.DocumentID) ([]domain.Chunk, error)
	// GetChunksByIDs returns the chunks with the given IDs that belong to the
	// collection, in input order. IDs with no stored chunk in the collection are
	// omitted (the caller diffs requested vs returned). An unknown collection
	// yields no chunks and no error; collection existence is the caller's concern.
	GetChunksByIDs(ctx context.Context, collection string, ids []string) ([]domain.Chunk, error)
	// GetDocuments hydrates documents by ID, preserving input order. IDs with no
	// stored document are skipped, so the result may be shorter than the input.
	// Used to attach source provenance to retrieval results.
	GetDocuments(ctx context.Context, ids []domain.DocumentID) ([]*domain.Document, error)
	// ListDocuments returns every document in the collection. An unknown or empty
	// collection yields no documents and no error; collection existence is the
	// CollectionRepository's concern. Order is unspecified.
	ListDocuments(ctx context.Context, collection string) ([]*domain.Document, error)
	// Delete removes the document and its chunks, returning the removed chunk
	// IDs so the use case can delete their vectors via the VectorIndex
	// (this port cannot reach it). Fails with ErrNotFound if no
	// such document exists in the collection.
	Delete(ctx context.Context, collection string, id domain.DocumentID) ([]domain.ChunkID, error)
	// DeleteChunks removes the chunks with the given IDs that belong to the
	// collection, returning the IDs actually removed so the use case can delete
	// their vectors. The owning documents are left in place — a
	// document keeps its record even after losing chunks (sub-document
	// redaction, not document deletion). IDs absent from the collection
	// (unknown, or owned by another collection) are skipped, so the caller can
	// diff requested vs removed; an unknown collection removes nothing. No error
	// for skips — a malformed-ID guard is the caller's concern.
	DeleteChunks(ctx context.Context, collection string, ids []domain.ChunkID) ([]domain.ChunkID, error)
	// DeleteCollection removes every document and its chunks in the collection,
	// returning all removed chunk IDs for the same cascade. A collection with no
	// documents is a no-op (no IDs, no error); collection existence is the
	// CollectionRepository's concern.
	DeleteCollection(ctx context.Context, collection string) ([]domain.ChunkID, error)
}

DocumentRepository persists Documents and their Chunks.

type Embedder 

type Embedder interface {
	Space(ctx context.Context) (domain.EmbeddingSpace, error)
	// Embed returns one vector per input text, in input order.
	Embed(ctx context.Context, texts []string) ([][]float32, error)
}

Embedder turns texts into vectors and reports the space it produces, so use cases can enforce space coherence against the target collection.

type Exporter 

type Exporter struct {
	// contains filtered or unexported fields
}

Exporter serializes a collection into a portable artifact — its embedding-space and chunker pins, metadata, documents, chunks, and vectors — gathered through the persistence ports so the format stays independent of the storage backend.

func NewExporter 

func NewExporter(collections CollectionRepository, docs DocumentRepository, index VectorIndex) *Exporter

NewExporter wires an Exporter over the three persistence ports.

func (*Exporter) Export 

func (e *Exporter) Export(ctx context.Context, collection string, w io.Writer) (TransferSummary, error)

Export writes the named collection to w as a self-contained, versioned artifact. The embedding-space pin (and chunker pin, if the collection carries one) travel inside it so the reconstructed collection enforces the same invariants. An unknown collection is ErrNotFound. The bytes written are the plaintext artifact; encryption, if any, wraps this stream in the caller.

type Extractor 

type Extractor interface {
	Supports(contentType string) bool
	Extract(contentType string, raw []byte) (string, error)
}

Extractor converts raw content of a supported type into plain text.

type FromQuery 

type FromQuery struct {
	From domain.ChunkHit   // the source chunk's identity + source URI (Score unused)
	Hits []domain.ChunkHit // its target hits, best first
}

FromQuery is one source-collection chunk used directly as a query vector and the target-collection hits it retrieved. It backs `query --from-collection`, which feeds a collection's own stored vectors back as queries (never re-embedding) to find each chunk's nearest neighbors in another collection.

type Generator 

type Generator interface {
	Synthesize(ctx context.Context, question string, hits []domain.ChunkHit, attachments []domain.Attachment) (Answer, error)
}

Generator synthesizes an answer grounded in retrieved chunks, optionally with raw attachments (images/documents) the model reads directly. Attachments are ephemeral context, not part of the collection.

type Importer 

type Importer struct {
	// contains filtered or unexported fields
}

Importer reconstructs a collection from a portable artifact, writing it into the local store through the persistence ports. The chunk and document IDs are re-derived from the (target collection, source URI, seq) rather than carried, so importing under a new name produces a coherent collection.

func NewImporter 

func NewImporter(collections CollectionRepository, docs DocumentRepository, index VectorIndex, remover *Remover) *Importer

NewImporter wires an Importer; the Remover handles the cascade when --force replaces an existing collection.

func (*Importer) Import 

func (im *Importer) Import(ctx context.Context, r io.Reader, name string, force bool) (TransferSummary, error)

Import reconstructs the collection in r (a plaintext artifact; any decryption has already happened upstream). A non-empty name renames it. A target name that already exists is refused with ErrAlreadyExists unless force, which replaces it via the cascade. A malformed or too-new artifact surfaces the artifact package's errors before anything is written.

type IngestOption 

type IngestOption func(*Ingestor)

IngestOption configures an Ingestor at construction.

func WithConcurrency 

func WithConcurrency(n int) IngestOption

WithConcurrency sets the bounded parallelism for ingestion. Values <= 0 are ignored, leaving DefaultIngestConcurrency in place. Lower it for providers with tight rate limits to avoid retry thrash.

type IngestSummary 

type IngestSummary struct {
	Added       int // documents ingested (new or changed)
	Skipped     int // unchanged or empty documents (idempotent no-ops)
	Unsupported int // documents whose content type no extractor handles
	Chunks      int // chunks embedded and stored
}

IngestSummary reports the outcome of an ingestion run.

type Ingestor 

type Ingestor struct {
	// contains filtered or unexported fields
}

Ingestor walks a Source, extracts and chunks each document, embeds the chunks, and stores chunks and their vectors. Ingestion is idempotent: a source whose extracted content is unchanged is a no-op.

func NewIngestor 

func NewIngestor(collections CollectionRepository, docs DocumentRepository, index VectorIndex, embedder Embedder, extractor Extractor, source Source, chunkers domain.Registry, opts ...IngestOption) *Ingestor

NewIngestor wires an Ingestor from the ports and domain services it needs. The chunker Registry selects a per-format chunking strategy by content type.

func (*Ingestor) Ingest 

func (i *Ingestor) Ingest(ctx context.Context, collection, root string) (IngestSummary, error)

Ingest processes every document the Source yields under root into the named collection, concurrently with bounded parallelism. It enforces space coherence up front and fails fast on the first error; because ingestion is idempotent, re-running resumes safely over already-stored documents.

func (*Ingestor) IngestContent 

func (i *Ingestor) IngestContent(ctx context.Context, collection, uri, contentType string, content []byte) (IngestSummary, error)

IngestContent ingests a single in-memory document (e.g. read from stdin) into the collection, identified by uri with the given content type. Like Ingest it enforces space coherence and is idempotent by content hash. Unlike Ingest it records no sync source — there is no path to replay — and an unsupported content type is reported, not an error.

type Querier 

type Querier struct {
	// contains filtered or unexported fields
}

Querier retrieves the chunks most similar to a query within a Collection.

func NewQuerier 

func NewQuerier(collections CollectionRepository, index VectorIndex, docs DocumentRepository, embedder Embedder) *Querier

NewQuerier wires a Querier from the ports it needs.

func (*Querier) Explain 

func (q *Querier) Explain(ctx context.Context, collection, query string, k int, source string) (Retrieval, error)

Explain runs the same retrieval as Query but also reports the best candidate just outside the returned top-k (the runner-up) for --explain diagnostics. It fetches one extra candidate (k+1) to surface that runner-up — no extra model call beyond the single query embed. Same errors as Query.

func (*Querier) ExplainAcross 

func (q *Querier) ExplainAcross(ctx context.Context, collections []string, query string, k int, source string) (Retrieval, error)

ExplainAcross is QueryAcross with the runner-up just outside the merged top-k, for --explain over a multi-collection query.

func (*Querier) Query 

func (q *Querier) Query(ctx context.Context, collection, query string, k int, source string) ([]domain.ChunkHit, error)

Query embeds the question and returns up to k ChunkHits from the collection, best match first. It enforces space coherence: the embedder must produce vectors in the collection's space, or it fails with ErrSpaceMismatch before touching the index.

A non-empty source restricts hits to documents whose source URI matches that glob (matched against the basename, or the path when the pattern contains a "/"); Query over-fetches to fill k, approximately (see sourceOverfetch).

An empty query is ErrInvalidArgument; an unknown collection is ErrNotFound. No matching chunks yields no hits and no error.

func (*Querier) QueryAcross 

func (q *Querier) QueryAcross(ctx context.Context, collections []string, query string, k int, source string) ([]domain.ChunkHit, error)

QueryAcross merges retrieval over several collections into one ranked top-k. All collections must share an embedding space (domain.SameSpace) — vectors are only comparable within a space — so the query is embedded once, each collection is searched, and the hits are merged by score. Each returned hit carries its origin Collection. Same per-collection source/over-fetch semantics as Query. Mismatched spaces fail with ErrSpaceMismatch before any search.

func (*Querier) QueryFrom 

func (q *Querier) QueryFrom(ctx context.Context, target, sourceCollection string, k int, source string) ([]FromQuery, error)

QueryFrom searches the target collection using the source collection's own stored chunk vectors as queries — one search per source chunk, grouped by source chunk. Nothing is re-embedded: source vectors are read straight from the index, so the comparison is sound only if source and target share an embedding space, which is enforced (domain.SameSpace → ErrSpaceMismatch) before any index access. A non-empty source glob restricts the *target* hits. An unknown source or target collection is ErrNotFound; an empty source collection yields no groups.

type RankResult 

type RankResult struct {
	Index int
	Score float64
}

RankResult is one scored document from a RerankProvider: Index points into the documents slice the provider was given; Score is its relevance (higher = more relevant).

type Remover 

type Remover struct {
	// contains filtered or unexported fields
}

Remover owns the cross-port deletion cascade: a document's chunks live in the DocumentRepository, their vectors in the VectorIndex, and the collection record in the CollectionRepository. No single port can reach the others, so this use case orchestrates them.

func NewRemover 

func NewRemover(collections CollectionRepository, docs DocumentRepository, index VectorIndex) *Remover

NewRemover wires a Remover over the three persistence ports.

func (*Remover) RemoveChunks 

func (r *Remover) RemoveChunks(ctx context.Context, collection string, ids []domain.ChunkID) ([]domain.ChunkID, error)

RemoveChunks deletes specific chunks by ID and their vectors, returning the IDs actually removed. This is sub-document redaction: the chunks' documents stay in place. IDs absent from the collection are skipped, so the caller can diff requested vs removed and report the misses. Fails with ErrNotFound if no such collection exists.

func (*Remover) RemoveCollection 

func (r *Remover) RemoveCollection(ctx context.Context, name string) error

RemoveCollection deletes the collection and everything in it: its documents, their chunks, and their vectors. Fails with ErrNotFound if no such collection exists.

func (*Remover) RemoveDocument 

func (r *Remover) RemoveDocument(ctx context.Context, collection, sourceURI string) error

RemoveDocument deletes one document, its chunks, and their vectors. Fails with ErrNotFound if the document was never ingested into the collection.

type RerankProvider 

type RerankProvider interface {
	// Rerank scores documents against query and returns results best-first. topN
	// is a hint passed to providers that support it; the Reranker use case is the
	// source of truth for final ordering and truncation.
	Rerank(ctx context.Context, query string, documents []string, topN int) ([]RankResult, error)
}

RerankProvider scores (query, document) pairs jointly with a cross-encoder — the rerank half of two-stage retrieval. It is a distinct port from Embedder/ Generator because rerank APIs are their own thing (Cohere-style /rerank), not the OpenAI chat/embeddings shape, and are usually a separate provider.

type Reranker 

type Reranker struct {
	// contains filtered or unexported fields
}

Reranker is the two-stage-retrieval use case: it reranks already-retrieved hits against the query via a RerankProvider, reordering them by relevance and attaching each hit's rerank score (the original similarity score is kept).

func NewReranker 

func NewReranker(provider RerankProvider) *Reranker

NewReranker wires a Reranker over a RerankProvider.

func (*Reranker) Rerank 

func (r *Reranker) Rerank(ctx context.Context, query string, hits []domain.ChunkHit, topN int) ([]domain.ChunkHit, error)

Rerank reorders hits by cross-encoder relevance to query and returns them with RerankScore attached, best first. topN > 0 truncates to that many after reranking; topN <= 0 re-emits all hits reordered. Empty hits is a no-op (the provider is not called). The use case owns ordering and truncation, so the result is identical regardless of whether the provider honored topN.

type Retrieval 

type Retrieval struct {
	Hits      []domain.ChunkHit
	NextScore float64
	HasNext   bool
}

Retrieval is what Explain returns: the top-k Hits plus the best candidate just outside them (the runner-up), for --explain diagnostics. HasNext is false when there is no further candidate (fewer than k+1 matched after filtering).

type Source 

type Source interface {
	Walk(ctx context.Context, root string, fn func(SourceItem) error) error
}

Source yields raw documents from somewhere (filesystem walk, stdin, URL). Walk stops and returns the first error from fn, or ctx.Err on cancellation.

type SourceItem 

type SourceItem struct {
	URI         string
	ContentType string
	Fingerprint string
	Open        func() ([]byte, error)
}

SourceItem is one raw document yielded by a Source. Content is read lazily via Open so a consumer can decide — from the cheap Fingerprint — not to read at all. Fingerprint is a source-side signature (size + sampled-content hash) that changes when the file changes; an empty Fingerprint disables fast-skip.

type StreamingGenerator 

type StreamingGenerator interface {
	SynthesizeStream(ctx context.Context, question string, hits []domain.ChunkHit, attachments []domain.Attachment, onDelta func(string)) (Answer, error)
}

StreamingGenerator is an optional capability a Generator may implement: it emits the answer's prose incrementally via onDelta (called in order, once per token or chunk) while still returning the complete Answer — the full text and citations a caller needs for the post-answer sources, --json, and caching. A generator that cannot stream a given request (structured-output mode, or a cache hit) may call onDelta once with the whole text. Callers detect support with a type assertion; the CLI uses it only for interactive streaming.

type SyncSummary 

type SyncSummary struct {
	Added       int      // documents ingested (new or changed)
	Skipped     int      // unchanged or empty documents (idempotent no-ops)
	Unsupported int      // documents whose content type no extractor handles
	Chunks      int      // chunks embedded and stored
	Pruned      int      // documents removed (or, in a dry run, that would be)
	PrunedURIs  []string // source URIs of the pruned documents, sorted
}

SyncSummary reports the outcome of a sync run.

type Syncer 

type Syncer struct {
	// contains filtered or unexported fields
}

Syncer brings a collection into line with its source(s). It re-ingests (add/update, idempotently) and, when prune is set, removes documents whose source files no longer exist. With no paths it replays the sources the collection remembers from prior add/sync runs.

func NewSyncer 

func NewSyncer(catalog *Catalog, ingestor *Ingestor, remover *Remover, source Source) *Syncer

NewSyncer wires a Syncer over the use cases and the Source it walks for prune.

func (*Syncer) Sync 

func (s *Syncer) Sync(ctx context.Context, collection string, paths []string, prune, dryRun bool) (SyncSummary, error)

Sync ingests paths into the collection and, when prune is set, removes documents no longer present at the source. With no paths, the collection's remembered sources are replayed; with none recorded it fails with ErrInvalidArgument. An unknown collection is ErrNotFound.

dryRun previews the prune set and changes nothing — no ingestion, no removal; it only applies to --prune (ErrInvalidArgument otherwise). The prune set does not depend on ingesting first: a freshly added document is always present at its source, so it is never in the removal set.

type TokenCounter 

type TokenCounter interface {
	Count(text string) int
}

TokenCounter approximates how many tokens a piece of text occupies in an embedding/generation model's context window. It backs the entire token-awareness story: chunk sizing (the domain chunkers take its Count as an injected func, keeping the tokenizer dependency out of stdlib-only domain) and budget-bounded retrieval. Counts are approximate-but-stable across models, not exact per-model — enough to size windows predictably.

type TransferSummary 

type TransferSummary struct {
	Collection string
	Model      string
	Dimensions int
	Documents  int
	Chunks     int
}

TransferSummary describes a collection that was exported or imported.

type VectorEntry 

type VectorEntry struct {
	ChunkID domain.ChunkID
	Vector  []float32
}

VectorEntry pairs a chunk identity with its vector for indexing.

type VectorIndex 

type VectorIndex interface {
	Upsert(ctx context.Context, collection string, entries []VectorEntry) error
	Search(ctx context.Context, collection string, query []float32, k int) ([]domain.VectorMatch, error)
	// Entries returns every stored (ChunkID, Vector) for the collection, in
	// unspecified order, as copies the caller may retain. An unknown collection
	// yields no entries and no error (mirrors Search). It feeds a collection's
	// own vectors back as queries (query --from-collection) without re-embedding.
	Entries(ctx context.Context, collection string) ([]VectorEntry, error)
	Delete(ctx context.Context, collection string, ids []domain.ChunkID) error
}

VectorIndex stores and searches vectors. It is deliberately dumb: space coherence is enforced by the use cases via Collection.AcceptsSpace before anything reaches the index.

Semantics:

  • Upsert replaces entries with the same ChunkID.
  • Search returns up to k matches, best first (higher score = more similar). Unknown collection or k <= 0 yields no matches, no error.
  • Delete of absent IDs is a no-op.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.