index

package
v0.22.146 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 2, 2026 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package index — embedding-backed semantic-search store for the SemanticSearch MCP tool (ADR-014 T6, design from the 2026-04-26 multi-CLI fan-out).

One in-memory chromem-go collection per repo, persisted to disk so `clawtool serve` boot can reload without re-embedding. The index builder walks the repo, chunks each file, embeds via the configured provider (OpenAI default, Ollama override), and adds each chunk to the collection.

Per ADR-007 we wrap [chromem-go](https://github.com/philippgille/chromem-go) (MIT, pure Go, no CGO) for the vector store and the embedding caller. We never reimplement HNSW / cosine / batching.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Options

type Options struct {
	// Provider picks the embedding backend. "openai" uses
	// text-embedding-3-small via the user's OPENAI_API_KEY; "ollama"
	// uses a local Ollama daemon at OLLAMA_HOST (default
	// http://localhost:11434) with the nomic-embed-text model.
	Provider string

	// Model overrides the per-provider default. Empty = pick from
	// provider's stable default.
	Model string

	// PersistPath, when non-empty, persists the collection to disk so
	// boot reloads skip re-embedding. Default
	// ~/.cache/clawtool/index/<repo-hash>.gob.
	PersistPath string

	// MaxFileBytes caps the size of any one file the indexer reads.
	// Files above the cap are skipped (binary blobs, generated
	// assets). Default 200 KiB — enough for source files, tight for
	// build artefacts.
	MaxFileBytes int64

	// Ignore globs (matched against the path relative to the repo
	// root) skip files. Defaults filter common build / vendor /
	// .git directories.
	Ignore []string
}

Options drive the semantic search pipeline.

type Result

type Result struct {
	Path      string  `json:"path"`
	LineStart int     `json:"line_start"`
	LineEnd   int     `json:"line_end"`
	Snippet   string  `json:"snippet"`
	Score     float64 `json:"score"`
}

Result is one ranked hit returned by Search.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is the single semantic-search index. Methods are safe to call from multiple goroutines after Build returns.

func New

func New(repo string, opts Options) *Store

New creates an empty Store rooted at `repo` with the given options. Build populates it; Search queries it.

func (*Store) Build

func (s *Store) Build(ctx context.Context) error

Build walks the repo and embeds every readable text file. Idempotent when a persisted collection at PersistPath already exists — that path is loaded and Build skips the walk entirely. Operators force a rebuild via `Rebuild`.

func (*Store) Count

func (s *Store) Count() int

Count reports how many chunks the store currently holds.

func (*Store) Search

func (s *Store) Search(ctx context.Context, query string, limit int) ([]Result, error)

Search queries the embedded collection with a natural-language query. Returns up to `limit` results ranked by similarity.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL