store

package
v0.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 17, 2026 License: MIT Imports: 14 Imported by: 0

Documentation

Overview

Package store implements the Store interface decided in https://github.com/cheetahbyte/lore/issues/10: everything above this package (ingestion, retrieval, MCP handlers) talks only to Store, never to a concrete database type, so a future hosted backend can be a second implementation rather than a rewrite.

Index

Constants

View Source
const LocalTenant = "local"

LocalTenant is the constant tenant_id every row and query uses outside a hosted deployment, per issue #10.

Variables

This section is empty.

Functions

This section is empty.

Types

type LibraryInfo

type LibraryInfo struct {
	LibraryID     string
	Versions      []string
	LastIndexedAt time.Time
}

LibraryInfo describes one indexed library, as returned by ListLibraries and ResolveLibraries (the list_libraries and resolve_library MCP tools from issue #2).

type SQLite

type SQLite struct {
	// contains filtered or unexported fields
}

SQLite is the Store implementation decided in issue #7/#10: ncruces/go-sqlite3 (pure Go, no CGO), FTS5 always on.

FTS5 and vec1 cannot both be registered as dynamically-linked WASM extensions on the *same* go-sqlite3 connection (confirmed while implementing this: only the first ext/*.Register call on a connection actually takes effect, the second silently fails to register its virtual table module — an upstream limitation, not a design choice). So vector search, when enabled, lives in a second SQLite file with only vec1 registered, rather than one database with both. Chunk rowids are shared by construction (the vector row is inserted with the same rowid SQLite assigned the chunk row) so the two stay joinable in application code even though they're not literally the same file.

func Open

func Open(ctx context.Context, indexPath, vectorPath string, vectorEnabled bool) (*SQLite, error)

Open opens (creating if needed) the SQLite-backed Store at dataDir. vectorEnabled mirrors whether embeddings are configured, per issue #7 — the vectors.db file and its vec1 registration are skipped entirely when vector search is off, keeping the FTS-only default path dependency-free.

func (*SQLite) Close

func (s *SQLite) Close() error

func (*SQLite) ContentHashes

func (s *SQLite) ContentHashes(ctx context.Context, tenantID, libraryID, version string) (map[string]string, error)

func (*SQLite) DeleteLibrary

func (s *SQLite) DeleteLibrary(ctx context.Context, tenantID, libraryID, version string) error

DeleteLibrary implements Store.DeleteLibrary.

func (*SQLite) LatestVersion

func (s *SQLite) LatestVersion(ctx context.Context, tenantID, libraryID string) (string, error)

LatestVersion implements the default-version rule decided in issue #9: highest semver / registry-latest, falling back to the one version that exists for versionless sources.

func (*SQLite) ListLibraries

func (s *SQLite) ListLibraries(ctx context.Context, tenantID string) ([]LibraryInfo, error)

func (*SQLite) ResolveLibraries

func (s *SQLite) ResolveLibraries(ctx context.Context, tenantID, query string) ([]LibraryInfo, error)

func (*SQLite) SearchChunks

func (s *SQLite) SearchChunks(ctx context.Context, req SearchRequest) ([]ScoredChunk, error)

SearchChunks implements the retrieval pipeline decided in issue #7: FTS5 always runs; when req.Vector is set (vector search is enabled and the caller supplied a query embedding), vec1's nearest-neighbor results are fused with the FTS5 ranking via Reciprocal Rank Fusion.

func (*SQLite) UpsertChunks

func (s *SQLite) UpsertChunks(ctx context.Context, tenantID, libraryID, version string, chunks []chunk.Chunk) error

UpsertChunks implements Store.UpsertChunks: a full re-index of tenantID/libraryID/version — existing chunks for that triple are replaced wholesale rather than diffed row-by-row, matching how #12's refresh flow already decides whether re-indexing is worth doing at all (via ContentHashes) before calling this.

type ScoredChunk

type ScoredChunk struct {
	Chunk chunk.Chunk
	Score float64
}

ScoredChunk is one search_docs result.

type SearchRequest

type SearchRequest struct {
	TenantID  string
	LibraryID string
	Version   string
	Query     string
	TopK      int
	Vector    []float32
}

SearchRequest is one search_docs query (issue #2). Version == "" resolves to the highest semver / latest version per issue #9. Vector == nil means the caller isn't running vector search (either it's disabled per issue #7, or this particular query is FTS-only).

type Store

type Store interface {
	// UpsertChunks replaces all chunks for the given library+version with
	// the ones provided (a full re-index of that library+version).
	UpsertChunks(ctx context.Context, tenantID, libraryID, version string, chunks []chunk.Chunk) error

	// DeleteLibrary removes a library, or one of its versions if version
	// is non-empty. Mirrors `lore remove`, issue #3.
	DeleteLibrary(ctx context.Context, tenantID, libraryID, version string) error

	// ListLibraries returns everything indexed for tenantID. Mirrors the
	// list_libraries MCP tool and `lore list`, issues #2/#3.
	ListLibraries(ctx context.Context, tenantID string) ([]LibraryInfo, error)

	// ResolveLibraries fuzzy-matches query against indexed library ids.
	// Mirrors the resolve_library MCP tool, issue #2.
	ResolveLibraries(ctx context.Context, tenantID, query string) ([]LibraryInfo, error)

	// LatestVersion returns the version SearchChunks resolves to when
	// SearchRequest.Version is "", per issue #9's default-version rule.
	LatestVersion(ctx context.Context, tenantID, libraryID string) (string, error)

	// SearchChunks runs the retrieval pipeline from issue #7: always FTS5,
	// plus vector search fused via RRF when req.Vector is set.
	SearchChunks(ctx context.Context, req SearchRequest) ([]ScoredChunk, error)

	// ContentHash returns the stored content_hash for every chunk
	// currently indexed under libraryID+version, keyed by doc_url, for the
	// freshness check from issue #12 to diff against.
	ContentHashes(ctx context.Context, tenantID, libraryID, version string) (map[string]string, error)

	Close() error
}

Store is the storage interface everything above it depends on. The SQLite implementation in this package is the only implementation for now; a hosted backend would add a second one.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL