store

package
v0.6.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 22, 2026 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package store provides the SQLite-backed implementation of types.GraphStore.

SQLiteStore is the sole persistent storage backend for the knowing knowledge graph. It stores nodes, edges, files, repos, snapshots, and edge events in a single SQLite database using WAL mode for concurrent read access. All graph traversals (transitive callers/callees, blast radius) are implemented as recursive CTEs executed directly in SQLite.

The schema is managed by embedded SQL migrations (see migrate.go). Batch insert methods (BatchPutNodes, BatchPutEdges, BatchPutFiles) wrap multiple inserts in a single transaction for performance during full-repo indexing.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Migrate 

func Migrate(db *sql.DB) error

Migrate applies all pending SQL migrations in order. Each migration runs inside its own transaction and the schema_version table is updated after each successful migration.

Types

type FeedbackStats 

type FeedbackStats struct {
	UsefulCount    int     `json:"useful_count"`
	NotUsefulCount int     `json:"not_useful_count"`
	Score          float64 `json:"score"` // useful / (useful + not_useful)
}

FeedbackStats holds aggregate feedback data for a symbol.

type RouteSymbolRow 

type RouteSymbolRow struct {
	ServiceName  string
	RoutePattern string
	MappingType  string
	NodeHash     types.Hash
	CreatedAt    int64
}

RouteSymbolRow represents a row in the route_symbols table, mapping a service route pattern to a graph node hash. This is a local struct to avoid importing the trace package from the store layer.

type RuntimeStatsRow 

type RuntimeStatsRow struct {
	TotalEdges  int
	ActiveEdges int            // observed in last 7 days
	StaleEdges  int            // not observed in 30+ days
	GCEligible  int            // not observed in 90+ days
	ByEdgeType  map[string]int // counts keyed by edge_type
}

RuntimeStatsRow contains aggregate statistics about runtime-derived edges.

type SQLiteStore 

type SQLiteStore struct {
	// contains filtered or unexported fields
}

SQLiteStore implements types.GraphStore backed by a SQLite database. It uses WAL (Write-Ahead Logging) mode, which allows concurrent readers while a single writer is active. All hash columns store raw 32-byte blobs; the Go layer handles hex encoding/decoding.

An in-process node/edge cache is layered on top of SQLite to eliminate redundant SQL round-trips on hot-path traversals such as blast_radius, which can walk hundreds of edges. The cache is invalidated at the start of each index run via InvalidateCache.

func NewSQLiteStore 

func NewSQLiteStore(dbPath string) (*SQLiteStore, error)

NewSQLiteStore opens (or creates) a SQLite database at dbPath, enables WAL mode, and runs any pending migrations.

func (*SQLiteStore) AllRepos 

func (s *SQLiteStore) AllRepos(ctx context.Context) ([]types.Repo, error)

AllRepos returns all tracked repositories ordered by URL.

func (*SQLiteStore) BatchPutEdges 

func (s *SQLiteStore) BatchPutEdges(ctx context.Context, edges []types.Edge) error

BatchPutEdges inserts multiple edges in a single transaction using multi-row INSERT statements for reduced per-row overhead.

func (*SQLiteStore) BatchPutFiles 

func (s *SQLiteStore) BatchPutFiles(ctx context.Context, files []types.File) error

BatchPutFiles inserts multiple files in a single transaction using multi-row INSERT statements.

func (*SQLiteStore) BatchPutNodes 

func (s *SQLiteStore) BatchPutNodes(ctx context.Context, nodes []types.Node) error

BatchPutNodes inserts multiple nodes in a single transaction.

func (*SQLiteStore) BatchPutNotes added in v0.4.0 

func (s *SQLiteStore) BatchPutNotes(ctx context.Context, notes []types.Note) error

BatchPutNotes upserts multiple notes in a single transaction. Significantly faster than individual PutNote calls for bulk operations like persisting community assignments.

func (*SQLiteStore) BlastRadius 

func (s *SQLiteStore) BlastRadius(ctx context.Context, target types.Hash, snapshot types.Hash) (*types.BlastRadiusResult, error)

BlastRadius computes the blast radius of a target symbol: all functions that transitively call it, grouped by repository. It combines TransitiveCallers with a repo lookup for each caller to produce the grouped result. The traversal is capped at 5 levels deep.

func (*SQLiteStore) Close 

func (s *SQLiteStore) Close() error

Close closes the underlying database connection.

func (*SQLiteStore) CreateSnapshot 

func (s *SQLiteStore) CreateSnapshot(ctx context.Context, snap types.Snapshot) error

CreateSnapshot upserts a snapshot record into the snapshots table.

func (*SQLiteStore) DB added in v0.2.0 

func (s *SQLiteStore) DB() *sql.DB

DB returns the underlying sql.DB for direct access (e.g., task memory).

func (*SQLiteStore) DanglingEdges 

func (s *SQLiteStore) DanglingEdges(ctx context.Context) ([]types.Edge, error)

DanglingEdges returns all edges whose target_hash does not match any existing node. These are cross-repo edges where the target was computed with the wrong repo URL. The resolver uses this to find and retarget them.

func (*SQLiteStore) DeadRoutes 

func (s *SQLiteStore) DeadRoutes(ctx context.Context, staleDays int) ([]RouteSymbolRow, error)

DeadRoutes returns route symbols that have no runtime observations in the last staleDays days (or have never been observed at all).

func (*SQLiteStore) DecayRuntimeConfidence 

func (s *SQLiteStore) DecayRuntimeConfidence(ctx context.Context, staleDays int, newConfidence float64) (int, error)

DecayRuntimeConfidence reduces the confidence of stale runtime-derived edges. An edge is considered stale if it has provenance starting with "otel_", last_observed is older than staleDays ago, and its confidence is higher than newConfidence. Returns the number of rows affected.

func (*SQLiteStore) DeleteEdge 

func (s *SQLiteStore) DeleteEdge(ctx context.Context, hash types.Hash) error

DeleteEdge removes an edge by its hash. Used by the enricher to replace ast_inferred edges with lsp_resolved edges, and by the resolver to retarget dangling edges.

func (*SQLiteStore) DeleteEdgesBySourceFile 

func (s *SQLiteStore) DeleteEdgesBySourceFile(ctx context.Context, fileHash types.Hash) ([]types.Edge, error)

DeleteEdgesBySourceFile removes all edges whose source node belongs to the given file. Returns the deleted edges so the indexer can record "removed" edge events for snapshot diffing.

func (*SQLiteStore) DeleteEdgesNotIn added in v0.3.0 

func (s *SQLiteStore) DeleteEdgesNotIn(ctx context.Context, keep map[types.Hash]struct{}) (int64, error)

DeleteEdgesNotIn deletes all edge rows whose edge_hash is not in the provided keep set. Uses a temporary table for efficient NOT IN checking with large sets. Returns the number of deleted edges.

func (*SQLiteStore) DeleteNodesByFile 

func (s *SQLiteStore) DeleteNodesByFile(ctx context.Context, fileHash types.Hash) (int, error)

DeleteNodesByFile removes all nodes belonging to a file. Returns the count of deleted nodes. Used during incremental re-indexing to clear stale nodes before inserting fresh ones from the updated file.

func (*SQLiteStore) DeleteNodesNotIn added in v0.3.0 

func (s *SQLiteStore) DeleteNodesNotIn(ctx context.Context, keep map[types.Hash]struct{}) (int64, error)

DeleteNodesNotIn deletes all node rows whose node_hash is not in the provided keep set. Uses a temporary table for efficient NOT IN checking with large sets. Returns the number of deleted nodes.

func (*SQLiteStore) DeleteNote added in v0.3.0 

func (s *SQLiteStore) DeleteNote(ctx context.Context, objectHash types.Hash, key string) error

DeleteNote removes a single note by object hash and key.

func (*SQLiteStore) DeleteNotesByObject added in v0.3.0 

func (s *SQLiteStore) DeleteNotesByObject(ctx context.Context, objectHash types.Hash) error

DeleteNotesByObject removes all notes attached to an object.

func (*SQLiteStore) DeleteSnapshot 

func (s *SQLiteStore) DeleteSnapshot(ctx context.Context, hash types.Hash) error

DeleteSnapshot removes a snapshot and its associated edge events.

func (*SQLiteStore) EdgesBySourceFile 

func (s *SQLiteStore) EdgesBySourceFile(ctx context.Context, fileHash types.Hash) ([]types.Edge, error)

EdgesBySourceFile returns all edges whose source node belongs to the given file.

func (*SQLiteStore) EdgesFrom 

func (s *SQLiteStore) EdgesFrom(ctx context.Context, sourceHash types.Hash, edgeType string) ([]types.Edge, error)

EdgesFrom returns all edges originating from the given source node. If edgeType is non-empty, only edges of that type are returned.

func (*SQLiteStore) EdgesTo 

func (s *SQLiteStore) EdgesTo(ctx context.Context, targetHash types.Hash, edgeType string) ([]types.Edge, error)

EdgesTo returns all edges pointing to the given target node. If edgeType is non-empty, only edges of that type are returned.

func (*SQLiteStore) FeedbackBoosts 

func (s *SQLiteStore) FeedbackBoosts(ctx context.Context, hashes []types.Hash, neighborhoodRoots map[types.Hash]types.Hash) (map[types.Hash]float64, error)

FeedbackBoosts returns a map of symbol hash to feedback score (0.0-1.0) for all provided hashes that have at least one feedback entry. Hashes with no feedback are omitted from the result.

neighborhoodRoots maps symbol hash to its current SubgraphRoot. If provided, only feedback entries where neighborhood_root matches are counted (merkleized expiration). When a symbol's package changes, its old feedback expires automatically.

func (*SQLiteStore) FileByPath 

func (s *SQLiteStore) FileByPath(ctx context.Context, repoHash types.Hash, path string) (*types.File, error)

FileByPath looks up a single file by repo hash and relative path. Returns nil if no matching file exists.

func (*SQLiteStore) FilesByRepo 

func (s *SQLiteStore) FilesByRepo(ctx context.Context, repoHash types.Hash) ([]types.File, error)

FilesByRepo returns all files belonging to a repository, ordered by path.

func (*SQLiteStore) GetEdge 

func (s *SQLiteStore) GetEdge(ctx context.Context, hash types.Hash) (*types.Edge, error)

GetEdge retrieves an edge by its content-addressed hash. Returns nil if not found. Results are cached in memory; the cache is bounded to edgeCacheMaxEntries and is invalidated by InvalidateCache at the start of each index run.

func (*SQLiteStore) GetNode 

func (s *SQLiteStore) GetNode(ctx context.Context, hash types.Hash) (*types.Node, error)

GetNode retrieves a node by its content-addressed hash. Returns nil if not found. Results are cached in memory; the cache is bounded to nodeCacheMaxEntries and is invalidated by InvalidateCache at the start of each index run.

func (*SQLiteStore) GetNote added in v0.3.0 

func (s *SQLiteStore) GetNote(ctx context.Context, objectHash types.Hash, key string) (*types.Note, error)

GetNote retrieves a single note by object hash and key. Returns nil if not found.

func (*SQLiteStore) GetNotes added in v0.3.0 

func (s *SQLiteStore) GetNotes(ctx context.Context, objectHash types.Hash) ([]types.Note, error)

GetNotes retrieves all notes attached to an object.

func (*SQLiteStore) GetNotesByKey added in v0.3.0 

func (s *SQLiteStore) GetNotesByKey(ctx context.Context, key string) ([]types.Note, error)

GetNotesByKey retrieves all notes with the given key across all objects.

func (*SQLiteStore) GetRepo 

func (s *SQLiteStore) GetRepo(ctx context.Context, hash types.Hash) (*types.Repo, error)

GetRepo retrieves a repo by its hash (sha256 of repo URL). Returns nil if not found.

func (*SQLiteStore) GetRouteSymbol 

func (s *SQLiteStore) GetRouteSymbol(ctx context.Context, serviceName, routePattern, mappingType string) (*RouteSymbolRow, error)

GetRouteSymbol retrieves a route symbol mapping by its composite key. Returns (nil, nil) if no matching row exists.

func (*SQLiteStore) GetSnapshot 

func (s *SQLiteStore) GetSnapshot(ctx context.Context, hash types.Hash) (*types.Snapshot, error)

GetSnapshot retrieves a snapshot by its Merkle root hash. Returns nil if not found.

func (*SQLiteStore) IntegrityCheck added in v0.3.0 

func (s *SQLiteStore) IntegrityCheck(ctx context.Context) error

IntegrityCheck runs PRAGMA integrity_check on the SQLite database. Returns nil if the database passes all checks, or an error describing the first corruption issues found.

func (*SQLiteStore) InvalidateCache added in v0.3.0 

func (s *SQLiteStore) InvalidateCache()

InvalidateCache clears the in-process node and edge caches. Call this at the start of each index run so that freshly written rows are not shadowed by stale cached values.

func (*SQLiteStore) LatestSnapshot 

func (s *SQLiteStore) LatestSnapshot(ctx context.Context, repoHash types.Hash) (*types.Snapshot, error)

LatestSnapshot returns the most recent snapshot for a repository, ordered by timestamp descending. Returns nil if no snapshots exist for the repo.

func (*SQLiteStore) NodesByFilePath 

func (s *SQLiteStore) NodesByFilePath(ctx context.Context, repoHash types.Hash, path string) ([]types.Node, error)

NodesByFilePath returns all nodes belonging to a file identified by repo hash and relative path. It joins through the files table using the path, so it works regardless of whether file content (and thus file_hash) has changed.

func (*SQLiteStore) NodesByName 

func (s *SQLiteStore) NodesByName(ctx context.Context, qualifiedPrefix string) ([]types.Node, error)

NodesByName returns all nodes whose qualified name starts with the given prefix. Used by the indexer to find all nodes for a repo (prefix = repoURL) and by the query CLI to search by symbol name.

func (*SQLiteStore) NodesByQualifiedName 

func (s *SQLiteStore) NodesByQualifiedName(ctx context.Context, qualifiedName string) ([]types.Node, error)

NodesByQualifiedName returns all nodes with an exact qualified name match.

func (*SQLiteStore) PutEdge 

func (s *SQLiteStore) PutEdge(ctx context.Context, e types.Edge) error

PutEdge upserts a single edge into the edges table.

func (*SQLiteStore) PutFile 

func (s *SQLiteStore) PutFile(ctx context.Context, f types.File) error

PutFile upserts a single file record into the files table.

func (*SQLiteStore) PutNode 

func (s *SQLiteStore) PutNode(ctx context.Context, n types.Node) error

PutNode upserts a single node into the nodes table.

func (*SQLiteStore) PutNote added in v0.3.0 

func (s *SQLiteStore) PutNote(ctx context.Context, n types.Note) error

PutNote upserts a note (object_hash + key is the composite key).

func (*SQLiteStore) PutRepo 

func (s *SQLiteStore) PutRepo(ctx context.Context, r types.Repo) error

PutRepo upserts a repo record into the repos table.

func (*SQLiteStore) PutRouteSymbol 

func (s *SQLiteStore) PutRouteSymbol(ctx context.Context, serviceName, routePattern string, nodeHash types.Hash, mappingType string) error

PutRouteSymbol upserts a route symbol mapping into the route_symbols table.

func (*SQLiteStore) QueryFeedback 

func (s *SQLiteStore) QueryFeedback(ctx context.Context, symbolHash types.Hash) (*FeedbackStats, error)

QueryFeedback returns aggregate feedback stats for a symbol. Returns zero stats (not nil) if no feedback exists.

func (*SQLiteStore) RebuildFTS added in v0.2.0 

func (s *SQLiteStore) RebuildFTS(ctx context.Context) error

RebuildFTS rebuilds the entire FTS index from the nodes and files tables. Splits CamelCase and snake_case identifiers into individual tokens so that searching for "ingest" matches "TraceIngestor". Call after batch indexing operations for best performance (avoids per-node rebuilds).

Optimization: pre-computes all splitForFTS strings in parallel (CPU-bound), then does a single batch INSERT (I/O-bound, sequential). The split computation is the expensive part for large repos (100K+ nodes).

func (*SQLiteStore) RebuildFTSForPackages added in v0.4.0 

func (s *SQLiteStore) RebuildFTSForPackages(ctx context.Context, packages []string) error

RebuildFTSForPackages deletes and re-inserts FTS rows only for nodes whose qualified name starts with one of the given package prefixes. Falls back to full RebuildFTS if packages is empty. This makes FTS rebuild proportional to the number of changed packages, not the total graph size.

func (*SQLiteStore) RecordEdgeEvent 

func (s *SQLiteStore) RecordEdgeEvent(ctx context.Context, ev types.EdgeEvent) error

RecordEdgeEvent appends an edge mutation event (added/removed) to the edge_events table. These events are append-only and power snapshot diffing.

func (*SQLiteStore) RecordFeedback 

func (s *SQLiteStore) RecordFeedback(ctx context.Context, symbolHash types.Hash, sessionID string, useful bool, neighborhoodRoot types.Hash) error

RecordFeedback inserts a feedback record for a symbol in a session. If neighborhoodRoot is not EmptyHash, it is stored to enable merkleized expiration: feedback becomes invalid when the symbol's package changes (detected via SubgraphRoot mismatch).

func (*SQLiteStore) RuntimeEdgeStatsAggregate 

func (s *SQLiteStore) RuntimeEdgeStatsAggregate(ctx context.Context) (*RuntimeStatsRow, error)

RuntimeEdgeStatsAggregate returns aggregate statistics about runtime-derived edges (those with provenance starting with "otel_").

func (*SQLiteStore) RuntimeEdgesByProvenance 

func (s *SQLiteStore) RuntimeEdgesByProvenance(ctx context.Context, provenancePrefix string) ([]types.Edge, error)

RuntimeEdgesByProvenance returns all edges whose provenance starts with the given prefix. The returned edges include the observation_count and last_observed columns populated on the Edge struct.

func (*SQLiteStore) RuntimeEdgesByService 

func (s *SQLiteStore) RuntimeEdgesByService(ctx context.Context, serviceName string, routePattern string, limit int) ([]types.Edge, error)

RuntimeEdgesByService returns runtime edges filtered by service name and optional route pattern. Only edges with provenance starting with "otel_" are returned. If serviceName is empty, all runtime edges are returned (up to limit). If routePattern is non-empty, it is used as a LIKE filter on the route_symbols.route_pattern column.

func (*SQLiteStore) SearchBM25Nodes added in v0.2.0 

func (s *SQLiteStore) SearchBM25Nodes(ctx context.Context, query string, limit int) ([]types.Node, error)

SearchBM25Nodes performs full-text search over the nodes_fts index using BM25 ranking. The query string uses FTS5 query syntax (terms joined by OR/AND). Returns up to limit nodes ordered by BM25 relevance (best matches first).

func (*SQLiteStore) SnapshotDiff 

func (s *SQLiteStore) SnapshotDiff(ctx context.Context, oldRoot, newRoot types.Hash) (*types.DiffResult, error)

SnapshotDiff computes the structural diff between two snapshots by querying edge_events recorded during the newer snapshot's index run. Added edges are events with type "added" in the new snapshot; removed edges are events with type "removed".

func (*SQLiteStore) StaleEdges 

func (s *SQLiteStore) StaleEdges(ctx context.Context, snapshot types.Hash) ([]types.Edge, error)

StaleEdges finds edges whose source file has been updated since the edge was created. An edge is stale when its source node's file_hash points to a File record whose content_hash no longer matches the latest file at that repo+path. This indicates the source file has changed and the edge may no longer be valid.

Implementation: joins edges -> nodes -> files, then uses an EXISTS subquery to find any other file at the same (repo, path) with a different content hash. If such a file exists, the edge is stale.

func (*SQLiteStore) TransitiveCallees 

func (s *SQLiteStore) TransitiveCallees(ctx context.Context, source types.Hash, maxDepth int, snapshot types.Hash) ([]types.CalleeResult, error)

TransitiveCallees finds all nodes that are transitively called by the source node, up to maxDepth hops. This is the forward traversal counterpart to TransitiveCallers.

Implementation: a recursive CTE walks "calls" edges forward from the source. The base case selects all direct callees (depth=1), and the recursive step follows outgoing call edges from each callee.

func (*SQLiteStore) TransitiveCallers 

func (s *SQLiteStore) TransitiveCallers(ctx context.Context, target types.Hash, maxDepth int, snapshot types.Hash) ([]types.CallerResult, error)

TransitiveCallers finds all nodes that transitively call the target node, up to maxDepth hops. The snapshot parameter is accepted for API compatibility but is not currently used for filtering.

Implementation: a recursive CTE walks the "calls" edges backwards from the target. The base case selects all direct callers (depth=1), and the recursive step joins each caller against edges pointing to it, incrementing depth. UNION (not UNION ALL) deduplicates cycles. Results are joined back to the nodes table for full node data and ordered by depth then qualified name.

func (*SQLiteStore) TruncateGraph 

func (s *SQLiteStore) TruncateGraph(ctx context.Context) error

TruncateGraph deletes all nodes, edges, and edge events from the database. This is used by the reindex command to clear stale data before re-indexing.

func (*SQLiteStore) UpdateNodeBlame added in v0.3.0 

func (s *SQLiteStore) UpdateNodeBlame(ctx context.Context, nodeHash types.Hash, author string, commitAt int64) error

UpdateNodeBlame stamps git blame metadata on a node without replacing it.

func (*SQLiteStore) UpdateNodeCoverage added in v0.3.0 

func (s *SQLiteStore) UpdateNodeCoverage(ctx context.Context, nodeHash types.Hash, pct float64) error

UpdateNodeCoverage stamps test coverage percentage on a node.

func (*SQLiteStore) UpdateObservation 

func (s *SQLiteStore) UpdateObservation(ctx context.Context, edgeHash types.Hash, count int, lastObserved int64, confidence float64) error

UpdateObservation updates the runtime observation fields on an edge.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.