store

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 11, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Index

Constants

View Source 
const MetadataDBFile = "metadata.db"

MetadataDBFile is the filename of the SQLite database inside a CodeDB directory.

Variables

View Source 
var BleveSubIndexNames = []string{"code", "diff", "comment"}

BleveSubIndexNames lists the bleve sub-indexes managed by Store, in the same order they are opened. Used by self-heal callers (daemon, doctor) so they don't have to hardcode the names.

View Source 
var ErrCorrupt = fmt.Errorf("codedb index is corrupt")

ErrCorrupt indicates the index is corrupted and needs re-indexing.

View Source 
var ErrFullReindexRequired = errors.New("full reindex required")

ErrFullReindexRequired is returned by RebuildBleveSubIndex when the requested sub-index (code/diff) cannot be repopulated from existing SQL data alone. Callers that hit this should fall back to a full reindex (wipe dataDir + run IndexLocalRepo) — the rebuild path is currently only safe for "comment" because ParseComments is gated on a per-blob SQL flag we can reset, while code/diff are populated only during the per-commit walk in IndexRepo and have no rebleve-from-blobs path yet.

Functions

func ClearAllNeedsReindexMarkers added in v0.9.0 

func ClearAllNeedsReindexMarkers(root string) error

ClearAllNeedsReindexMarkers removes every marker in root. Called by the indexer after a successful full pass. Aggregates per-marker failures into a joined error so callers can surface the problem; a swallowed failure here would leave the daemon re-forcing --full on every freshness pass.

func ClearNeedsReindexMarker added in v0.9.0 

func ClearNeedsReindexMarker(root, name string) error

ClearNeedsReindexMarker removes the self-heal marker for a sub-index. Safe to call when the marker is absent. Returns nil on success or ENOENT.

func CreateSchema 

func CreateSchema(db *sql.DB) error

CreateSchema initializes the SQLite tables and indexes, and runs all pending migrations. "Migrations" here include both schema migrations (idempotent ALTER/CREATE) and one-shot data repairs that recover from historical bugs (see migrateInvalidateGitHubMtimesForIssue474). New migrations should be appended at the end and gated by their own idempotency check (column existence, sentinel row, etc.).

func HasNeedsReindexMarker added in v0.9.0 

func HasNeedsReindexMarker(root, name string) bool

HasNeedsReindexMarker reports whether a self-heal marker exists for the named sub-index. Used by the daemon (to force --full on next pass) and by status/doctor commands (to surface "rebuilding" state to the user).

func NeedsReindexMarkerPath added in v0.9.0 

func NeedsReindexMarkerPath(root, name string) string

NeedsReindexMarkerPath returns the on-disk marker path for a bleve sub-index. Exported so the daemon and doctor can locate markers without hardcoding the prefix.

func NeedsReindexMarkers added in v0.9.0 

func NeedsReindexMarkers(root string) []string

NeedsReindexMarkers returns the names of all sub-indexes with self-heal markers present. Returns an empty (non-nil) slice when none are set so callers only have one empty-state to handle (`len(out) == 0`).

func QueriesFromTx added in v0.6.0 

func QueriesFromTx(tx *sql.Tx) *codedbsqlc.Queries

QueriesFromTx returns sqlc-generated typed queries bound to a transaction.

func RebuildBleveSubIndex added in v0.8.0 

func RebuildBleveSubIndex(root, name string) error

RebuildBleveSubIndex performs a targeted rebuild of a single bleve sub-index. Currently supported only for "comment", which can be fully repopulated from existing SQL data via the comments_parsed flag.

For "code" and "diff", this function returns ErrFullReindexRequired without modifying state — those sub-indexes are populated during the per-commit walk in IndexRepo, gated on `commits` SQL rows, and there is no rebleve-from-blobs path that could refill them from SQL alone. A surgical rebuild would leave search permanently empty; callers must fall back to a full reindex (wipe dataDir + IndexLocalRepo).

On success for "comment": removes bleve/comment/, recreates empty, and resets blobs.comments_parsed=0 so ParseComments re-extracts every blob on the next indexing pass. SQL/Open failures during the flag reset are surfaced as errors — a rebuild that "succeeds" with comments_parsed still set would silently leave search empty forever.

This function works without an open Store — by design, since Open fails when the sub-index is in the corrupt state we are recovering from.

func WriteNeedsReindexMarker added in v0.9.0 

func WriteNeedsReindexMarker(root, name string) error

WriteNeedsReindexMarker creates the marker file that signals to the daemon's next indexing pass that a sub-index was nuked and needs a full rebuild. The file body is a short human-readable string with the trigger time, so `ox doctor` and `cat .needs_reindex_*` can give a meaningful answer to "why is this here?". Exported so the daemon can restore markers after a failed marker-forced reindex (without restore, a single rebuild failure would silently drop the signal and code search would stay empty until `ox code index --full`).

Types

type MaintenanceResult added in v0.6.0 

type MaintenanceResult struct {
	OrphanBlobsPruned int64
	OldDiffsPruned    int64
	StaleSymbolsCount int64
	Vacuumed          bool
	IntegrityOK       bool
	SizeBefore        int64
	SizeAfter         int64
	Duration          time.Duration
}

MaintenanceResult captures what happened during codedb maintenance.

func (MaintenanceResult) TotalPruned added in v0.6.0 

func (r MaintenanceResult) TotalPruned() int64

TotalPruned returns the total number of rows removed.

type MappingCorruptError added in v0.8.0 

type MappingCorruptError struct {
	Name string
	Path string
}

MappingCorruptError indicates that a Bleve sub-index is in a structurally broken state that bleve.Open cannot recover from on its own. The Name identifies which sub-index ("code", "diff", or "comment") so callers can perform a targeted rebuild via RebuildBleveSubIndex without nuking the whole dataDir.

Detected conditions (see isBleveIndexCorrupt):

  • persisted `_mapping` doc is empty/missing in the latest snapshot, or
  • the latest snapshot references segment IDs whose `.zap` files are missing on disk (the field-observed poison pill: bolt + mapping intact but a previous incomplete write left the snapshot pointing at segments that never landed)

Distinct from "real lock contention" (another goroutine/process actively writing): we only return this after a successful read-only bbolt open with 100ms timeout — a held exclusive lock blocks the read and we stay in the safe lock-contention path.

func (*MappingCorruptError) Error added in v0.8.0 

func (e *MappingCorruptError) Error() string

type Store 

type Store struct {
	CodeIndex    bleve.Index
	DiffIndex    bleve.Index
	CommentIndex bleve.Index
	Root         string

	CombinedCodeIndex bleve.Index // alias of CodeIndex + all dirty indexes, or just CodeIndex
	// contains filtered or unexported fields
}

Store wraps a SQLite database and Bleve full-text search indexes. All SQL access goes through the convenience methods below.

The store supports a two-tier architecture:

  • Shared indexes (on-disk): committed content, shared across worktrees
  • Dirty overlay (on-disk or in-memory): uncommitted worktree files, per-worktree

When a dirty overlay is attached, CombinedCodeIndex transparently merges results from both tiers via Bleve IndexAlias.

func Open 

func Open(root string) (*Store, error)

Open opens (or creates) a Store at the given root directory. It creates the directory structure, initializes SQLite and Bleve indexes. If SQLite corruption is detected, the database is removed and ErrCorrupt is returned so the caller can trigger a full re-index.

Bleve self-heal: when a sub-index has a structurally broken `_mapping` doc (kill-9 mid-flush, partial scorch snapshot), Open nukes the sub-index dir, recreates an empty bleve, and writes a `.needs_reindex_<name>` marker so the daemon's next pass does a full rebuild. Open never returns a typed MappingCorruptError to callers — the entire recovery is internal. Callers that need bleve (e.g. `ox code search`) just see empty results until the daemon repopulates; callers that don't need bleve (e.g. `ox code insights`) should use OpenSQLOnly to skip bleve entirely.

func OpenSQLOnly added in v0.9.0 

func OpenSQLOnly(root string) (*Store, error)

OpenSQLOnly opens the SQLite half of a Store without touching bleve. Bleve sub-indexes are left nil — search and dirty-overlay APIs MUST NOT be used on a SQL-only store; the Store will panic on nil bleve access.

Used by read paths that only query SQL data (e.g. `ox code insights`, `ox code status` counters) so they keep working when bleve is mid-rebuild or otherwise unavailable. SQLite's WAL mode makes concurrent reads safe even while the daemon is actively writing.

func (*Store) AttachDirtyIndex added in v0.5.0 

func (s *Store) AttachDirtyIndex(dirtyBlevePath string) error

AttachDirtyIndex opens an existing on-disk dirty overlay index (built by the daemon) and aliases it with the shared CodeIndex for transparent search. Uses a default key; for multi-worktree support use AttachDirtyIndexByID.

func (*Store) AttachDirtyIndexByID added in v0.6.1 

func (s *Store) AttachDirtyIndexByID(id, dirtyBlevePath string) error

AttachDirtyIndexByID opens an on-disk dirty overlay and adds it to the overlay map under the given ID. If the ID is already attached, the old overlay is detached first. Rebuilds the combined alias to include all active overlays.

func (*Store) AttachDirtyOverlay added in v0.5.0 

func (s *Store) AttachDirtyOverlay() error

AttachDirtyOverlay creates an in-memory Bleve index for dirty worktree files and combines it with the shared CodeIndex via IndexAlias. Search code using CombinedCodeIndex will transparently search both. Primarily used in tests; production uses AttachDirtyIndex for on-disk overlays.

func (*Store) Begin 

func (s *Store) Begin() (*sql.Tx, error)

Begin starts a new transaction.

func (*Store) BeginTx 

func (s *Store) BeginTx(ctx context.Context, opts *sql.TxOptions) (*sql.Tx, error)

BeginTx starts a new transaction with the given context and options.

func (*Store) CheckIntegrity 

func (s *Store) CheckIntegrity() error

CheckIntegrity validates that the SQLite database and all Bleve indexes are healthy. Returns nil if everything is fine, ErrCorrupt otherwise.

On a SQL-only store (constructed via OpenSQLOnly), the bleve indexes are nil by design — those checks are skipped, not treated as a failure. Callers that genuinely need bleve integrity must use Open, not OpenSQLOnly.

func (*Store) Close 

func (s *Store) Close() error

Close closes all resources. It is safe to call multiple times. Bleve sub-indexes may be nil on a SQL-only store (see OpenSQLOnly); Close skips nil indexes rather than panicking.

func (*Store) DBPath added in v0.6.0 

func (s *Store) DBPath() string

DBPath returns the path to the SQLite database file.

func (*Store) DetachDirtyIndexByID added in v0.6.1 

func (s *Store) DetachDirtyIndexByID(id string)

DetachDirtyIndexByID closes and removes a specific dirty overlay by ID. Rebuilds the combined alias with remaining overlays.

func (*Store) DetachDirtyOverlay added in v0.5.0 

func (s *Store) DetachDirtyOverlay()

DetachDirtyOverlay closes all attached dirty overlays and resets CombinedCodeIndex.

func (*Store) DiffSnippet added in v0.10.0 

func (s *Store) DiffSnippet(oldHash, newHash, path string) string

DiffSnippet returns the diff text (indexer-format) for one file change, re-derived from the two blob content_hashes. Empty hashes are treated as "no content on that side" (add/delete). A per-Store LRU caches recent diffs so repeat hits during a search session don't re-read blobs.

Returns "" when neither side could be read (blob missing, binary, or too large) — callers treat empty as "no snippet available" rather than an error.

func (*Store) DirtyCodeIndex added in v0.6.1 

func (s *Store) DirtyCodeIndex() bleve.Index

DirtyCodeIndex returns the first dirty overlay index found, or nil. Used by callers that need direct access to a dirty index (e.g., for indexing docs).

func (*Store) DirtyOverlayCount added in v0.6.1 

func (s *Store) DirtyOverlayCount() int

DirtyOverlayCount returns the number of currently attached dirty overlays.

func (*Store) Exec 

func (s *Store) Exec(query string, args ...interface{}) (sql.Result, error)

Exec executes a SQL statement that doesn't return rows.

func (*Store) Maintain added in v0.6.0 

func (s *Store) Maintain(ctx context.Context) MaintenanceResult

Maintain runs cleanup: prunes orphaned blobs, old diffs, and vacuums. Safe to call while the store is in use — all operations use transactions.

func (*Store) Queries added in v0.6.0 

func (s *Store) Queries() *codedbsqlc.Queries

Queries returns the sqlc-generated typed queries bound to the store's DB.

func (*Store) Query 

func (s *Store) Query(query string, args ...interface{}) (*sql.Rows, error)

Query executes a SQL query and returns the rows.

func (*Store) QueryContext 

func (s *Store) QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)

QueryContext executes a SQL query with context and returns the rows.

func (*Store) QueryRow 

func (s *Store) QueryRow(query string, args ...interface{}) *sql.Row

QueryRow executes a SQL query expected to return at most one row.

func (*Store) ReadBlob added in v0.10.0 

func (s *Store) ReadBlob(contentHash string) []byte

ReadBlob returns blob content by content_hash by scanning the bare repos registered in the `repos` table. Lazily opens repo handles on the first call and caches them on the Store. Returns nil if the blob is not found in any repo, unreadable, larger than maxReadBlobBytes, or if no repos are registered. Safe for concurrent callers (per-repo mutex inside).

Used by the search read path (ADR-018 phase 1b) to re-derive code snippets + line numbers from source now that Bleve no longer stores the content.

func (*Store) ReposDir 

func (s *Store) ReposDir() string

ReposDir returns the path to the bare git repos directory.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.