codedb

package
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 4, 2026 License: MIT Imports: 7 Imported by: 0

README

codedb

Local code search engine integrated into ox. Indexes git repositories into SQLite + Bleve and supports queries.

Packages

  • codedb — Facade (Open, Search, RawSQL, IndexRepo)
  • index — Git clone/fetch via go-git, commit walking, blob dedup, Bleve indexing
  • search — Query parser, execution planner (SQL/Bleve/Intersect), SQL translator
  • store — SQLite + Bleve storage layer (schema, migrations, convenience methods)
  • symbols — Symbol/ref types and parser interface (stub — no CGO tree-sitter)
  • comments — Comment extraction via character-level scanner (pure Go, no CGO)
  • language — File extension → language detection

Query syntax

spawn                          # bare text search
lang:rust file:*.rs fn         # filters
type:symbol Runtime            # symbol search
type:comment retry             # comment search
type:comment ckind:doc lang:go # doc comments in Go
type:commit author:alice       # commit search
type:diff streaming            # diff search
calls:groupby                  # call graph
/err\d+/                       # regex
foo OR bar                     # disjunction

CLI

ox code index <url>          # clone + index a repo
ox code search <query>       # search indexed code
ox code sql <sql>            # raw SQL against the DB

Data lives in ~/.local/share/sageox/codedb/ (XDG).

Benchmarks

Run with bash internal/codedb/bench.sh. Only the default branch (main/master) is indexed.

Metric sageox/ox ylow/SFrameRust tokio-rs/tokio
Commits 118 195 4,415
Blobs 1,729 662 18,689
Index (s) 9 4 82
Re-index (s) 1 0 49*
SQLite DB (MB) 1 1 7
Bleve FTS (MB) 89 55 629
Git repos (MB) 7 3 20
Total (MB) 96 57 655
Search latency (ms) ~68 ~76 ~68

* tokio re-index time is dominated by git fetch network I/O (48.8s); actual re-index work is 0ms.

Search queries tested: spawn, type:symbol Runtime, lang:go func, type:diff streaming, file:main.go func (median of 3 iterations).

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB is the top-level CodeDB facade.

func Open

func Open(root string) (*DB, error)

Open opens (or creates) a CodeDB at the given root directory.

func (*DB) AttachAllDirtyIndexes added in v0.6.1

func (db *DB) AttachAllDirtyIndexes() int

AttachAllDirtyIndexes scans the dirty index directory for manifest files and attaches all valid dirty overlays by worktree ID. This gives CLI searches access to all active worktree overlays simultaneously.

func (*DB) AttachDirtyIndex added in v0.5.0

func (db *DB) AttachDirtyIndex(worktreePath string) error

AttachDirtyIndex opens the daemon-built on-disk dirty overlay and aliases it with the shared CodeIndex for transparent search. Uses a default key; for multi-worktree support use AttachDirtyIndexByID.

func (*DB) AttachDirtyIndexByID added in v0.6.1

func (db *DB) AttachDirtyIndexByID(id, dirtyBlevePath string) error

AttachDirtyIndexByID opens an on-disk dirty overlay by worktree ID and path. Multiple overlays can be attached simultaneously; all are merged at query time.

func (*DB) AttachDirtyOverlay added in v0.5.0

func (db *DB) AttachDirtyOverlay() error

AttachDirtyOverlay creates an in-memory Bleve overlay for dirty worktree files. Primarily used in tests; production uses AttachDirtyIndex for on-disk overlays.

func (*DB) BuildDirtyIndex added in v0.5.0

func (db *DB) BuildDirtyIndex(ctx context.Context, localPath string, opts index.IndexOptions) (int, error)

BuildDirtyIndex builds an on-disk Bleve index of dirty (uncommitted) files. Called by the daemon after committed content indexing.

func (*DB) Close

func (db *DB) Close() error

Close releases all resources.

func (*DB) DetachDirtyIndexByID added in v0.6.1

func (db *DB) DetachDirtyIndexByID(id string)

DetachDirtyIndexByID removes a specific dirty overlay by ID.

func (*DB) DetachDirtyOverlay added in v0.5.0

func (db *DB) DetachDirtyOverlay()

DetachDirtyOverlay closes all attached dirty overlays.

func (*DB) DirtyOverlayCount added in v0.6.1

func (db *DB) DirtyOverlayCount() int

DirtyOverlayCount returns the number of currently attached dirty overlays.

func (*DB) GCDirtyIndexes added in v0.6.0

func (db *DB) GCDirtyIndexes() (int, error)

GCDirtyIndexes removes stale dirty overlay directories for worktrees that no longer exist on disk. Returns the number of overlays removed.

func (*DB) IndexGitHubData added in v0.5.0

func (db *DB) IndexGitHubData(ctx context.Context, ledgerPath string, progress func(string)) (*index.GitHubIndexStats, error)

IndexGitHubData reads PR/issue JSON files from the ledger and indexes them into CodeDB.

func (*DB) IndexLocalRepo

func (db *DB) IndexLocalRepo(ctx context.Context, localPath string, opts index.IndexOptions) error

IndexLocalRepo indexes a local git repository's committed content.

func (*DB) IndexRepo

func (db *DB) IndexRepo(ctx context.Context, url string, opts index.IndexOptions) error

IndexRepo clones/fetches and indexes a git repository.

func (*DB) ParseComments added in v0.5.0

func (db *DB) ParseComments(ctx context.Context, progress func(string)) (index.CommentStats, error)

ParseComments extracts comments from all unparsed blobs with supported languages.

func (*DB) ParseSymbols

func (db *DB) ParseSymbols(ctx context.Context, progress func(string)) (index.ParseStats, error)

ParseSymbols extracts symbols from all unparsed blobs with supported languages.

func (*DB) RawSQL

func (db *DB) RawSQL(query string) ([]string, [][]string, error)

RawSQL executes a raw SQL query and returns results as column-value pairs.

func (*DB) Search

func (db *DB) Search(ctx context.Context, input string) ([]search.Result, error)

Search parses and executes a query.

func (*DB) Store

func (db *DB) Store() *store.Store

Store returns the underlying store for direct access.

func (*DB) TranslateQuery

func (db *DB) TranslateQuery(input string) (*search.TranslatedQuery, error)

TranslateQuery parses a query and returns the generated SQL without executing.

Directories

Path Synopsis
Package comments extracts comments from source code using a character-level scanner configured by language-specific syntax families.
Package comments extracts comments from source code using a character-level scanner configured by language-specific syntax families.
Package symbols extracts symbol definitions and references from source code using tree-sitter via gotreesitter (pure Go, no CGO required).
Package symbols extracts symbol definitions and references from source code using tree-sitter via gotreesitter (pure Go, no CGO required).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL