codedb

package
v0.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 16, 2026 License: MIT Imports: 7 Imported by: 0

README

codedb

Local code search engine integrated into ox. Indexes git repositories into SQLite + Bleve and supports queries.

Packages

  • codedb — Facade (Open, Search, RawSQL, IndexRepo)
  • index — Git clone/fetch via go-git, commit walking, blob dedup, Bleve indexing
  • search — Query parser, execution planner (SQL/Bleve/Intersect), SQL translator
  • store — SQLite + Bleve storage layer (schema, migrations, convenience methods)
  • symbols — Symbol/ref types and parser interface (stub — no CGO tree-sitter)
  • comments — Comment extraction via character-level scanner (pure Go, no CGO)
  • language — File extension → language detection

Query syntax

spawn                          # bare text search
lang:rust file:*.rs fn         # filters
type:symbol Runtime            # symbol search
type:comment retry             # comment search
type:comment ckind:doc lang:go # doc comments in Go
type:commit author:alice       # commit search
type:diff streaming            # diff search
calls:groupby                  # call graph
/err\d+/                       # regex
foo OR bar                     # disjunction

CLI

ox code index <url>          # clone + index a repo
ox code search <query>       # search indexed code
ox code sql <sql>            # raw SQL against the DB

Data lives in ~/.local/share/sageox/codedb/ (XDG).

Benchmarks

Run with bash internal/codedb/bench.sh. Only the default branch (main/master) is indexed.

Metric sageox/ox ylow/SFrameRust tokio-rs/tokio
Commits 118 195 4,415
Blobs 1,729 662 18,689
Index (s) 9 4 82
Re-index (s) 1 0 49*
SQLite DB (MB) 1 1 7
Bleve FTS (MB) 89 55 629
Git repos (MB) 7 3 20
Total (MB) 96 57 655
Search latency (ms) ~68 ~76 ~68

* tokio re-index time is dominated by git fetch network I/O (48.8s); actual re-index work is 0ms.

Search queries tested: spawn, type:symbol Runtime, lang:go func, type:diff streaming, file:main.go func (median of 3 iterations).

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB is the top-level CodeDB facade.

func Open

func Open(root string) (*DB, error)

Open opens (or creates) a CodeDB at the given root directory.

func (*DB) AttachDirtyIndex added in v0.5.0

func (db *DB) AttachDirtyIndex(worktreePath string) error

AttachDirtyIndex opens the daemon-built on-disk dirty overlay and aliases it with the shared CodeIndex for transparent search.

func (*DB) AttachDirtyOverlay added in v0.5.0

func (db *DB) AttachDirtyOverlay() error

AttachDirtyOverlay creates an in-memory Bleve overlay for dirty worktree files. Primarily used in tests; production uses AttachDirtyIndex for on-disk overlays.

func (*DB) BuildDirtyIndex added in v0.5.0

func (db *DB) BuildDirtyIndex(ctx context.Context, localPath string, opts index.IndexOptions) (int, error)

BuildDirtyIndex builds an on-disk Bleve index of dirty (uncommitted) files. Called by the daemon after committed content indexing.

func (*DB) Close

func (db *DB) Close() error

Close releases all resources.

func (*DB) DetachDirtyOverlay added in v0.5.0

func (db *DB) DetachDirtyOverlay()

DetachDirtyOverlay closes any attached dirty overlay.

func (*DB) IndexGitHubData added in v0.5.0

func (db *DB) IndexGitHubData(ctx context.Context, ledgerPath string, progress func(string)) (*index.GitHubIndexStats, error)

IndexGitHubData reads PR/issue JSON files from the ledger and indexes them into CodeDB.

func (*DB) IndexLocalRepo

func (db *DB) IndexLocalRepo(ctx context.Context, localPath string, opts index.IndexOptions) error

IndexLocalRepo indexes a local git repository's committed content.

func (*DB) IndexRepo

func (db *DB) IndexRepo(ctx context.Context, url string, opts index.IndexOptions) error

IndexRepo clones/fetches and indexes a git repository.

func (*DB) ParseComments added in v0.5.0

func (db *DB) ParseComments(ctx context.Context, progress func(string)) (index.CommentStats, error)

ParseComments extracts comments from all unparsed blobs with supported languages.

func (*DB) ParseSymbols

func (db *DB) ParseSymbols(ctx context.Context, progress func(string)) (index.ParseStats, error)

ParseSymbols extracts symbols from all unparsed blobs with supported languages.

func (*DB) RawSQL

func (db *DB) RawSQL(query string) ([]string, [][]string, error)

RawSQL executes a raw SQL query and returns results as column-value pairs.

func (*DB) Search

func (db *DB) Search(ctx context.Context, input string) ([]search.Result, error)

Search parses and executes a query.

func (*DB) Store

func (db *DB) Store() *store.Store

Store returns the underlying store for direct access.

func (*DB) TranslateQuery

func (db *DB) TranslateQuery(input string) (*search.TranslatedQuery, error)

TranslateQuery parses a query and returns the generated SQL without executing.

Directories

Path Synopsis
Package comments extracts comments from source code using a character-level scanner configured by language-specific syntax families.
Package comments extracts comments from source code using a character-level scanner configured by language-specific syntax families.
Package symbols extracts symbol definitions and references from source code using tree-sitter for accurate AST-based parsing.
Package symbols extracts symbol definitions and references from source code using tree-sitter for accurate AST-based parsing.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL