Documentation
¶
Overview ¶
Package store defines the persistence interface and data-transfer types used by all storage backends.
All types in this file are intentionally plain data-transfer objects (DTOs): they have only exported fields and no methods. Their purpose is to carry data across the boundary between the storage layer (SQLite, or any future backend) and consumers (the crawler and the public query client). Do not add behaviour to these types; logic belongs in the Store implementation or the caller.
Package store defines the persistence interface and data-transfer types for documentation storage. The Store interface is the single boundary between the crawler / query layers and any concrete storage backend. The only current implementation is internal/store/sqlite.SQLiteStore, which is backed by a SQLite database with FTS5 full-text search.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrFKConstraint = errors.New("store: foreign key constraint")
ErrFKConstraint is returned when an upsert violates a foreign-key constraint, for example when UpsertMethod is called with an entityID that does not exist.
var ErrNotFound = errors.New("store: not found")
ErrNotFound is returned by Get* methods when the requested entity does not exist in the store. Callers should use errors.Is(err, store.ErrNotFound) to distinguish a missing record from a real database error.
Functions ¶
This section is empty.
Types ¶
type CrawlProgressItem ¶
type CrawlProgressItem struct {
URL string
ItemType string
Status string
ErrorType string
ErrorMessage string
ParentEntity string
}
CrawlProgressItem is a DTO that maps one-to-one to a row in the crawl_progress table. Each row records the outcome of crawling one URL. ErrorType and ErrorMessage are empty when Status is "success". ParentEntity is the entity URL that owns this method URL, or empty for entity-level items.
type CrawlSession ¶
type CrawlSession struct {
ID int64
LibraryID string
Status string
TotalURLs int
SuccessCount int
FailCount int
SkipCount int
StartedAt time.Time
CompletedAt *time.Time
}
CrawlSession is a DTO that maps one-to-one to a row in the crawl_sessions table. Status is one of "running", "completed", or "interrupted". CompletedAt is nil while the session is still running.
type CrawlStats ¶
type CrawlStats struct {
Total int
Success int
Failed int
Skipped int
FailuresByType map[string]int
}
CrawlStats is a DTO holding aggregate counts computed from the crawl_progress table for a single session. FailuresByType maps error type strings (e.g., "http_404", "parse_error") to their occurrence counts.
type EntityRecord ¶
type EntityRecord struct {
ID int64 `json:"id"`
LibraryID string `json:"library_id"`
Slug string `json:"slug"`
Name string `json:"name"`
Kind string `json:"kind"`
Description string `json:"description"`
SourceFile string `json:"source_file"`
SourceCode string `json:"source_code"`
URL string `json:"url"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
EntityRecord is a DTO that maps one-to-one to a row in the entities table. An entity is a top-level PHP construct — class, interface, trait, or stand-alone function. Slug is derived from the entity's URL path, not from the display name, to guarantee uniqueness across the library.
type EntityStore ¶ added in v0.1.3
type EntityStore interface {
// UpsertEntity inserts or replaces an entity (class, function, etc.) within
// the given library. Properties are replaced atomically inside a transaction.
// Returns the row ID of the upserted entity.
UpsertEntity(ctx context.Context, libraryID string, entity *source.Entity) (int64, error)
// GetEntity returns the entity with the given slug inside the specified
// library, or (nil, ErrNotFound) when no such entity exists.
GetEntity(ctx context.Context, libraryID, slug string) (*EntityRecord, error)
// GetEntityByID returns the entity with the given row ID, or (nil, ErrNotFound)
// when no such entity exists.
GetEntityByID(ctx context.Context, id int64) (*EntityRecord, error)
// ListEntities returns all entities belonging to the given library.
ListEntities(ctx context.Context, libraryID string) ([]EntityRecord, error)
// UpdateSnippetCount sets the pre-computed snippet count on the library record.
// The authoritative final count is written once after a full crawl completes.
UpdateSnippetCount(ctx context.Context, libraryID string, count int) error
// ComputeSnippetCount calculates the total number of searchable snippets for
// the library by aggregating entity and method counts in the database.
// Returns the count without modifying the stored snippet_count field.
ComputeSnippetCount(ctx context.Context, libraryID string) (int, error)
}
EntityStore handles entity and method persistence. This is the primary write interface for the crawler worker pool.
type LibraryRecord ¶
type LibraryRecord struct {
ID string `json:"id"`
Name string `json:"name"`
Description string `json:"description"`
SourceURL string `json:"source_url"`
Version string `json:"version"`
TrustScore float64 `json:"trust_score"`
SnippetCount int `json:"snippet_count"`
CrawledAt time.Time `json:"crawled_at"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
LibraryRecord is a DTO that maps one-to-one to a row in the libraries table. All fields are exposed; callers read them directly. The record includes CreatedAt and UpdatedAt timestamps that are not surfaced in the public defsource.Library type — they are internal housekeeping fields.
type LibraryStore ¶ added in v0.1.3
type LibraryStore interface {
// UpsertLibrary inserts or updates a library record identified by id, using
// the fields from meta. If the library already exists its metadata is refreshed.
UpsertLibrary(ctx context.Context, id string, meta source.LibraryMeta) error
// GetLibrary returns the library record for the given id, or (nil, ErrNotFound)
// when no such library exists.
GetLibrary(ctx context.Context, id string) (*LibraryRecord, error)
// SearchLibraries performs a case-insensitive substring search on library
// names and returns matching records ranked by relevance.
SearchLibraries(ctx context.Context, query string) ([]LibraryRecord, error)
// ListLibraries returns all library records in the store.
ListLibraries(ctx context.Context) ([]LibraryRecord, error)
}
LibraryStore handles library registration and lookup. Consumers that only need library-level operations (e.g., the HTTP search endpoint) may declare their dependency as LibraryStore rather than the full Store interface.
type MethodRecord ¶
type MethodRecord struct {
ID int64 `json:"id"`
EntityID int64 `json:"entity_id"`
Slug string `json:"slug"`
Name string `json:"name"`
Signature string `json:"signature"`
Description string `json:"description"`
ParametersJSON string `json:"parameters_json"`
ReturnType string `json:"return_type"`
ReturnDesc string `json:"return_desc"`
SourceCode string `json:"source_code"`
WrappedSource string `json:"wrapped_source"`
WrappedMethod string `json:"wrapped_method"`
URL string `json:"url"`
Since string `json:"since"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
MethodRecord is a DTO that maps one-to-one to a row in the methods table. ParametersJSON stores the method's parameter list as a JSON array; callers are responsible for unmarshalling it (see defsource.parseParameters). This is the raw storage representation — the JSON encoding detail is part of the contract for consumers of this type.
WrappedSource and WrappedMethod are non-empty only for wrapper functions whose implementation delegates to another function or method; they carry the resolved source code and identifier of the ultimate target.
type MethodStore ¶ added in v0.1.3
type MethodStore interface {
// UpsertMethod inserts or replaces a method record under the given entity.
// Relations are replaced atomically inside a transaction.
UpsertMethod(ctx context.Context, entityID int64, method *source.Method) error
// GetMethod returns the method with the given slug under the specified entity,
// or (nil, ErrNotFound) when no such method exists.
GetMethod(ctx context.Context, entityID int64, slug string) (*MethodRecord, error)
// ListMethods returns all methods belonging to the given entity.
ListMethods(ctx context.Context, entityID int64) ([]MethodRecord, error)
// ListRelations returns all cross-reference relations for the given method.
ListRelations(ctx context.Context, methodID int64) ([]RelationRecord, error)
}
MethodStore handles method and relation persistence.
type RelationRecord ¶
type RelationRecord struct {
ID int64 `json:"id"`
MethodID int64 `json:"method_id"`
Kind string `json:"kind"`
TargetName string `json:"target_name"`
TargetURL string `json:"target_url"`
Description string `json:"description"`
}
RelationRecord is a DTO that maps one-to-one to a row in the relations table. A relation is a PHPDoc cross-reference (@see / @uses) from a method to another function or method. Kind is "uses" for both @see and @uses tags.
type SearchResult ¶
type SearchResult struct {
EntityID int64 `json:"entity_id"`
MethodID *int64 `json:"method_id,omitempty"`
SnippetType string `json:"snippet_type"` // "class" or "method"
EntityName string `json:"entity_name"`
MethodName string `json:"method_name,omitempty"`
Rank float64 `json:"rank"`
}
SearchResult is a DTO that represents a single BM25-ranked FTS5 search hit. SnippetType is "class" when MethodID is nil (entity-level hit) and "method" when MethodID is non-nil (method-level hit). Rank holds the raw BM25 score; lower (more negative) values indicate a better match.
type SearchStore ¶ added in v0.1.3
type SearchStore interface {
// Search executes a BM25-ranked FTS5 query against the search index for the
// given library. limit caps returned results. mode is "all" (AND semantics,
// default) or "any" (OR semantics). Returns nil, not an empty slice, when
// the query is empty or produces no hits.
Search(ctx context.Context, libraryID, query string, limit int, mode string) ([]SearchResult, error)
// RebuildIndex rebuilds the FTS5 search index for the given library from
// scratch. This is a potentially long-running operation that acquires the
// write lock for the duration of the full transaction.
RebuildIndex(ctx context.Context, libraryID string) error
}
SearchStore handles full-text search index management and query execution.
type SessionStore ¶ added in v0.1.3
type SessionStore interface {
// CreateCrawlSession opens a new crawl session for the library and returns
// its session ID. totalURLs is the count of entity URLs discovered.
CreateCrawlSession(ctx context.Context, libraryID string, totalURLs int) (int64, error)
// UpdateCrawlSession updates the status and counters for an existing session.
// status should be one of "running", "completed", or "interrupted".
UpdateCrawlSession(ctx context.Context, sessionID int64, status string, success, fail, skip int) error
// GetLastSession returns the most recent crawl session for the library, or
// (nil, ErrNotFound) when no session exists.
GetLastSession(ctx context.Context, libraryID string) (*CrawlSession, error)
// RecordProgress records a single URL's crawl outcome in the session log.
RecordProgress(ctx context.Context, sessionID int64, item *CrawlProgressItem) error
// GetProcessedURLs returns a map of URL → status for all URLs recorded in
// the given session. Used to skip already-processed URLs on resume.
GetProcessedURLs(ctx context.Context, sessionID int64) (map[string]string, error)
// GetCrawlStats returns aggregate success/failure/skip counts for a session.
GetCrawlStats(ctx context.Context, sessionID int64) (*CrawlStats, error)
// GetFailures returns all failed crawl progress items for a session. Used by
// the --retry-failed crawl mode.
GetFailures(ctx context.Context, sessionID int64) ([]CrawlProgressItem, error)
}
SessionStore handles crawl session audit records. Consumers that only perform resume/retry operations may declare their dependency as SessionStore.
type Store ¶
type Store interface {
LibraryStore
EntityStore
MethodStore
SearchStore
SessionStore
// Close releases any resources held by the store (e.g., the database
// connection pool). Must be called when the store is no longer needed.
Close() error
}
Store is the unified persistence interface for documentation data. It composes all focused sub-interfaces so that a single injection point serves both the crawler and the query client. Consumers that need only a subset of operations should declare their dependency on the appropriate sub-interface:
- LibraryStore — library registration and lookup
- EntityStore — entity and method persistence
- MethodStore — method and relation persistence
- SearchStore — FTS5 index management and query execution
- SessionStore — crawl session audit records
Implementations are responsible for their own thread safety. The SQLiteStore implementation serialises all writes behind a sync.Mutex while allowing concurrent reads via SQLite WAL mode; callers may invoke any method from multiple goroutines simultaneously.
Source Files