Documentation
¶
Overview ¶
Package storage provides SQLite persistence for mnemos: opening the database with sane PRAGMAs and running embedded goose migrations.
Index ¶
- func AppendEvent(ctx context.Context, tx *sql.Tx, ...) error
- func CollectionByURI(ctx context.Context, db *sql.DB, uri string) (string, bool, error)
- func CountEmbeddings(ctx context.Context, db *sql.DB, model string) (int, error)
- func CountInboundLinks(ctx context.Context, db *sql.DB, dstURI string) (int, error)
- func DeleteByURI(ctx context.Context, db *sql.DB, uri string) error
- func DeleteByURITx(ctx context.Context, tx *sql.Tx, uri string) error
- func DocumentHashByURI(ctx context.Context, db *sql.DB, uri string) (string, bool, error)
- func EncodeVector(vec []float32) []byte
- func EscapeLike(s string) string
- func GetChunkByID(ctx context.Context, db *sql.DB, id string) (*model.Chunk, error)
- func GetChunkContentsByIDs(ctx context.Context, db *sql.DB, ids []string) (map[string]string, error)
- func GetChunksByDocURI(ctx context.Context, db *sql.DB, uri string) ([]model.Chunk, error)
- func GetDocumentByID(ctx context.Context, db *sql.DB, id string) (*model.Document, error)
- func GetDocumentByURI(ctx context.Context, db *sql.DB, uri string) (*model.Document, error)
- func ListURIsByCollection(ctx context.Context, db *sql.DB, collection string) ([]string, error)
- func Migrate(db *sql.DB) error
- func Open(ctx context.Context, path string) (*sql.DB, error)
- func ReplaceChunks(ctx context.Context, tx *sql.Tx, documentID string, chunks []model.Chunk) error
- func ReplaceLinks(ctx context.Context, tx *sql.Tx, srcDoc string, links []model.Link) error
- func UpsertDocument(ctx context.Context, tx *sql.Tx, d model.Document) error
- func UpsertEmbedding(ctx context.Context, tx *sql.Tx, chunkID, model string, dim int, vector []byte) error
- type ChunkRef
- type DocumentRow
- type ListFilter
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AppendEvent ¶
func AppendEvent(ctx context.Context, tx *sql.Tx, id, documentID, eventType, payloadJSON, createdAt string) error
AppendEvent inserts one events row within tx. id must be unique (the pipeline derives it from the document id and timestamp); documentID ties the event to its document so it is cascaded away when the document is deleted (empty maps to NULL for document-less events); payloadJSON is opaque JSON; createdAt is RFC3339.
func CollectionByURI ¶
CollectionByURI returns the stored collection for a uri, or ("", false) when no document with that uri exists. move uses it to preserve a document's collection when re-ingesting it under a new uri (the document id is derived from collection + uri, so a move is a delete-old + ingest-new).
func CountEmbeddings ¶
CountEmbeddings returns the number of stored vectors for model. It backs the reindex summary and status reporting, and lets cross-package tests assert that the reindex path actually persisted vectors.
func CountInboundLinks ¶
CountInboundLinks returns how many link edges point at dstURI (links.dst_doc is a plain uri string with no foreign key). move reports this as the number of inbound markdown links left dangling after a rename (a V0 limitation).
func DeleteByURI ¶
DeleteByURI removes the document with the given uri. Its chunks and links are removed by the ON DELETE CASCADE foreign keys, and the chunks delete trigger (chunks_ad) cleans the FTS external-content index — so a single DELETE on documents fully evicts the document from every searchable surface. Deleting a uri that does not exist is a no-op (no error). The watcher calls this when a tracked file vanishes from disk.
func DeleteByURITx ¶
DeleteByURITx is DeleteByURI within a caller-managed transaction, so a batch of deletes (e.g. the watcher's vanished-file reconcile) commits atomically.
func DocumentHashByURI ¶
DocumentHashByURI returns the stored content_hash for a uri, or ("", false) when no document with that uri exists. The ingest pipeline uses it to skip unchanged files before parsing.
func EncodeVector ¶
EncodeVector serializes an embedding to a little-endian float32 BLOB (4 bytes per dimension). It is the inverse of decodeVector. Storing the raw float32 bytes keeps vectors compact and lets the linear scan decode them directly.
func EscapeLike ¶
EscapeLike escapes the LIKE wildcards (% and _) and the escape character in a literal so a path prefix/extension is matched verbatim rather than as a pattern. Use it together with `ESCAPE '\'` in a LIKE clause. Shared by the document lister and the search filter so both treat user-supplied path/type values literally.
func GetChunkByID ¶
GetChunkByID returns the chunk with the given id, or (nil, nil) when no such chunk exists. It is a read-only accessor used by the MCP read tool.
func GetChunkContentsByIDs ¶
func GetChunkContentsByIDs(ctx context.Context, db *sql.DB, ids []string) (map[string]string, error)
GetChunkContentsByIDs fetches the content of many chunks in a single query, returning a map from chunk id to content. Missing ids are simply absent from the map. An empty ids slice short-circuits to an empty map with no query. It is the batch accessor used by mnemos.context to avoid a per-result round-trip (the N+1 the single-id GetChunkByID would cause in a loop). The id count is bounded by the search limit, well under SQLite's bound-parameter cap.
func GetChunksByDocURI ¶
GetChunksByDocURI returns every chunk owned by the document with the given uri, ordered by ordinal. The chunks table is the source of truth for document content (the file on disk may have moved); reconstructing a document means reassembling these chunks. An empty slice means the uri matched no chunks.
func GetDocumentByID ¶
GetDocumentByID returns the document with the given id, or (nil, nil) when no such document exists. It is a read-only accessor used by the MCP read tool to resolve a chunk's owning document.
func GetDocumentByURI ¶
GetDocumentByURI returns the document with the given uri, or (nil, nil) when no such document exists. It is a read-only accessor used by the MCP read tool.
func ListURIsByCollection ¶
ListURIsByCollection returns every documents.uri in the given collection, unordered. The watcher uses it during startup reconcile to find documents whose backing file has disappeared from disk.
func Migrate ¶
Migrate runs all pending goose migrations against db using the embedded SQL files. It is idempotent: applying it to an already-migrated database is a no-op. The "sqlite3" dialect name is goose's identifier for the SQLite dialect and is independent of the underlying driver.
func Open ¶
Open opens (creating if needed) the SQLite database at path and applies the standard PRAGMAs. The returned *sql.DB is ready for migrations and queries.
func ReplaceChunks ¶
ReplaceChunks deletes the existing chunks for documentID and inserts the provided set within tx. Deleting first lets the FTS triggers cascade-clean the external-content index; the inserts then repopulate it. tags and doc_type are stored denormalized on each chunk so FTS5 external content stays correct.
func ReplaceLinks ¶
ReplaceLinks deletes the outbound edges for srcDoc and inserts the provided set within tx. dst_doc is a plain URI string with no foreign key, so link targets need not be ingested yet.
func UpsertDocument ¶
UpsertDocument inserts or updates a document by uri within tx. On conflict it refreshes the mutable fields (hash, title, sizes, timestamps, frontmatter) and re-stamps indexed_at, keeping the existing id stable.
func UpsertEmbedding ¶
func UpsertEmbedding(ctx context.Context, tx *sql.Tx, chunkID, model string, dim int, vector []byte) error
UpsertEmbedding inserts or replaces the vector for chunkID within tx. The vector is stored as the encoded BLOB; dim and model are stored alongside so a later model change can be detected. The reindex path batches many upserts in a single transaction.
Types ¶
type ChunkRef ¶
ChunkRef is a minimal chunk projection (id + content) used by the reindex path to embed every stored chunk without loading the full chunk rows.
func AllChunkRefs ¶
AllChunkRefs returns the id and content of every chunk, ordered by id for deterministic batching. The reindex path embeds these and stores one vector per chunk. The count is bounded by the corpus size (tens of thousands for the personal-use target), so materializing id+content is acceptable.
type DocumentRow ¶
type DocumentRow struct {
URI string
Collection string
Title string
ModifiedAt string
IndexedAt string
FrontmatterJSON string
SizeBytes int64
}
DocumentRow is a document's stored metadata as returned by ListDocuments. The type/tags a caller may want live inside FrontmatterJSON (the raw YAML re-encoded as JSON); decode it on the caller side rather than joining chunks.
func ListDocuments ¶
func ListDocuments(ctx context.Context, db *sql.DB, f ListFilter) ([]DocumentRow, error)
ListDocuments returns document metadata rows ordered by uri, narrowed by the filter. It backs the browse/list feature and directory move (which lists every document under a uri prefix to re-index it). title/modified_at may be NULL in the schema, so they are scanned through sql.NullString.
type ListFilter ¶
type ListFilter struct {
Collection string // exact collection match
PathPrefix string // documents.uri starts with this slash-relative prefix
FileType string // file extension without the dot (e.g. "md")
Limit int
}
ListFilter narrows ListDocuments. Empty string fields are ignored; Limit <= 0 returns all matching rows.
Source Files