Documentation
¶
Overview ¶
Package kb provides a knowledge base system for storing and searching documents.
Documents are chunked and embedded for hybrid search combining vector similarity and full-text search using Reciprocal Rank Fusion (RRF).
Index ¶
- func ExtractTagsFromMetadata(meta map[string]interface{}) []string
- func InjectTagsIntoMetadata(meta map[string]interface{}, tags []string) map[string]interface{}
- func SerializeFrontMatter(fm FrontMatter, body string) string
- func StripFrontMatter(raw string) string
- type Document
- type FrontMatter
- type KBStore
- func (s *KBStore) AddDocument(ctx context.Context, filePath, content string, metadata map[string]interface{}) error
- func (s *KBStore) AddDocumentWithEmbeddings(ctx context.Context, filePath, content string, metadata map[string]interface{}, ...) error
- func (s *KBStore) ConfigureFilesystemMirror(dirPath string) error
- func (s *KBStore) CountDocuments(ctx context.Context) (int64, error)
- func (s *KBStore) DeleteDocument(ctx context.Context, filePath string) error
- func (s *KBStore) DeleteDocumentFromFilesystem(filePath string) error
- func (s *KBStore) FilesystemMirrorPath() string
- func (s *KBStore) GetDocument(ctx context.Context, filePath string) (*Document, error)
- func (s *KBStore) ListDocuments(ctx context.Context, limit, offset int) ([]Document, error)
- func (s *KBStore) RebuildFTS(ctx context.Context) error
- func (s *KBStore) SearchDocuments(ctx context.Context, query string, limit int) ([]SearchResult, error)
- func (s *KBStore) SearchDocumentsWithOptions(ctx context.Context, query string, limit int, opts SearchOptions) ([]SearchResult, error)
- func (s *KBStore) SetSyncWorkers(workers int)
- func (s *KBStore) SetWriteProxy(proxy *dbproxy.DBProxy)
- func (s *KBStore) SyncDirectory(ctx context.Context, dirPath string) error
- func (s *KBStore) SyncDirectoryWithStats(ctx context.Context, dirPath string, deleteMissing bool) (SyncStats, error)
- func (s *KBStore) UpdateDocument(ctx context.Context, filePath, content string, metadata map[string]interface{}) error
- func (s *KBStore) WatchDirectory(ctx context.Context, dirPath string) error
- func (s *KBStore) WriteDocumentToFilesystem(filePath, content string) error
- type SearchOptions
- type SearchResult
- type SyncStats
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ExtractTagsFromMetadata ¶ added in v0.413.0
ExtractTagsFromMetadata reads the "tags" key from a metadata map and returns them as a string slice. Returns nil if no tags are present.
func InjectTagsIntoMetadata ¶ added in v0.413.0
InjectTagsIntoMetadata ensures tags are stored in the metadata map under "tags". If tags is nil/empty and no existing tags, the key is left absent.
func SerializeFrontMatter ¶ added in v0.413.0
func SerializeFrontMatter(fm FrontMatter, body string) string
SerializeFrontMatter produces a document string with YAML front matter prepended to the body content.
func StripFrontMatter ¶ added in v0.413.0
StripFrontMatter removes any YAML front matter from raw content and returns only the body. This is a convenience wrapper around ParseFrontMatter.
Types ¶
type Document ¶
type Document struct {
ID int64
FilePath string
Content string
Metadata map[string]interface{}
Tags []string
CreatedAt time.Time
UpdatedAt time.Time
}
Document represents a stored document in the knowledge base.
type FrontMatter ¶ added in v0.413.0
type FrontMatter struct {
CreatedAt time.Time `yaml:"created_at,omitempty"`
UpdatedAt time.Time `yaml:"updated_at,omitempty"`
Tags []string `yaml:"tags,omitempty"`
}
FrontMatter holds the YAML front matter fields for KB documents.
func MergeFrontMatter ¶ added in v0.413.0
func MergeFrontMatter(existing, incoming FrontMatter) FrontMatter
MergeFrontMatter combines existing and incoming front matter. It preserves created_at from existing, sets updated_at to now, and replaces tags with new if provided (otherwise keeps existing).
func NewFrontMatter ¶ added in v0.413.0
func NewFrontMatter(tags []string) FrontMatter
NewFrontMatter creates a fresh FrontMatter with created_at/updated_at set to now.
func ParseFrontMatter ¶ added in v0.413.0
func ParseFrontMatter(raw string) (FrontMatter, string, error)
ParseFrontMatter splits YAML front matter from the body of a document. If no valid front matter is found, it returns a zero FrontMatter and the original content as body.
type KBStore ¶
type KBStore struct {
// contains filtered or unexported fields
}
KBStore manages knowledge base documents with chunking, embeddings, and hybrid search. It uses SQLite for storage with FTS5 for full-text search and in-memory vector search.
func NewKBStore ¶
NewKBStore creates a new KBStore instance.
Parameters:
- db: SQLite database connection
- embedder: Embedder for generating chunk embeddings
- chunkSize: Maximum chunk size in characters (0 = use default)
- chunkOverlap: Overlap between consecutive chunks (0 = use default)
func (*KBStore) AddDocument ¶
func (s *KBStore) AddDocument(ctx context.Context, filePath, content string, metadata map[string]interface{}) error
AddDocument adds a new document to the knowledge base. It chunks the content, generates embeddings, and updates the FTS index.
func (*KBStore) AddDocumentWithEmbeddings ¶ added in v0.326.0
func (s *KBStore) AddDocumentWithEmbeddings(ctx context.Context, filePath, content string, metadata map[string]interface{}, chunks []string, embeddings [][]float32) error
AddDocumentWithEmbeddings inserts a document using pre-computed chunks and embeddings. Called by the primary IPC dispatcher when a secondary forwards a KBAddDocument write. No embedding generation is performed; the provided values are stored directly.
func (*KBStore) ConfigureFilesystemMirror ¶ added in v0.160.0
ConfigureFilesystemMirror sets a base directory where KB documents are mirrored as markdown files on add/update/delete operations.
func (*KBStore) CountDocuments ¶
CountDocuments returns the total number of documents.
func (*KBStore) DeleteDocument ¶
DeleteDocument removes a document and all its chunks from the knowledge base.
func (*KBStore) DeleteDocumentFromFilesystem ¶ added in v0.160.0
DeleteDocumentFromFilesystem removes a mirrored KB document from disk. If no mirror path is configured, this is a no-op.
func (*KBStore) FilesystemMirrorPath ¶ added in v0.160.0
FilesystemMirrorPath returns the configured filesystem mirror path.
func (*KBStore) GetDocument ¶
GetDocument retrieves a document by file path.
func (*KBStore) ListDocuments ¶
ListDocuments returns paginated documents.
func (*KBStore) RebuildFTS ¶
RebuildFTS rebuilds the FTS5 index from kb_chunks.
func (*KBStore) SearchDocuments ¶
func (s *KBStore) SearchDocuments(ctx context.Context, query string, limit int) ([]SearchResult, error)
SearchDocuments performs hybrid search combining vector similarity and FTS. Results are fused using Reciprocal Rank Fusion (RRF).
func (*KBStore) SearchDocumentsWithOptions ¶ added in v0.413.0
func (s *KBStore) SearchDocumentsWithOptions(ctx context.Context, query string, limit int, opts SearchOptions) ([]SearchResult, error)
SearchDocumentsWithOptions performs hybrid search with optional tag filtering and chronological ordering. When opts.Tags is non-empty, results are filtered to documents whose tags fuzzy-match any of the requested tags. When opts.SortByDate is true, results are sorted by updated_at descending.
func (*KBStore) SetSyncWorkers ¶ added in v0.160.0
SetSyncWorkers configures how many concurrent workers are used for KB filesystem import preprocessing (file reads and in-memory preparation). SQLite writes remain serialized through a single writer goroutine.
func (*KBStore) SetWriteProxy ¶ added in v0.326.0
SetWriteProxy configures a DB proxy for mutating operations.
func (*KBStore) SyncDirectory ¶
SyncDirectory imports or syncs all .md files from a directory. Existing documents are updated, new files are added.
func (*KBStore) SyncDirectoryWithStats ¶ added in v0.160.0
func (s *KBStore) SyncDirectoryWithStats(ctx context.Context, dirPath string, deleteMissing bool) (SyncStats, error)
SyncDirectoryWithStats imports or syncs all markdown files from a directory. It recursively scans for .md files, upserts modified documents, and optionally deletes KB documents that no longer exist on disk for that directory source.
func (*KBStore) UpdateDocument ¶
func (s *KBStore) UpdateDocument(ctx context.Context, filePath, content string, metadata map[string]interface{}) error
UpdateDocument updates an existing document's content and metadata. It re-chunks and re-embeds the content.
func (*KBStore) WatchDirectory ¶ added in v0.160.0
WatchDirectory monitors a KB source directory recursively and keeps KB documents in sync with .md file changes in near real-time.
func (*KBStore) WriteDocumentToFilesystem ¶ added in v0.160.0
WriteDocumentToFilesystem writes a KB document to the configured mirror path. If no mirror path is configured, this is a no-op.
type SearchOptions ¶ added in v0.413.0
type SearchOptions struct {
Tags []string // filter results by tags (fuzzy match)
SortByDate bool // sort results by updated_at descending
}
SearchOptions provides optional filtering/sorting for search queries.
type SearchResult ¶
SearchResult represents a ranked search result from the knowledge base.
Source Files