kb

package
v0.715.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 15, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package kb provides a knowledge base system for storing and searching documents.

Documents are chunked and embedded for hybrid search combining vector similarity and full-text search using Reciprocal Rank Fusion (RRF).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DocumentSlugs added in v0.608.1 

func DocumentSlugs(filePath string, aliases []string) []string

DocumentSlugs returns every slug a document can be addressed by: its full path, its basename, and any declared aliases. Link resolution matches a target slug against this set.

func ExtractAliasesFromMetadata added in v0.608.1 

func ExtractAliasesFromMetadata(meta map[string]interface{}) []string

ExtractAliasesFromMetadata reads the "aliases" key from a metadata map. Aliases are the extra slugs a document can be reached by from a [[wiki link]].

func ExtractTagsFromMetadata added in v0.413.0 

func ExtractTagsFromMetadata(meta map[string]interface{}) []string

ExtractTagsFromMetadata reads the "tags" key from a metadata map and returns them as a string slice. Returns nil if no tags are present.

func InjectAliasesIntoMetadata added in v0.608.1 

func InjectAliasesIntoMetadata(meta map[string]interface{}, aliases []string) map[string]interface{}

InjectAliasesIntoMetadata stores aliases in the metadata map under "aliases" so link resolution can read them without re-parsing the document body.

func InjectTagsIntoMetadata added in v0.413.0 

func InjectTagsIntoMetadata(meta map[string]interface{}, tags []string) map[string]interface{}

InjectTagsIntoMetadata ensures tags are stored in the metadata map under "tags". If tags is nil/empty and no existing tags, the key is left absent.

func MergeUnknownFrontMatterKeys added in v0.715.6 

func MergeUnknownFrontMatterKeys(meta map[string]interface{}, rawFM map[string]interface{}) map[string]interface{}

MergeUnknownFrontMatterKeys copies every key from rawFM into meta that is not part of reservedFrontMatterKeys, so host-defined front-matter fields (e.g. "status", "project", "milestone") survive into a document's stored metadata instead of being silently dropped by the typed FrontMatter struct. Values that aren't JSON-representable scalars, lists or maps are stringified, since metadata is marshalled to JSON when persisted.

func NormalizeSlug added in v0.608.1 

func NormalizeSlug(raw string) string

NormalizeSlug converts a link target or document path into its canonical slug: lower-cased, markdown extension stripped, spaces/underscores collapsed into single dashes, path separators preserved. 

"pando/features/Foo Bar.md" -> "pando/features/foo-bar"
"Feature_Foo"               -> "feature-foo"

func SerializeFrontMatter added in v0.413.0 

func SerializeFrontMatter(fm FrontMatter, body string) string

SerializeFrontMatter produces a document string with YAML front matter prepended to the body content.

func StripFrontMatter added in v0.413.0 

func StripFrontMatter(raw string) string

StripFrontMatter removes any YAML front matter from raw content and returns only the body. This is a convenience wrapper around ParseFrontMatter.

func WithWriteOrigin added in v0.700.0 

func WithWriteOrigin(ctx context.Context, origin string) context.Context

WithWriteOrigin tags ctx with the subsystem responsible for the writes made under it ("tool", "api", "sync", "watcher", "gc", "remote"). Unset means "tool", which is where writes come from unless something says otherwise.

func WithoutWriteObserver added in v0.700.0 

func WithoutWriteObserver(ctx context.Context) context.Context

WithoutWriteObserver suppresses write publication for everything done under ctx. The IPC dispatcher uses it when applying a write forwarded by another instance: that instance already published the event, and republishing it here would ship the same memory twice.

func WriteOriginOf added in v0.700.0 

func WriteOriginOf(ctx context.Context) string

WriteOriginOf returns the origin tagged on ctx, or "tool".

Types

type BackfillStats added in v0.608.1 

type BackfillStats struct {
	// Candidates is how many documents were selected for (re)scanning.
	Candidates int
	// Scanned is how many of them were actually processed (may be lower than
	// Candidates when the context is cancelled mid-pass).
	Scanned int
	// Documents is how many produced at least one link.
	Documents int
	// Links is the number of link rows written.
	Links int
}

BackfillStats reports what a link backfill pass did. 

type Backlink struct {
	// FilePath of the document that holds the link.
	FilePath string
	// Slug is the target the source wrote, which may be an alias or a bare
	// basename rather than this document's full path.
	Slug string
	// Label is the [[target|label]] text, empty when the link had none.
	Label string
}

Backlink is an incoming link: another document pointing at this one.

type Document 

type Document struct {
	ID       int64
	FilePath string
	// Content is the full document body. It is populated by GetDocument and by
	// direct row-level queries such as the pinned-memory path in
	// GetMemoriesForInjection, but it is NOT populated when a Document arrives
	// embedded in a SearchResult from SearchDocuments/SearchDocumentsWithOptions
	// (searchVector/searchFTS never select it — PANDO-US-0027, kb.go). Read
	// SearchResult.ChunkContent for the matched excerpt instead; a caller that
	// genuinely needs the full body for its top-k hits backfills it with one
	// keyed query (see documentContentByID in kb.go), never by re-adding the
	// column to the scan.
	Content   string
	Metadata  map[string]interface{}
	Tags      []string
	CreatedAt time.Time
	UpdatedAt time.Time
	// Memory system fields (populated only when reading memory documents)
	MemoryKey   string
	MemoryScope string
	Importance  float64
	Hits        int
	Source      string
	Outdated    bool
	ExpiresAt   *time.Time
}

Document represents a stored document in the knowledge base.

type DocumentConverter added in v0.605.1 

type DocumentConverter interface {
	// ConvertFile converts the file at path to Markdown text.
	ConvertFile(path string) (string, error)
	// IsConvertibleDocument reports whether path is a document format that
	// should be auto-converted during ingestion (curated subset, excludes
	// plain markdown/text which is indexed verbatim).
	IsConvertibleDocument(path string) bool
}

DocumentConverter converts rich document formats (docx, pdf, xlsx, …) to Markdown for on-the-fly KB ingestion. It is implemented by internal/convert.Converter and injected via SetDocumentConverter to avoid a hard dependency from the kb package on the conversion library.

type DocumentRef added in v0.416.3 

type DocumentRef struct {
	ID       int64
	FilePath string
}

DocumentRef is a minimal reference to a KB document (ID + path).

type FrontMatter added in v0.413.0 

type FrontMatter struct {
	CreatedAt time.Time `yaml:"created_at,omitempty"`
	UpdatedAt time.Time `yaml:"updated_at,omitempty"`
	Tags      []string  `yaml:"tags,omitempty"`
	// Aliases are extra names a document can be addressed by from a [[wiki link]],
	// on top of its path and basename.
	Aliases []string `yaml:"aliases,omitempty"`
	// Memory fields — only serialized when non-zero/non-empty
	Key        string     `yaml:"key,omitempty"`
	Scope      string     `yaml:"scope,omitempty"`
	Outdated   bool       `yaml:"outdated,omitempty"`
	ExpiresAt  *time.Time `yaml:"expires_at,omitempty"`
	Hits       int        `yaml:"hits,omitempty"`
	Importance float64    `yaml:"importance,omitempty"`
	Source     string     `yaml:"source,omitempty"`
}

FrontMatter holds the YAML front matter fields for KB documents.

func MergeFrontMatter added in v0.413.0 

func MergeFrontMatter(existing, incoming FrontMatter) FrontMatter

MergeFrontMatter combines existing and incoming front matter. It preserves created_at from existing, sets updated_at to now, and replaces tags with new if provided (otherwise keeps existing).

func NewFrontMatter added in v0.413.0 

func NewFrontMatter(tags []string, opts *MemoryOptions) FrontMatter

NewFrontMatter creates a fresh FrontMatter with created_at/updated_at set to now. When opts is non-nil, memory fields are populated from it.

func ParseFrontMatter added in v0.413.0 

func ParseFrontMatter(raw string) (FrontMatter, string, error)

ParseFrontMatter splits YAML front matter from the body of a document. If no valid front matter is found, it returns a zero FrontMatter and the original content as body.

func ParseFrontMatterWithRaw added in v0.715.6 

func ParseFrontMatterWithRaw(raw string) (FrontMatter, map[string]interface{}, string, error)

ParseFrontMatterWithRaw splits YAML front matter from the body of a document, like ParseFrontMatter, but also unmarshals the same YAML block a second time into a generic map so callers can recover front-matter keys the typed FrontMatter struct does not name (e.g. host-defined fields such as "status" or "project"). If no valid front matter is found, it returns a zero FrontMatter, a nil map, and the original content as body.

type KBStore 

type KBStore struct {
	// contains filtered or unexported fields
}

KBStore manages knowledge base documents with chunking, embeddings, and hybrid search. It uses SQLite for storage with FTS5 for full-text search and in-memory vector search.

func NewKBStore 

func NewKBStore(db *sql.DB, embedder embeddings.Embedder, chunkSize, chunkOverlap int) *KBStore

NewKBStore creates a new KBStore instance.

Parameters:

  • db: SQLite database connection
  • embedder: Embedder for generating chunk embeddings
  • chunkSize: Maximum chunk size in characters (0 = use default)
  • chunkOverlap: Overlap between consecutive chunks (0 = use default)

func (*KBStore) AddDocument 

func (s *KBStore) AddDocument(ctx context.Context, filePath, content string, metadata map[string]interface{}) error

AddDocument adds a new document to the knowledge base. It chunks the content, generates embeddings, and updates the FTS index.

The write is published to the observer (observer.go) only after it commits.

func (*KBStore) AddDocumentWithEmbeddings added in v0.326.0 

func (s *KBStore) AddDocumentWithEmbeddings(ctx context.Context, filePath, content string, metadata map[string]interface{}, chunks []string, embeddings [][]float32, embeddingModel string) error

AddDocumentWithEmbeddings inserts a document using pre-computed chunks and embeddings. Called by the primary IPC dispatcher when a secondary forwards a KBAddDocument write. No embedding generation is performed; the provided values are stored directly. embeddingModel is recorded on every inserted chunk that has an embedding (PANDO-US-0029); embedding_dims is derived from each vector's own length. 

func (s *KBStore) BackfillLinks(ctx context.Context) (BackfillStats, error)

BackfillLinks indexes the [[wiki links]] of documents that were stored before the link graph existed. It is the compatibility path for databases upgraded from a version without kb_links: those documents keep their content but have no link rows, and the filesystem sync will not re-add them because it skips files whose mtime has not changed.

No chunking or embedding happens here: the links are extracted from the content already stored, so the pass costs local CPU and a few writes, never a call to the embedding provider.

Candidates are documents whose content contains "[[" and that have no link rows yet — the content itself acts as the marker, so no schema column is needed and the pass is idempotent: once a document is indexed it stops being a candidate. A document whose only "[[" occurrences sit inside code fences yields no links and is re-scanned by later passes, which is harmless.

The pass is a no-op on instances that write through the IPC proxy: the primary owns the writer connection and runs the backfill itself. 

func (s *KBStore) Backlinks(ctx context.Context, filePath string) ([]Backlink, error)

Backlinks returns the documents that link to this one, through its path, its basename or any of its aliases.

A link only counts when it actually resolves here: two documents may share a basename, and the shorter form belongs to whichever one the resolver awards it to, so a link to "[[foo]]" is not a backlink of every document named foo.md.

func (*KBStore) ConfigureFilesystemMirror added in v0.160.0 

func (s *KBStore) ConfigureFilesystemMirror(dirPath string) error

ConfigureFilesystemMirror sets a base directory where KB documents are mirrored as markdown files on add/update/delete operations.

func (*KBStore) CountDocuments 

func (s *KBStore) CountDocuments(ctx context.Context) (int64, error)

CountDocuments returns the total number of documents.

func (*KBStore) CountStaleEmbeddings added in v0.715.6 

func (s *KBStore) CountStaleEmbeddings(ctx context.Context, configuredDims int) (StaleEmbeddingStats, error)

CountStaleEmbeddings counts chunks whose recorded embedding dimension does not match configuredDims. Used by the startup staleness check (internal/app/remembrances.go) and by the enrichment status REST route.

func (*KBStore) DeleteDocument 

func (s *KBStore) DeleteDocument(ctx context.Context, filePath string) error

DeleteDocument removes a document and all its chunks from the knowledge base.

func (*KBStore) DeleteDocumentFromFilesystem added in v0.160.0 

func (s *KBStore) DeleteDocumentFromFilesystem(filePath string) error

DeleteDocumentFromFilesystem removes a mirrored KB document from disk. If no mirror path is configured, this is a no-op.

func (*KBStore) EmbeddingModel added in v0.715.6 

func (s *KBStore) EmbeddingModel() string

EmbeddingModel returns the configured document embedder's model id, as set by SetEmbeddingModel. Empty when never set.

func (*KBStore) FilesystemMirrorPath added in v0.160.0 

func (s *KBStore) FilesystemMirrorPath() string

FilesystemMirrorPath returns the configured filesystem mirror path.

func (*KBStore) GetDocument 

func (s *KBStore) GetDocument(ctx context.Context, filePath string) (*Document, error)

GetDocument retrieves a document by file path.

func (*KBStore) GetExpiredDocuments added in v0.416.3 

func (s *KBStore) GetExpiredDocuments(ctx context.Context) ([]DocumentRef, error)

GetExpiredDocuments returns all non-outdated documents whose expires_at is in the past.

func (*KBStore) GetMemoriesForInjection added in v0.416.3 

func (s *KBStore) GetMemoriesForInjection(ctx context.Context, query string, limit int, maxChars int, pinnedScopes []string) ([]MemoryResult, error)

GetMemoriesForInjection retrieves memory documents ranked for context injection. It combines semantic search results with pinned-scope documents, deduplicates, scores, and trims to fit within the maxChars budget.

func (*KBStore) GetMemoryByKey added in v0.416.3 

func (s *KBStore) GetMemoryByKey(ctx context.Context, key string) (*Document, error)

GetMemoryByKey retrieves a document by its memory_key. Returns nil, nil when no document with the key exists. 

func (s *KBStore) HasLinks(ctx context.Context) (bool, error)

HasLinks reports whether the knowledge base holds any wiki link at all.

It is the guard the tools use before doing any graph work: a knowledge base written before wiki links existed (or one that simply never uses the syntax) answers false, so the tools can skip the resolver entirely and present exactly the output they presented before the graph was added.

func (*KBStore) IncrementMemoryHits added in v0.416.3 

func (s *KBStore) IncrementMemoryHits(ctx context.Context, docID int64, defaultTTLDays int) error

IncrementMemoryHits increments the hit counter for a document and extends its TTL.

func (*KBStore) LinkCountsFor added in v0.608.1 

func (s *KBStore) LinkCountsFor(ctx context.Context, filePaths []string) (map[string]LinkCounts, error)

LinkCountsFor returns the link counts of the given documents in a single pass over the graph, for callers (search results) that want to hint at connections without fetching them. Documents with no links in either direction are absent from the map, and a link-less knowledge base returns an empty map.

func (*KBStore) LinksFrom added in v0.608.1 

func (s *KBStore) LinksFrom(ctx context.Context, filePath string) ([]WikiLink, error)

LinksFrom returns the wiki links stored for a document, ordered by position. Targets are returned unresolved; resolution against real documents is done by the graph queries.

func (*KBStore) ListDocuments 

func (s *KBStore) ListDocuments(ctx context.Context, limit, offset int) ([]Document, error)

ListDocuments returns paginated documents.

func (*KBStore) MarkDocumentOutdated added in v0.416.3 

func (s *KBStore) MarkDocumentOutdated(ctx context.Context, filePath string) error

MarkDocumentOutdated marks a document as outdated and updates its filesystem mirror. 

func (s *KBStore) OutgoingLinks(ctx context.Context, filePath string) ([]LinkTarget, error)

OutgoingLinks returns the wiki links a document declares, each resolved against the current documents. A document with no links yields an empty slice and no error, which is the normal state of a knowledge base that does not use the syntax.

func (*KBStore) RebuildFTS 

func (s *KBStore) RebuildFTS(ctx context.Context) error

RebuildFTS rebuilds the FTS5 index from kb_chunks.

func (*KBStore) RelatedDocuments added in v0.608.1 

func (s *KBStore) RelatedDocuments(ctx context.Context, filePath string, limit int) ([]RelatedDocument, error)

RelatedDocuments returns the neighbours of a document in the graph, ordered by how strongly they are connected: documents it links to, documents that link to it, and — weakly — documents sharing tags with it. limit <= 0 uses a default.

func (*KBStore) RelinkAll added in v0.608.1 

func (s *KBStore) RelinkAll(ctx context.Context) (BackfillStats, error)

RelinkAll rebuilds the whole link graph: every stored link row is dropped and re-extracted from the documents' content. Unlike BackfillLinks it does not skip documents that already have links, so it is the way to pick up a change in the extractor (a new syntax, a different slug rule) that made the stored rows stale.

func (*KBStore) RepairFrontMatterMetadata added in v0.715.6 

func (s *KBStore) RepairFrontMatterMetadata(ctx context.Context, dirPath string) (RepairStats, error)

RepairFrontMatterMetadata re-indexes documents whose stored metadata is missing front-matter-derived keys (tags, aliases, or any other host-defined key) that their source file on disk still declares. It repairs the damage the pre-fix watcher inflicted: every edit made through the watcher replaced a document's metadata with bare source_* fields, discarding tags and any other front-matter key, and recorded a fresh source_mtime_unix so the sync path never noticed the file "changed" again and so never repaired it on its own.

The pass ignores the mtime-skip semantics the regular sync uses: it re-parses every filesystem-backed document's current source file regardless of what source_mtime_unix says, because that is exactly the field the bug left in a false "unchanged" state.

The repair is bounded and one-shot per dirPath: on success it records a marker document and returns immediately without scanning on a later call for the same directory. A corpus already indexed by the fixed sync/watcher code has nothing to repair, so the pass still scans it once but writes nothing (RepairStats.Repaired stays 0).

func (*KBStore) SearchDocuments 

func (s *KBStore) SearchDocuments(ctx context.Context, query string, limit int) ([]SearchResult, error)

SearchDocuments performs hybrid search combining vector similarity and FTS. Results are fused using Reciprocal Rank Fusion (RRF).

func (*KBStore) SearchDocumentsWithOptions added in v0.413.0 

func (s *KBStore) SearchDocumentsWithOptions(ctx context.Context, query string, limit int, opts SearchOptions) ([]SearchResult, error)

SearchDocumentsWithOptions performs hybrid search with optional tag filtering and chronological ordering. When opts.Tags is non-empty, results are filtered to documents whose tags fuzzy-match any of the requested tags. When opts.SortByDate is true, results are sorted by updated_at descending.

func (*KBStore) SearchDocumentsWithOptionsAndStats added in v0.715.6 

func (s *KBStore) SearchDocumentsWithOptionsAndStats(ctx context.Context, query string, limit int, opts SearchOptions) ([]SearchResult, SearchStats, error)

SearchDocumentsWithOptionsAndStats is SearchDocumentsWithOptions plus SearchStats. Callers that need to warn on a stale-embedding skip (kb_search_documents, the REST search route) use this instead of SearchDocumentsWithOptions; every other caller is unaffected. When a search extension middleware is installed (observer.go), stats are not tracked through it — the middleware only ever saw the ranked results, the same as SearchDocumentsWithOptions, so this returns a zero SearchStats in that case rather than misreporting.

func (*KBStore) SetDocumentConverter added in v0.605.1 

func (s *KBStore) SetDocumentConverter(c DocumentConverter)

SetDocumentConverter installs a converter so that supported document files found in the KB source directory are converted to Markdown and indexed, referencing the original file. Passing nil disables conversion (the default), in which case only .md files are indexed.

func (*KBStore) SetEmbeddingModel added in v0.715.6 

func (s *KBStore) SetEmbeddingModel(model string)

SetEmbeddingModel records the document embedder's model id so it can be written alongside every chunk this store inserts. Called once at startup from Remembrances.DocumentEmbeddingModel (internal/rag/service.go).

func (*KBStore) SetSearchMiddleware added in v0.700.0 

func (s *KBStore) SetSearchMiddleware(fn SearchMiddleware)

SetSearchMiddleware installs the search middleware, replacing any previous one. Passing nil removes it.

func (*KBStore) SetSyncWorkers added in v0.160.0 

func (s *KBStore) SetSyncWorkers(workers int)

SetSyncWorkers configures how many concurrent workers are used for KB filesystem import preprocessing (file reads and in-memory preparation). SQLite writes remain serialized through a single writer goroutine.

func (*KBStore) SetWikiLinksEnabled added in v0.609.1 

func (s *KBStore) SetWikiLinksEnabled(enabled bool)

SetWikiLinksEnabled toggles the [[wiki link]] graph. When false no link rows are written and every graph query answers empty, so the KB tools present the output they presented before the graph existed.

Turning it off does not wipe the graph: the rows already indexed stay in the database, invisible, and light up again if it is turned back on — no reindex needed. Only a document rewritten while it is off loses its rows, like any update does.

func (*KBStore) SetWriteObserver added in v0.700.0 

func (s *KBStore) SetWriteObserver(fn WriteObserver)

SetWriteObserver installs the write observer, replacing any previous one. Passing nil removes it.

func (*KBStore) SetWriteProxy added in v0.326.0 

func (s *KBStore) SetWriteProxy(proxy *dbproxy.DBProxy)

SetWriteProxy configures a DB proxy for mutating operations.

func (*KBStore) SyncDirectory 

func (s *KBStore) SyncDirectory(ctx context.Context, dirPath string) error

SyncDirectory imports or syncs all .md files from a directory. Existing documents are updated, new files are added.

func (*KBStore) SyncDirectoryWithStats added in v0.160.0 

func (s *KBStore) SyncDirectoryWithStats(ctx context.Context, dirPath string, deleteMissing bool) (SyncStats, error)

SyncDirectoryWithStats imports or syncs all markdown files from a directory. It recursively scans for .md files, upserts modified documents, and optionally deletes KB documents that no longer exist on disk for that directory source.

func (*KBStore) UpdateDocument 

func (s *KBStore) UpdateDocument(ctx context.Context, filePath, content string, metadata map[string]interface{}) error

UpdateDocument updates an existing document's content and metadata. It re-chunks and re-embeds the content.

func (*KBStore) UpsertMemory added in v0.416.3 

func (s *KBStore) UpsertMemory(ctx context.Context, opts MemoryUpsertOptions) (created bool, err error)

UpsertMemory stores or updates a memory document, keyed by opts.Key when provided. Returns created=true when a new document was inserted, false when an existing one was updated.

It delegates to AddDocument/UpdateDocument on some paths, so the write observer is suppressed for the nested call: the memory event published here carries the scope and key those events cannot know, and one write must be reported once.

func (*KBStore) WantedConcepts added in v0.608.1 

func (s *KBStore) WantedConcepts(ctx context.Context, limit int) ([]WantedConcept, error)

WantedConcepts returns the link targets that no document defines, most linked first: the concepts the knowledge base refers to but never explains. limit <= 0 returns them all.

func (*KBStore) WatchDirectory added in v0.160.0 

func (s *KBStore) WatchDirectory(ctx context.Context, dirPath string) error

WatchDirectory monitors a KB source directory recursively and keeps KB documents in sync with .md file changes in near real-time.

func (*KBStore) WikiLinksEnabled added in v0.609.1 

func (s *KBStore) WikiLinksEnabled() bool

WikiLinksEnabled reports whether the wiki link graph is active.

func (*KBStore) WriteDocumentToFilesystem added in v0.160.0 

func (s *KBStore) WriteDocumentToFilesystem(filePath, content string) error

WriteDocumentToFilesystem writes a KB document to the configured mirror path. If no mirror path is configured, this is a no-op.

type LinkCounts added in v0.608.1 

type LinkCounts struct {
	Outgoing  int
	Backlinks int
}

LinkCounts is how connected a document is: the links it declares and the links it receives.

type LinkTarget added in v0.608.1 

type LinkTarget struct {
	WikiLink
	// FilePath of the document the link points at, empty when unresolved.
	FilePath string
	// Resolved reports whether the target slug matched a document.
	Resolved bool
}

LinkTarget is a wiki link together with the document it resolves to, if any. Unresolved targets are kept on purpose: they are concepts the author referred to but nobody has documented yet.

type MemoryGCService added in v0.416.3 

type MemoryGCService struct {
	// contains filtered or unexported fields
}

MemoryGCService periodically marks expired memory documents as outdated.

func NewMemoryGCService added in v0.416.3 

func NewMemoryGCService(store *KBStore, interval time.Duration) *MemoryGCService

NewMemoryGCService creates a new MemoryGCService.

func (*MemoryGCService) Start added in v0.416.3 

func (g *MemoryGCService) Start(ctx context.Context)

Start launches the GC loop in a background goroutine.

func (*MemoryGCService) Stop added in v0.416.3 

func (g *MemoryGCService) Stop()

Stop signals the background goroutine to exit.

type MemoryOptions added in v0.416.3 

type MemoryOptions struct {
	Key        string
	Scope      string
	Source     string
	Importance float64
	TTLDays    int // 0 = no expiry
}

MemoryOptions carries optional memory-layer settings for NewFrontMatter.

type MemoryResult added in v0.416.3 

type MemoryResult struct {
	Document     Document
	ChunkContent string
	Score        float64
}

MemoryResult wraps a Document with its chunk content and relevance score.

type MemoryUpsertOptions added in v0.416.3 

type MemoryUpsertOptions struct {
	FilePath       string
	Key            string
	Scope          string
	Content        string
	Tags           []string
	Metadata       map[string]interface{}
	Importance     float64
	Source         string
	DefaultTTLDays int // 0 → 180
}

MemoryUpsertOptions configures a memory document upsert operation.

type RelatedDocument added in v0.608.1 

type RelatedDocument struct {
	FilePath string
	Score    float64
	// Reasons explains the score, e.g. "links to it", "links here",
	// "shared tags: kb, wiki".
	Reasons []string
}

RelatedDocument is a neighbour in the document graph, scored by how it is connected.

type RepairStats added in v0.715.6 

type RepairStats struct {
	Scanned  int `json:"scanned"`
	Repaired int `json:"repaired"`
}

RepairStats reports the outcome of RepairFrontMatterMetadata: how many filesystem-backed documents it looked at, and how many of those had their metadata rebuilt because it was missing front-matter-derived keys their source file still declares.

type SearchMiddleware added in v0.700.0 

type SearchMiddleware func(ctx context.Context, query string, limit int, opts SearchOptions, next SearchNext) ([]SearchResult, error)

SearchMiddleware decorates document search. It receives the rest of the chain and may add, drop or reorder results.

type SearchNext added in v0.700.0 

type SearchNext func(ctx context.Context, query string, limit int, opts SearchOptions) ([]SearchResult, error)

SearchNext is the remaining search chain, ending in the store's own search.

type SearchOptions added in v0.413.0 

type SearchOptions struct {
	Tags            []string // filter results by tags (fuzzy match)
	SortByDate      bool     // sort results by updated_at descending
	ExcludeOutdated bool     // exclude documents where outdated = 1
	Scope           string   // filter by memory_scope prefix (empty = all)
	// PathPrefix restricts both search legs to documents whose file_path
	// starts with this prefix, pushed into the SQL as a bound
	// `AND d.file_path LIKE ? || '%' ESCAPE '\'` clause (PANDO-US-0028) rather
	// than filtered in Go after fusion — a prefix filtered after the fact would
	// under-return whenever the unfiltered top candidates all belong to another
	// prefix. Empty (the default) changes nothing about current behaviour.
	PathPrefix string
}

SearchOptions provides optional filtering/sorting for search queries.

type SearchResult 

type SearchResult struct {
	Document     Document
	ChunkContent string
	Score        float64
	Rank         int
}

SearchResult represents a ranked search result from the knowledge base.

Document.Content is unpopulated: see the field's doc comment on Document. Use ChunkContent for the matched text.

type SearchStats added in v0.715.6 

type SearchStats struct {
	// SkippedForDimensionMismatch counts chunks the vector leg skipped
	// because their recorded embedding dimension does not match the query
	// embedding's — evidence the document embedding model changed and some
	// chunks have not been re-embedded yet (PANDO-US-0029). Zero on a
	// consistent corpus.
	SkippedForDimensionMismatch int
}

SearchStats reports auxiliary counters from a search call that are not carried by the ranked results themselves.

type StaleEmbeddingStats added in v0.715.6 

type StaleEmbeddingStats struct {
	// Count is the number of embedded chunks whose recorded embedding_dims
	// differs from the configured dimension.
	Count int64
	// RecordedModels lists the distinct, non-empty embedding_model values
	// found among the mismatched chunks (sorted). A chunk written before
	// PANDO-US-0029 has an empty embedding_model — treated as "unknown", not
	// listed here — even though it still counts towards Count if its
	// backfilled dimension does not match.
	RecordedModels []string
}

StaleEmbeddingStats reports chunks whose recorded embedding dimension does not match a given (normally the currently configured) embedder dimension — evidence the document embedding model changed since those chunks were written (PANDO-US-0029).

type SyncStats added in v0.160.0 

type SyncStats struct {
	Scanned   int `json:"scanned"`
	Added     int `json:"added"`
	Updated   int `json:"updated"`
	Unchanged int `json:"unchanged"`
	Deleted   int `json:"deleted"`
	// LinksIndexed counts the [[wiki links]] found in the documents this run
	// added or updated. Unchanged documents keep the links they already had and
	// are not counted, so this is what the run wrote, not the size of the graph.
	LinksIndexed int `json:"links_indexed"`
}

SyncStats contains counters from a filesystem-to-KB synchronization run.

type WantedConcept added in v0.608.1 

type WantedConcept struct {
	// Slug is the normalized target.
	Slug string
	// Raw is the target as an author first wrote it, which reads better than the
	// slug when suggesting a title.
	Raw string
	// Count is how many documents point at it.
	Count int
	// Sources are the documents that point at it.
	Sources []string
}

WantedConcept is a link target that no document defines yet: the wiki equivalent of a red link. Frequently wanted concepts are the best candidates for the next document to write. 

type WikiLink struct {
	Raw      string
	Slug     string
	Label    string
	Position int
}

WikiLink is a [[target]] or [[target|label]] reference found in a document body.

Slug is the normalized form used for storage and resolution; Raw preserves the target exactly as the author wrote it. A link is not required to resolve to an existing document: unresolved targets are kept on purpose, since they record concepts that are worth documenting later. 

func ExtractWikiLinks(body string) []WikiLink

ExtractWikiLinks returns the wiki links found in a document body, in order of appearance and deduplicated by slug (the first occurrence wins). Links inside fenced code blocks and inline code spans are ignored, so documentation that merely shows the syntax does not pollute the graph.

type WriteEvent added in v0.700.0 

type WriteEvent struct {
	Kind      WriteKind
	Op        WriteOp
	FilePath  string
	Key       string
	Scope     string
	Content   string
	Tags      []string
	Metadata  map[string]any
	Origin    string
	Timestamp time.Time
}

WriteEvent describes one committed write. Observers see it only after the write succeeded: an observer must never be able to report something that did not happen.

type WriteKind added in v0.700.0 

type WriteKind string

WriteKind distinguishes a keyed memory from a plain knowledge-base document. 

const (
	WriteKindMemory   WriteKind = "memory"
	WriteKindDocument WriteKind = "document"
)

type WriteObserver added in v0.700.0 

type WriteObserver func(ctx context.Context, ev WriteEvent)

WriteObserver receives committed writes. It is called synchronously on the write path and must return promptly; whether the work behind it is async is the observer's decision, not the store's.

type WriteOp added in v0.700.0 

type WriteOp string

WriteOp is what happened to the record. 

const (
	WriteCreated WriteOp = "created"
	WriteUpdated WriteOp = "updated"
	WriteDeleted WriteOp = "deleted"
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.