Documentation
¶
Overview ¶
Package indexqueue assembles the shared background embedding queue (pkg/indexjobs) behind one Handle: the Postgres store, the Source/Sink registry, the worker/reaper/reconciler, the optional retention sweep and LISTEN/NOTIFY adapter, and every enabled consumer (api-catalog, tools, memory, prompts, portal assets/collections/knowledge-pages).
New takes an explicit Config: callers translate their own config into Config at the boundary and wire the returned Handle's Start/Stop into their own lifecycle. The package must not import pkg/platform. The tools source obtains the live in-process tool corpus through the injected ToolEnumerator.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// DB backs the queue store and every consumer sink. Required.
DB *sql.DB
// Embedder is the worker's embedding provider, already resolved by the
// caller (e.g. a dedicated longer-timeout Ollama provider). ModelName is the
// platform embedder's model name, recorded on each consumer sink so a model
// swap invalidates stale vectors.
Embedder embedding.Provider
ModelName string
// Worker tuning, already defaulted by the caller.
LeaseDuration time.Duration
BatchSize int
Workers int
// RetentionDays > 0 wires a retention sweep that bounds finished history;
// <= 0 disables it (the worker/reaper/reconciler still run, history is never
// purged).
RetentionDays int
// DSN enables the LISTEN/NOTIFY adapter when non-empty; empty falls back to
// the worker's poll tick.
DSN string
// CatalogStore, when non-nil, registers the api-catalog consumer and backs
// the admin view. ToolkitRegistry lets a successful api-catalog embed reload
// live api-gateway connections so their in-memory vector map picks up the
// new rows; may be nil.
CatalogStore apigatewaycatalog.Store
ToolkitRegistry *registry.Registry
// ToolEnumerator supplies the live, globally-visible tool corpus the tools
// source embeds. DiscoveryToolName is the discovery tool's own name, excluded
// from the corpus so a find-tools query never ranks the discovery tool itself.
ToolEnumerator ToolEnumerator
DiscoveryToolName string
// Consumers gates the optional DB-backed consumers.
Consumers Consumers
}
Config carries the values New needs to assemble the queue. Callers build it from their own config so this package stays free of platform config types.
type Consumers ¶
type Consumers struct {
Memory bool
Prompts bool
PortalAssets bool
PortalCollections bool
PortalKnowledgePages bool
}
Consumers gates the optional DB-backed consumers by the presence of their platform sub-store. Each consumer's Source/Sink needs only the queue's *sql.DB and the embedding model name to build, so the caller passes a boolean per consumer rather than threading the stores themselves through Config.
type Handle ¶
type Handle struct {
// contains filtered or unexported fields
}
Handle owns the assembled queue and its runtime goroutines. All components are constructed by New and driven by Start/Stop; the read accessors expose the admin view, cross-kind reporter, and tools vector store the platform's read paths consume.
The listener and retainer are nil when disabled (no DSN / retention off); the listener is additionally cleared to nil inside Start when the database role lacks LISTEN privilege, degrading to poll-only.
func New ¶
New assembles the queue from cfg: it builds the store and registry, registers every enabled consumer, and constructs the worker, reaper, reconciler, and — when enabled — the retention sweep and LISTEN adapter.
It returns nil (with no error) when no consumer registered: a worker with no consumers has nothing to do, so the caller wires nothing. The caller is responsible for the db-present and configured-embedder preconditions (#429); New trusts them.
func (*Handle) AdminStore ¶
func (h *Handle) AdminStore() catalogindex.Store
AdminStore returns the api-catalog admin view of the queue (enqueue + read-side queries for the UI), or nil when no api-catalog consumer is wired or the Handle is nil. Returned as the interface so callers stay off the concrete type.
func (*Handle) Registry ¶
Registry returns the source/sink registry the queue routes by source_kind, or nil on a nil Handle. It backs the Reporter and exposes the registered kinds (and their sources) for introspection.
func (*Handle) Reporter ¶
Reporter returns the cross-kind index-jobs reporter the admin Indexing dashboard reads (per-kind counts, coverage, job list, re-index), or nil on a nil Handle (no queue wired). The dashboard renders a degraded empty state for the nil case.
func (*Handle) Start ¶
Start launches the worker, reaper, reconciler, and (when enabled) the retention sweep and LISTEN adapter, then enqueues the initial tools index job. A LISTEN-privilege failure is non-fatal: the listener is cleared and the worker's poll tick takes over. It satisfies the lifecycle start signature so the caller can wire it directly.
func (*Handle) Stop ¶
Stop runs the index-jobs shutdown sequence inside the bounded shutdown helper. Each component's Stop signals its goroutines and blocks on their WaitGroup; boundedStop races the sequence against ctx.Done so shutdown always returns within its deadline. Abandoned work is safe: leases expire and another replica reclaims any uncompleted job on its next poll.
func (*Handle) ToolsIndexStore ¶
func (h *Handle) ToolsIndexStore() *toolsindex.Store
ToolsIndexStore returns the tools vector store the platform_find_tools semantic ranking reads, or nil when the tools consumer did not register or the Handle is nil.
type ToolEnumerator ¶
ToolEnumerator enumerates the live, globally-visible tool corpus the tools source embeds. The platform implements it over its in-process MCP server; the owner depends on this narrow interface instead of a *Platform back-reference, so the queue is constructible and testable without a platform.
Source Files