Documentation
¶
Overview ¶
Package knowledgepageindex is the knowledge-page consumer of the shared indexjobs framework (#633). It registers a Source/Sink pair under source_kind = "portal-knowledge-pages" so the reconciler embeds canonical knowledge pages off the request path: a newly created page (whose vector is still NULL) or one left stale by a content edit or a provider model swap self-heals on the next sweep.
Like the asset, prompt, and memory consumers, pages store their vectors inline on the portal_knowledge_pages table (one embedding per row), not in a dedicated vector table. So this package's Store reads and writes the embedding / embedding_model / embedding_text_hash columns directly: a page IS its own indexing unit. SourceID is the page id; each unit yields exactly one Item whose text is portal.KnowledgePageIndexText (title + body + tags). The body is indexed (unlike assets, whose body lives unindexed in S3), so page CONTENT is semantically searchable.
Only non-deleted pages are indexed. Gap detection and coverage both filter on deleted_at IS NULL, so a soft-deleted page is never embedded and never counted as missing coverage.
Index ¶
- Constants
- func RegisterConsumer(reg interface{ ... }, db *sql.DB, currentModel string) error
- type Sink
- func (s *Sink) Coverage(ctx context.Context) (indexjobs.Coverage, error)
- func (s *Sink) FindGaps(ctx context.Context) ([]string, error)
- func (*Sink) Kind() string
- func (s *Sink) ListExisting(ctx context.Context, key indexjobs.Key) (map[string]indexjobs.Vector, error)
- func (*Sink) StampExpected(context.Context, indexjobs.Key, int) error
- func (s *Sink) Upsert(ctx context.Context, key indexjobs.Key, rows []indexjobs.Vector) error
- func (s *Sink) UpsertBatch(ctx context.Context, key indexjobs.Key, rows []indexjobs.Vector) error
- type Source
- type Store
- func (s *Store) Coverage(ctx context.Context) (indexed, expected int, err error)
- func (s *Store) FindGaps(ctx context.Context, currentModel string) ([]string, error)
- func (s *Store) GetIndexText(ctx context.Context, id string) (string, error)
- func (s *Store) ListVectors(ctx context.Context, id string) (map[string]indexjobs.Vector, error)
- func (s *Store) UpsertVectors(ctx context.Context, id string, rows []indexjobs.Vector) error
Constants ¶
const SourceKind = "portal-knowledge-pages"
SourceKind is the indexjobs source_kind this package serves.
Variables ¶
This section is empty.
Functions ¶
func RegisterConsumer ¶
func RegisterConsumer(reg interface {
Register(indexjobs.Source, indexjobs.Sink) error
}, db *sql.DB, currentModel string,
) error
RegisterConsumer registers the knowledge-pages Source/Sink pair on the shared indexjobs registry. Keeping the wiring here (rather than inline in the platform) keeps the platform package thin. currentModel is the embedding provider's model identifier.
Types ¶
type Sink ¶
type Sink struct {
// contains filtered or unexported fields
}
Sink implements indexjobs.Sink for the portal-knowledge-pages kind over the embedding columns of the portal_knowledge_pages table. currentModel is the provider model the gap query diffs stored rows against, so a model swap re-embeds rows stamped with the previous model.
func NewSink ¶
NewSink returns a Sink backed by the given store. currentModel is the embedding provider's model identifier (embedding.ModelName); pass "" on a deployment whose provider does not name its model, in which case only NULL-embedding rows are treated as gaps.
func (*Sink) Coverage ¶
Coverage reports the portal-knowledge-pages kind's indexed-vs-expected totals (non-deleted pages with an embedding vs all non-deleted pages). ExpectedKnown is true: every non-deleted page is expected to converge to one vector.
func (*Sink) FindGaps ¶
FindGaps returns non-deleted page ids whose embedding is missing or was produced by a model other than the current one.
func (*Sink) ListExisting ¶
func (s *Sink) ListExisting(ctx context.Context, key indexjobs.Key) (map[string]indexjobs.Vector, error)
ListExisting returns the page's persisted vector keyed by item id for the worker's dedup pass.
func (*Sink) StampExpected ¶
StampExpected is a no-op for pages. Gap detection is condition-based (embedding IS NULL OR model mismatch), not count-based.
type Source ¶
type Source struct {
// contains filtered or unexported fields
}
Source implements indexjobs.Source for the portal-knowledge-pages kind. A unit is one page (SourceID = page id) and yields exactly one item: the page's composed embed text. The worker embeds it and the Sink writes the vector back onto the same row.
func (*Source) LoadItems ¶
LoadItems returns the page's single embeddable item. A page soft-deleted between enqueue and claim yields an empty slice (a clean completion that writes no vector), per the Source contract.
func (*Source) OnSucceeded ¶
OnSucceeded is a no-op: the ranked search reads embeddings from the portal_knowledge_pages table directly on every query, so there is no in-memory cache to refresh after a backfill writes a vector.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store reads and writes knowledge-page embedding state on the portal_knowledge_pages table for the indexjobs knowledge-pages consumer. It is intentionally separate from portal.KnowledgePageStore: it touches only the embedding columns and is scoped to the backfill path, so it does not widen the request-path store contract. The request-path Create/Update clears these columns when a page's indexed text changes; this Store writes them back.
func (*Store) Coverage ¶
Coverage returns the number of non-deleted pages with an embedding (indexed) and the total number of non-deleted pages (expected).
func (*Store) FindGaps ¶
FindGaps returns the ids of non-deleted pages whose embedding is missing or was produced by a model other than the current provider's.
func (*Store) GetIndexText ¶
GetIndexText returns the composed embed text for a non-deleted page. A page soft-deleted between enqueue and claim yields errNotIndexable so the Source returns an empty item set. The composition is knowledgepage.IndexText, the same one the request-path search ranks against.
func (*Store) ListVectors ¶
ListVectors returns the page's persisted embedding keyed by item id (the page id), for the worker's text-hash + model dedup pass. A page with no embedding yields an empty map, so the worker embeds it.
func (*Store) UpsertVectors ¶
UpsertVectors writes the embedding back onto the page. The page unit holds exactly one item; a missing or empty row set is a no-op. updated_at is left untouched: a background embed is not a user-visible edit, so the page's "last modified" timestamp must not move.
Source Files