Documentation
¶
Overview ¶
Package assetindex is the saved-asset consumer of the shared indexjobs framework (#550). It registers a Source/Sink pair under source_kind = "portal-assets" so saved assets are embedded off the request path. A newly saved asset, and one whose name/description/tags changed, enqueue their own job at write time (#1256); the reconciler is the backstop for those a write could not produce and for the corpus a provider model swap invalidates.
Like the prompt and memory consumers, assets store their vectors inline on the portal_assets table (one embedding per row), not in a dedicated vector table. So this package's Store reads and writes the embedding / embedding_model / embedding_text_hash columns of portal_assets directly: an asset IS its own indexing unit. SourceID is the asset id; each unit yields exactly one Item whose text is portal.AssetIndexText (name + description + tags).
Only non-deleted assets are indexed. Gap detection and coverage both filter on deleted_at IS NULL, so a soft-deleted asset is never embedded and never counted as missing coverage.
Index ¶
- Constants
- type Sink
- func (s *Sink) Coverage(ctx context.Context) (indexjobs.Coverage, error)
- func (s *Sink) FindGaps(ctx context.Context) ([]string, error)
- func (*Sink) Kind() string
- func (s *Sink) ListExisting(ctx context.Context, key indexjobs.Key) (map[string]indexjobs.Vector, error)
- func (*Sink) StampExpected(context.Context, indexjobs.Key, int) error
- func (s *Sink) Upsert(ctx context.Context, key indexjobs.Key, rows []indexjobs.Vector) error
- func (s *Sink) UpsertBatch(ctx context.Context, key indexjobs.Key, rows []indexjobs.Vector) error
- type Source
- type Store
- func (s *Store) Coverage(ctx context.Context) (indexed, expected int, err error)
- func (s *Store) FindGaps(ctx context.Context, currentModel string) ([]string, error)
- func (s *Store) GetIndexText(ctx context.Context, id string) (string, error)
- func (s *Store) ListVectors(ctx context.Context, id string) (map[string]indexjobs.Vector, error)
- func (s *Store) UpsertVectors(ctx context.Context, id string, rows []indexjobs.Vector) error
Constants ¶
const SourceKind = "portal-assets"
SourceKind is the indexjobs source_kind this package serves.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Sink ¶
type Sink struct {
// contains filtered or unexported fields
}
Sink implements indexjobs.Sink for the portal-assets kind over the embedding columns of the portal_assets table. currentModel is the provider model the gap query diffs stored rows against, so a model swap re-embeds rows stamped with the previous model.
func NewSink ¶
NewSink returns a Sink backed by the given store. currentModel is the embedding provider's model identifier (embedding.ModelName); pass "" on a deployment whose provider does not name its model, in which case every row matches "" and only NULL-embedding rows are treated as gaps.
func (*Sink) Coverage ¶
Coverage reports the portal-assets kind's indexed-vs-expected totals (non-deleted assets with an embedding vs all non-deleted assets). ExpectedKnown is true: every non-deleted asset is expected to converge to one vector.
func (*Sink) FindGaps ¶
FindGaps returns non-deleted asset ids whose embedding is missing or was produced by a model other than the current one.
func (*Sink) ListExisting ¶
func (s *Sink) ListExisting(ctx context.Context, key indexjobs.Key) (map[string]indexjobs.Vector, error)
ListExisting returns the asset's persisted vector keyed by item id for the worker's dedup pass.
func (*Sink) StampExpected ¶
StampExpected is a no-op for assets. Gap detection is condition-based (embedding IS NULL OR model mismatch), not count-based, so there is no expected count to record per unit.
type Source ¶
type Source struct {
// contains filtered or unexported fields
}
Source implements indexjobs.Source for the portal-assets kind. A unit is one saved asset (SourceID = asset id) and yields exactly one item: the asset's composed embed text. The worker embeds it and the Sink writes the vector back onto the same row.
func (*Source) LoadItems ¶
LoadItems returns the asset's single embeddable item. An asset soft-deleted between enqueue and claim yields an empty slice (a clean completion that writes no vector), per the Source contract.
func (*Source) OnSucceeded ¶
OnSucceeded is a no-op: the ranked search reads embeddings from the portal_assets table directly on every query, so there is no in-memory cache to refresh after a backfill writes a vector.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store reads and writes asset embedding state on the portal_assets table for the indexjobs assets consumer. It is intentionally separate from portal.AssetStore: it touches only the embedding columns (embedding, embedding_model, embedding_text_hash) and is scoped to the backfill path, so it does not widen the request-path store contract. The request-path Update clears these columns when an asset's indexed text changes; this Store writes them back.
func (*Store) Coverage ¶
Coverage returns the number of non-deleted assets with an embedding (indexed) and the total number of non-deleted assets (expected).
func (*Store) FindGaps ¶
FindGaps returns the ids of non-deleted assets whose embedding is missing or was produced by a model other than the current provider's. Missing embeddings cover a freshly saved asset (and a name/description/tags edit, which the request-path Update clears the embedding for); the model mismatch covers a provider model swap.
func (*Store) GetIndexText ¶
GetIndexText returns the composed embed text for a non-deleted asset. An asset soft-deleted between enqueue and claim yields errNotIndexable so the Source returns an empty item set (a clean "nothing to index" completion). The composition is portal.AssetIndexText, the same one the request-path search ranks against.
func (*Store) ListVectors ¶
ListVectors returns the asset's persisted embedding keyed by item id (the asset id), for the worker's text-hash + model dedup pass. An asset with no embedding yields an empty map, so the worker embeds it.
func (*Store) UpsertVectors ¶
UpsertVectors writes the embedding back onto the asset. The asset unit holds exactly one item; a missing or empty row set is a no-op. updated_at is deliberately left untouched: a background embed is not a user-visible edit, so the asset's "last modified" timestamp must not move.
Source Files