Documentation
¶
Overview ¶
Package scriptindex is the managed-script consumer of the shared indexjobs framework (#1370). It registers a Source/Sink pair under source_kind = "scripts" so scripts are embedded off the request path. Scripts were the only kind of knowledge the platform holds with no consumer here, which meant a script was found only when the words a caller typed matched the words its author wrote — hardest exactly where a script is most valuable, since a person looking for automation asks for what they want done rather than for the identifier somebody assigned it.
Like the prompt and memory consumers, and unlike api-catalog/tools, scripts store their vectors inline on the scripts table (one embedding per row), not in a dedicated vector table. So this package's Store reads and writes the embedding / embedding_model / embedding_text_hash columns of scripts directly: a script IS its own indexing unit. SourceID is the script id; each unit yields exactly one Item whose text is script.IndexText.
What is embedded is the script's description card and never its Starlark. docs/scripts/security.md admits the contract to the script's owner and the source to that owner and to administrators; one vector per row, stored inline, cannot be split along that line, so a vector built partly from source would let code a caller may not read decide how their results rank.
Every enabled script is indexed regardless of lifecycle status, mirroring prompts: ranked search decides visibility at query time — the store's Search applies the ownership predicate and the discoverable-status filter itself — so the index covers what any caller can rank. Gap detection and coverage filter on enabled only; a disabled script is never embedded and never counted as missing coverage.
Index ¶
- Constants
- type Sink
- func (s *Sink) Coverage(ctx context.Context) (indexjobs.Coverage, error)
- func (s *Sink) FindGaps(ctx context.Context) ([]string, error)
- func (*Sink) Kind() string
- func (s *Sink) ListExisting(ctx context.Context, key indexjobs.Key) (map[string]indexjobs.Vector, error)
- func (*Sink) StampExpected(context.Context, indexjobs.Key, int) error
- func (s *Sink) Upsert(ctx context.Context, key indexjobs.Key, rows []indexjobs.Vector) error
- func (s *Sink) UpsertBatch(ctx context.Context, key indexjobs.Key, rows []indexjobs.Vector) error
- type Source
- type Store
- func (s *Store) Coverage(ctx context.Context) (indexed, expected int, err error)
- func (s *Store) FindGaps(ctx context.Context, currentModel string) ([]string, error)
- func (s *Store) GetIndexText(ctx context.Context, id string) (string, error)
- func (s *Store) ListVectors(ctx context.Context, id string) (map[string]indexjobs.Vector, error)
- func (s *Store) UpsertVectors(ctx context.Context, id string, rows []indexjobs.Vector) error
Constants ¶
const SourceKind = "scripts"
SourceKind is the indexjobs source_kind this package serves.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Sink ¶
type Sink struct {
// contains filtered or unexported fields
}
Sink implements indexjobs.Sink for the scripts kind over the embedding columns of the scripts table. currentModel is the provider model the gap query diffs stored rows against, so a model swap re-embeds rows stamped with the previous model.
func NewSink ¶
NewSink returns a Sink backed by the given store. currentModel is the embedding provider's model identifier (embedding.ModelName); pass "" on a deployment whose provider does not name its model, in which case every row matches "" and only NULL-embedding rows are treated as gaps.
func (*Sink) Coverage ¶
Coverage reports the scripts kind's indexed-vs-expected totals (enabled scripts with an embedding vs all enabled scripts). ExpectedKnown is true: every enabled script is expected to converge to one vector.
func (*Sink) FindGaps ¶
FindGaps returns enabled script ids whose embedding is missing or was produced by a model other than the current one.
func (*Sink) ListExisting ¶
func (s *Sink) ListExisting(ctx context.Context, key indexjobs.Key) (map[string]indexjobs.Vector, error)
ListExisting returns the script's persisted vector keyed by item id for the worker's dedup pass.
func (*Sink) StampExpected ¶
StampExpected is a no-op for scripts. Gap detection is condition-based (embedding IS NULL OR model mismatch), not count-based, so there is no expected count to record per unit.
type Source ¶
type Source struct {
// contains filtered or unexported fields
}
Source implements indexjobs.Source for the scripts kind. A unit is one enabled script (SourceID = script id) and yields exactly one item: the script's composed embed text. The worker embeds it and the Sink writes the vector back onto the same row.
func (*Source) LoadItems ¶
LoadItems returns the script's single embeddable item. A script disabled or deleted between enqueue and claim yields an empty slice (a clean completion that writes no vector), per the Source contract.
func (*Source) OnSucceeded ¶
OnSucceeded is a no-op: the ranked search reads embeddings from the scripts table directly on every query, so there is no in-memory cache to refresh after a backfill writes a vector.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store reads and writes script embedding state on the scripts table for the indexjobs scripts consumer. It is intentionally separate from the request-path script store: it touches only the embedding columns (embedding, embedding_model, embedding_text_hash) and is scoped to the backfill path, so it does not widen the store contract manage_script and the portal are built on. The request-path store clears these columns when a script's indexed text changes; this Store writes them back.
func (*Store) Coverage ¶
Coverage returns the number of enabled scripts with an embedding (indexed) and the total number of enabled scripts (expected). Every enabled script is expected to carry a vector once converged.
func (*Store) FindGaps ¶
FindGaps returns the ids of enabled scripts whose embedding is missing or was produced by a model other than the current provider's. Missing embeddings cover a freshly created script and an edit that moved the indexed text (the request-path write clears the vector), and the model mismatch covers a provider model swap. Both converge off the request path when the reconciler enqueues them.
func (*Store) GetIndexText ¶
GetIndexText returns the composed embed text for an enabled script. Every enabled script is embedded regardless of status: search visibility is decided at query time (the store's own ownership predicate and discoverable-status filter), so the index must cover what any caller can rank. A script disabled or deleted between enqueue and claim yields errNotIndexable so the Source returns an empty item set (a clean "nothing to index" completion).
The composition is script.IndexText, the same one the discovery source shows a caller, and it reads only the description card: no source_code column is selected here, which is the storage-level expression of that rule.
func (*Store) ListVectors ¶
ListVectors returns the script's persisted embedding keyed by item id (the script id), for the worker's text-hash + model dedup pass. A script with no embedding yields an empty map, so the worker embeds it.
func (*Store) UpsertVectors ¶
UpsertVectors writes the embedding back onto the script. The script unit holds exactly one item (the script itself); a missing or empty row set is a no-op. updated_at is deliberately left untouched: a background embed is not a user-visible edit, so the script's "last modified" timestamp must not move — which matters more here than elsewhere, because the portal listing orders on it and a re-embed would otherwise reshuffle a person's scripts.
Source Files