Documentation
¶
Overview ¶
Package catalog models versioned, globally-owned OpenAPI spec bundles that api-gateway connections reference.
An API catalog represents the *API* (Salesforce REST, Google Drive, Stripe), not the credential pointing at it. A single Salesforce catalog backs both the sandbox and production connections in a deployment, so operators don't paste the same documentation N times to talk to N environments.
Each (name, version) pair is its own catalog row. Cloning to a new version creates a new row; existing connections stay on the old catalog until the operator explicitly migrates them. The model never sees catalogs directly — it queries connections, and the toolkit resolves connection → catalog → specs internally.
Index ¶
- Constants
- Variables
- func CountOperations(raw string) int
- func NormalizeBasePath(s string) (string, error)
- func ParseSpec(raw string) (*openapi3.T, error)
- func ValidateContent(raw string) error
- func ValidateID(s string) error
- func ValidateSourceKind(s string) error
- func ValidateSpecName(s string) error
- type Catalog
- type ConnectionRef
- type FetchOptions
- type FetchResult
- type MemoryStore
- func (s *MemoryStore) CreateCatalog(_ context.Context, c Catalog) error
- func (s *MemoryStore) DeleteCatalog(_ context.Context, id string) error
- func (s *MemoryStore) DeleteOperationEmbeddings(_ context.Context, catalogID, specName string) error
- func (s *MemoryStore) DeleteSpec(_ context.Context, catalogID, specName string) error
- func (s *MemoryStore) GetCatalog(_ context.Context, id string) (*Catalog, error)
- func (s *MemoryStore) GetSpec(_ context.Context, catalogID, specName string) (*SpecEntry, error)
- func (s *MemoryStore) ListCatalogs(_ context.Context) ([]Catalog, error)
- func (s *MemoryStore) ListOperationEmbeddings(_ context.Context, catalogID, specName string) ([]OperationEmbedding, error)
- func (s *MemoryStore) ListSpecs(_ context.Context, catalogID string) ([]SpecEntry, error)
- func (*MemoryStore) ReferencingConnections(_ context.Context, _ string) ([]ConnectionRef, error)
- func (s *MemoryStore) SetOperationCount(_ context.Context, catalogID, specName string, count int) error
- func (s *MemoryStore) UpdateCatalog(_ context.Context, id string, u Update) error
- func (s *MemoryStore) UpsertOperationEmbeddings(_ context.Context, catalogID, specName string, rows []OperationEmbedding) error
- func (s *MemoryStore) UpsertOperationEmbeddingsBatch(_ context.Context, catalogID, specName string, rows []OperationEmbedding) error
- func (s *MemoryStore) UpsertSpec(_ context.Context, catalogID string, spec SpecEntry) error
- type OperationEmbedding
- type PostgresStore
- func (s *PostgresStore) CreateCatalog(ctx context.Context, c Catalog) error
- func (s *PostgresStore) DeleteCatalog(ctx context.Context, id string) error
- func (s *PostgresStore) DeleteOperationEmbeddings(ctx context.Context, catalogID, specName string) error
- func (s *PostgresStore) DeleteSpec(ctx context.Context, catalogID, specName string) error
- func (s *PostgresStore) GetCatalog(ctx context.Context, id string) (*Catalog, error)
- func (s *PostgresStore) GetSpec(ctx context.Context, catalogID, specName string) (*SpecEntry, error)
- func (s *PostgresStore) ListCatalogs(ctx context.Context) ([]Catalog, error)
- func (s *PostgresStore) ListOperationEmbeddings(ctx context.Context, catalogID, specName string) ([]OperationEmbedding, error)
- func (s *PostgresStore) ListSpecs(ctx context.Context, catalogID string) ([]SpecEntry, error)
- func (s *PostgresStore) ReferencingConnections(ctx context.Context, catalogID string) ([]ConnectionRef, error)
- func (s *PostgresStore) SetOperationCount(ctx context.Context, catalogID, specName string, count int) error
- func (s *PostgresStore) UpdateCatalog(ctx context.Context, id string, u Update) error
- func (s *PostgresStore) UpsertOperationEmbeddings(ctx context.Context, catalogID, specName string, rows []OperationEmbedding) error
- func (s *PostgresStore) UpsertOperationEmbeddingsBatch(ctx context.Context, catalogID, specName string, rows []OperationEmbedding) error
- func (s *PostgresStore) UpsertSpec(ctx context.Context, catalogID string, spec SpecEntry) error
- type SpecEntry
- type Store
- type Update
Constants ¶
const ( SourceInline = "inline" SourceUpload = "upload" SourceURL = "url" )
SourceKind enumerates how a spec entered the system.
Variables ¶
var ErrConflict = errors.New("catalog: conflict")
ErrConflict is returned when a uniqueness invariant would be violated (duplicate (name, version) on a catalog, duplicate spec_name within a catalog).
var ErrInvalidBasePath = errors.New("catalog: invalid base_path")
ErrInvalidBasePath is returned when an operator-supplied per-spec base path fails validation. Exported so callers can map it to a 400 response without string-matching the error message.
var ErrInvalidContent = errors.New("catalog: spec failed OpenAPI 3.x validation")
ErrInvalidContent is returned when spec content fails OpenAPI 3.x parsing. The wrapped parser error carries the diagnostic line / path so admin UIs can surface it.
var ErrInvalidID = errors.New("catalog: invalid id")
ErrInvalidID is returned when a catalog ID does not match the slug shape required by the store. IDs are operator-supplied and immutable after creation, so we validate aggressively up front.
var ErrInvalidSpecName = errors.New("catalog: spec name must be lowercase letters, digits, hyphens, or underscores (1 to 64 chars, must start and end with a letter or digit)")
ErrInvalidSpecName is returned when a spec name doesn't match the component-slug shape. Spec names appear in MCP tool output (the `spec` field on OperationSummary) so we constrain them to a model-friendly subset. The message is operator-facing (it surfaces in the admin handler's 400 response) so it spells out the rule rather than just saying "invalid".
var ErrNotFound = errors.New("catalog: not found")
ErrNotFound is returned when a catalog or spec lookup misses.
var ErrSSRFBlocked = errors.New("catalog: SSRF guard blocked URL")
ErrSSRFBlocked is returned when a URL fails the SSRF guards. Wrapped errors describe which guard tripped (scheme, private IP, loopback, ...) so the admin handler can surface a precise message.
var ErrTooLarge = errors.New("catalog: response body exceeds size limit")
ErrTooLarge is returned when the response body exceeds MaxBytes.
var ErrUpstream = errors.New("catalog: upstream returned non-2xx status")
ErrUpstream is returned when the upstream HTTP status is not 2xx.
Functions ¶
func CountOperations ¶ added in v1.62.0
CountOperations parses raw and returns the total number of HTTP operations declared across every PathItem. The admin handler stores this on api_catalog_specs.operation_count so the embedding reconciler can compare it against persisted vector rows in pure SQL without re-parsing the spec content on every tick.
Returns 0 on parse failure (the admin write path validates content separately via ValidateContent; a count of 0 here is indistinguishable from an empty spec, which is the correct behavior for the reconciler).
func NormalizeBasePath ¶ added in v1.61.10
NormalizeBasePath validates and normalizes an operator-supplied SpecEntry.BasePath. Empty input returns empty output (the "no override" sentinel). Non-empty input is required to start with "/", must not contain CR/LF/NUL (header-smuggling vector when the path lands in a request line) and must not contain "?" or "#" (those terminate the path component of an URL). A trailing slash on a non-root value is stripped so the prepended segment joins cleanly with operation paths that all start with "/".
func ParseSpec ¶
ParseSpec loads raw as an OpenAPI 3.x document and validates it. External $ref resolution is disabled to prevent SSRF through a malicious spec that references private-network URLs at parse time. Returns ErrInvalidContent on any parse or validation failure. Single source of truth for spec parsing across the admin handler (validating an upload), the toolkit (materializing a connection's catalog), and tests.
Three categories of strict-validation checks are disabled to match what production OpenAPI consumers (Swagger UI, Postman, Insomnia) accept. The structural validation that matters for invocation (operation IDs, path templating, parameter shapes, security scheme references, request and response schemas) still runs. The disabled checks are:
- Example-vs-schema conformance. Vendor specs routinely declare a property as one type but include richer examples (e.g. `value: object` with `"Blue"` as an example). Examples are documentation hints, not part of the wire contract.
- Schema patterns that use ECMA regex constructs Go's regexp engine does not support (lookahead, named backrefs).
- Default-value-vs-schema conformance: same drift pattern as examples, same documentation-only role.
func ValidateContent ¶
ValidateContent is a wrapper around ParseSpec for callers that only need to assert validity (e.g. the admin upload route).
func ValidateID ¶
ValidateID reports whether s is acceptable as a catalog ID. Returns ErrInvalidID on failure so callers can wrap it without constructing a fresh sentinel.
func ValidateSourceKind ¶
ValidateSourceKind reports whether s is one of the three known source kinds. Returns nil on match, an error otherwise.
func ValidateSpecName ¶
ValidateSpecName reports whether s is acceptable as a component spec name within a catalog.
Types ¶
type Catalog ¶
type Catalog struct {
ID string
Name string
Version string
DisplayName string
Description string
CreatedBy string
CreatedAt time.Time
UpdatedAt time.Time
}
Catalog is the header row in api_catalogs. The (Name, Version) pair is unique across the table; (ID) is the immutable handle connections reference. ID is operator-chosen at create and never changes — editing the catalog's display fields preserves it so downstream references stay valid.
type ConnectionRef ¶
ConnectionRef identifies a connection_instances row by its composite key.
type FetchOptions ¶
type FetchOptions struct {
// MaxBytes caps the downloaded body. Zero means use defaultMaxBytes.
MaxBytes int64
// ConnectTimeout caps DNS + TCP + TLS handshake. Zero means use defaults.
ConnectTimeout time.Duration
// TotalTimeout caps the entire fetch (dial + headers + body read).
TotalTimeout time.Duration
// AllowInsecureScheme allows http:// alongside https://. Production
// callers must NOT set this; it exists for the test suite, which
// runs httptest.NewServer in plain http on a loopback.
AllowInsecureScheme bool
// AllowPrivateNetworks disables the private-IP SSRF guard. Same
// constraint as AllowInsecureScheme: never set in production.
AllowPrivateNetworks bool
// Resolver, when set, replaces net.DefaultResolver for the host
// lookup. Used by tests to control DNS without spinning up a real
// resolver.
Resolver hostResolver
// HTTPClient, when set, replaces the SSRF-guarded client. Tests
// substitute httptest.Client to terminate TLS / loopback without
// fighting the dialer's private-IP guard.
HTTPClient *http.Client
}
FetchOptions controls the URL-fetch step used by the admin layer when an operator picks "URL" as the spec source. Zero-values are safe defaults; tests inject smaller limits and a stubbed resolver.
type FetchResult ¶
FetchResult is what FetchFromURL returns on success.
func FetchFromURL ¶
func FetchFromURL(ctx context.Context, urlStr string, opts FetchOptions) (*FetchResult, error)
FetchFromURL downloads an OpenAPI spec from urlStr, enforcing the SSRF guards. Returns the raw content plus the captured ETag (empty if the server didn't send one). The caller (admin handler) is responsible for passing the content through parseOpenAPISpec in the apigateway package before persisting — keeping the validation step there avoids a circular import.
type MemoryStore ¶
type MemoryStore struct {
// contains filtered or unexported fields
}
MemoryStore is an in-memory Store implementation. Used by tests to avoid spinning up Postgres for table-driven cases that only exercise the Store contract. Safe for concurrent use.
MemoryStore does NOT track connection references — calls to ReferencingConnections always return nil. The admin handler that uses ReferencingConnections to gate catalog deletion is exercised against the Postgres implementation in integration tests, which has the real cross-table query.
func NewMemoryStore ¶
func NewMemoryStore() *MemoryStore
NewMemoryStore returns an empty in-memory Store.
func (*MemoryStore) CreateCatalog ¶
func (s *MemoryStore) CreateCatalog(_ context.Context, c Catalog) error
CreateCatalog adds a new catalog. Returns ErrInvalidID when the id fails the slug check or ErrConflict when (id) or (name, version) is already taken.
func (*MemoryStore) DeleteCatalog ¶
func (s *MemoryStore) DeleteCatalog(_ context.Context, id string) error
DeleteCatalog removes the catalog and all its specs (CASCADE behavior matches the Postgres FK). Operation embeddings keyed on (catalog_id, spec_name) are dropped at the same time so the in-memory backend matches the Postgres ON DELETE CASCADE chain (api_catalogs → api_catalog_specs → api_catalog_operation_embeddings).
func (*MemoryStore) DeleteOperationEmbeddings ¶ added in v1.61.11
func (s *MemoryStore) DeleteOperationEmbeddings(_ context.Context, catalogID, specName string) error
DeleteOperationEmbeddings removes every embedding row for the (catalogID, specName) pair. No-op when no rows exist.
func (*MemoryStore) DeleteSpec ¶
func (s *MemoryStore) DeleteSpec(_ context.Context, catalogID, specName string) error
DeleteSpec removes one spec from the catalog. Associated embedding rows are dropped at the same time so the in-memory backend mirrors the Postgres FK CASCADE from api_catalog_specs down to api_catalog_operation_embeddings.
func (*MemoryStore) GetCatalog ¶
GetCatalog returns the catalog by id or ErrNotFound.
func (*MemoryStore) ListCatalogs ¶
func (s *MemoryStore) ListCatalogs(_ context.Context) ([]Catalog, error)
ListCatalogs returns every catalog sorted by (name, version, id).
func (*MemoryStore) ListOperationEmbeddings ¶ added in v1.61.11
func (s *MemoryStore) ListOperationEmbeddings(_ context.Context, catalogID, specName string) ([]OperationEmbedding, error)
ListOperationEmbeddings returns every embedding row for the (catalogID, specName) pair. Empty slice (not error) when the spec has no vectors yet.
func (*MemoryStore) ReferencingConnections ¶
func (*MemoryStore) ReferencingConnections(_ context.Context, _ string) ([]ConnectionRef, error)
ReferencingConnections always returns nil — the in-memory store doesn't know about connection_instances. Use Postgres in production / integration tests where this matters.
func (*MemoryStore) SetOperationCount ¶ added in v1.62.0
func (s *MemoryStore) SetOperationCount(_ context.Context, catalogID, specName string, count int) error
SetOperationCount updates the operation_count on one spec row. Returns ErrNotFound when (catalogID, specName) does not exist. Mirrors the Postgres backend's behavior so the embedjobs worker tests can run against either store.
func (*MemoryStore) UpdateCatalog ¶
UpdateCatalog applies the partial update. Returns ErrNotFound when id is unknown, ErrConflict when the edit would collide on (name, version).
func (*MemoryStore) UpsertOperationEmbeddings ¶ added in v1.61.11
func (s *MemoryStore) UpsertOperationEmbeddings(_ context.Context, catalogID, specName string, rows []OperationEmbedding) error
UpsertOperationEmbeddings replaces every embedding row for the given (catalogID, specName) pair. The MemoryStore mirrors the Postgres backend's all-or-nothing semantics by clearing the existing rows before writing the new set.
func (*MemoryStore) UpsertOperationEmbeddingsBatch ¶ added in v1.67.1
func (s *MemoryStore) UpsertOperationEmbeddingsBatch(_ context.Context, catalogID, specName string, rows []OperationEmbedding) error
UpsertOperationEmbeddingsBatch inserts or updates the supplied rows in place. Unlike UpsertOperationEmbeddings, it does not delete absent rows: prior embeddings for operations not in rows survive untouched. Mirrors the Postgres backend's per-batch path used by the embed-jobs worker for incremental persistence across chunks.
func (*MemoryStore) UpsertSpec ¶
UpsertSpec inserts or replaces a spec entry. Returns ErrNotFound when catalog_id is unknown.
type OperationEmbedding ¶ added in v1.61.11
type OperationEmbedding struct {
OperationID string
TextHash []byte
Embedding []float32
Model string
Dim int
}
OperationEmbedding is one persisted embedding row. OperationID is the synthesized id buildOperationIndex assigns to each path/ method pair (the spec's operationId when set, "METHOD path" otherwise). TextHash is the SHA-256 of the source text fed to the embedding provider — used at upsert time to skip recomputing vectors whose text did not change between two spec refreshes. Model and Dim record the provider identity and vector dimensionality at write time so a deployment that swaps models has a row-level breadcrumb that the cached vectors no longer match the current provider's output.
type PostgresStore ¶
type PostgresStore struct {
// contains filtered or unexported fields
}
PostgresStore implements Store against PostgreSQL.
func NewPostgresStore ¶
func NewPostgresStore(db *sql.DB) *PostgresStore
NewPostgresStore returns a Store backed by the given *sql.DB. The caller owns the connection lifecycle; Close on the DB after the store is no longer in use.
func (*PostgresStore) CreateCatalog ¶
func (s *PostgresStore) CreateCatalog(ctx context.Context, c Catalog) error
CreateCatalog inserts a new catalog header. Returns ErrConflict when the ID already exists or (name, version) collides.
func (*PostgresStore) DeleteCatalog ¶
func (s *PostgresStore) DeleteCatalog(ctx context.Context, id string) error
DeleteCatalog removes the catalog and (via ON DELETE CASCADE) all of its specs. Returns ErrNotFound when no row matches. Callers are expected to consult ReferencingConnections first; this method does not check for connection references.
func (*PostgresStore) DeleteOperationEmbeddings ¶ added in v1.61.11
func (s *PostgresStore) DeleteOperationEmbeddings(ctx context.Context, catalogID, specName string) error
DeleteOperationEmbeddings removes every embedding row for the (catalogID, specName) pair. Used by the reembed admin endpoint before recomputing. Spec deletion does not need to call this — the FK ON DELETE CASCADE drops the rows automatically.
func (*PostgresStore) DeleteSpec ¶
func (s *PostgresStore) DeleteSpec(ctx context.Context, catalogID, specName string) error
DeleteSpec removes one component spec from a catalog. Returns ErrNotFound when (catalog_id, spec_name) has no row.
func (*PostgresStore) GetCatalog ¶
GetCatalog returns the catalog by ID or ErrNotFound.
func (*PostgresStore) GetSpec ¶
func (s *PostgresStore) GetSpec(ctx context.Context, catalogID, specName string) (*SpecEntry, error)
GetSpec returns the named spec from the catalog or ErrNotFound.
func (*PostgresStore) ListCatalogs ¶
func (s *PostgresStore) ListCatalogs(ctx context.Context) ([]Catalog, error)
ListCatalogs returns every catalog sorted by (name, version). Sort is stable so the portal list view doesn't reshuffle on each refresh.
func (*PostgresStore) ListOperationEmbeddings ¶ added in v1.61.11
func (s *PostgresStore) ListOperationEmbeddings(ctx context.Context, catalogID, specName string) ([]OperationEmbedding, error)
ListOperationEmbeddings returns every embedding row for the (catalogID, specName) pair. Empty slice (not error) when the spec has not been embedded yet — the caller surfaces this as "fall back to lexical, vectors not yet computed".
func (*PostgresStore) ListSpecs ¶
ListSpecs returns every component spec in the catalog, sorted by spec_name. Empty slice (not error) when the catalog has no specs yet.
func (*PostgresStore) ReferencingConnections ¶
func (s *PostgresStore) ReferencingConnections(ctx context.Context, catalogID string) ([]ConnectionRef, error)
ReferencingConnections scans connection_instances for any row whose config JSONB has `catalog_id` equal to the given id. Sorted by (kind, name) for stable output to the admin handler's "still referenced by" error.
func (*PostgresStore) SetOperationCount ¶ added in v1.62.0
func (s *PostgresStore) SetOperationCount(ctx context.Context, catalogID, specName string, count int) error
SetOperationCount updates the operation_count column on one spec row. Called by the embedding worker after a successful Upsert so the reconciler's "operation_count <> embedded" predicate sees a fully-indexed spec.
Returns ErrNotFound when no row matches. The worker treats that as a best-effort failure (the spec may have been deleted between claim and completion); other errors are wrapped.
func (*PostgresStore) UpdateCatalog ¶
UpdateCatalog applies a partial edit. Nil fields in u are skipped. Returns ErrNotFound when no row matches the id, ErrConflict when the edit would violate (name, version) uniqueness.
func (*PostgresStore) UpsertOperationEmbeddings ¶ added in v1.61.11
func (s *PostgresStore) UpsertOperationEmbeddings(ctx context.Context, catalogID, specName string, rows []OperationEmbedding) error
UpsertOperationEmbeddings replaces every embedding row for (catalogID, specName) with the supplied rows in a single transaction. Atomic: ranking reads either see the prior set or the new set, never a partial mix. Returns ErrNotFound when the referenced spec does not exist (FK violation).
func (*PostgresStore) UpsertOperationEmbeddingsBatch ¶ added in v1.67.1
func (s *PostgresStore) UpsertOperationEmbeddingsBatch(ctx context.Context, catalogID, specName string, rows []OperationEmbedding) error
UpsertOperationEmbeddingsBatch inserts or updates the supplied rows without disturbing rows outside the batch. The embed-jobs worker calls this per chunk so a job that fails on chunk N leaves chunks 0..N-1 persisted for the next attempt's dedup pass. Atomic per call: the upsert runs in one transaction so a ranking read either sees the prior set or the prior set plus this chunk, never a half-written chunk. Returns ErrNotFound on FK violation (spec was deleted between dispatch and write).
func (*PostgresStore) UpsertSpec ¶
UpsertSpec inserts or replaces a (catalog_id, spec_name) row. Returns ErrNotFound when catalog_id has no matching catalog (translated from the FK violation).
type SpecEntry ¶
type SpecEntry struct {
SpecName string
Content string
SourceKind string
SourceURL string
ETag string
BasePath string
LastFetchedAt time.Time
CreatedAt time.Time
UpdatedAt time.Time
// OperationCount is the number of operations the spec
// content parses to. The admin handler sets this on every
// write so the embedding-job reconciler can compare it
// against the row count in api_catalog_operation_embeddings
// to detect gaps in pure SQL. A value of 0 means either an
// empty spec or a spec that has not been re-saved since
// migration 000045 added the column; the reconciler treats
// both the same way (no work to enqueue when the embedding
// row count is also 0).
OperationCount int
}
SpecEntry is a single component OpenAPI document inside a catalog. Content is plain text (YAML or JSON); the toolkit parses it at connection-load time. SourceURL/ETag/LastFetchedAt populate when SourceKind == SourceURL so the portal can offer a "Refresh" action.
BasePath is the operator-supplied override for the URL path segment prepended to every operation in this spec when the connection invokes the upstream. Empty means "no override"; the toolkit falls back to deriving the prefix from servers[0].url in the spec content. Set this when the spec ships without a servers[] entry, or when the operator's deployment targets a path that does not match what the spec author wrote (sandbox, proxy, version pin). Must start with "/" when non-empty; trailing slash is stripped at validation time.
type Store ¶
type Store interface {
// Catalog header CRUD.
CreateCatalog(ctx context.Context, c Catalog) error
GetCatalog(ctx context.Context, id string) (*Catalog, error)
ListCatalogs(ctx context.Context) ([]Catalog, error)
UpdateCatalog(ctx context.Context, id string, u Update) error
DeleteCatalog(ctx context.Context, id string) error
// Component specs within a catalog. UpsertSpec creates the row
// when (catalog_id, spec_name) is new and replaces content +
// source metadata when it already exists. The portal "Edit"
// flow uses the same write path as "Add".
UpsertSpec(ctx context.Context, catalogID string, spec SpecEntry) error
GetSpec(ctx context.Context, catalogID, specName string) (*SpecEntry, error)
ListSpecs(ctx context.Context, catalogID string) ([]SpecEntry, error)
DeleteSpec(ctx context.Context, catalogID, specName string) error
// ReferencingConnections returns the (kind, name) of every
// connection_instances row whose config JSONB has
// catalog_id == catalogID. The admin handler calls this before
// DeleteCatalog to refuse with a clear "still referenced by ..."
// error instead of failing the FK at delete time. (There is no
// SQL FK from connection_instances → api_catalogs because the
// reference lives inside the JSONB, not in its own column.)
ReferencingConnections(ctx context.Context, catalogID string) ([]ConnectionRef, error)
// UpsertOperationEmbeddings replaces every embedding row for
// (catalog_id, spec_name) with the supplied vectors. Atomic —
// either all rows for the spec are present afterward or none
// are (no partial state visible to ranking reads). Used by
// catalog_handler at spec-write time so semantic ranking can
// read pre-computed vectors at connection registration without
// hitting the embedding provider on the request path.
UpsertOperationEmbeddings(ctx context.Context, catalogID, specName string, rows []OperationEmbedding) error
// UpsertOperationEmbeddingsBatch inserts or updates the
// supplied rows in place. Unlike UpsertOperationEmbeddings,
// this method does NOT delete absent rows: rows for operations
// not in the batch survive untouched. Used by the embed-jobs
// worker for per-chunk incremental persistence so a job that
// fails on chunk N leaves chunks 0..N-1 visible to the next
// attempt's dedup pass via ListOperationEmbeddings.
//
// Atomic per call: the upsert runs inside one transaction so
// a concurrent ranking read either sees the prior set or the
// prior set plus this chunk, never a half-written chunk.
// Returns ErrNotFound when the referenced spec does not
// exist (FK violation on insert).
UpsertOperationEmbeddingsBatch(ctx context.Context, catalogID, specName string, rows []OperationEmbedding) error
// ListOperationEmbeddings returns every embedding row for
// (catalog_id, spec_name) so the toolkit can populate its
// per-connection vector map at registration time without
// re-embedding. Empty slice (not error) when the spec has no
// vectors yet.
ListOperationEmbeddings(ctx context.Context, catalogID, specName string) ([]OperationEmbedding, error)
// DeleteOperationEmbeddings removes every embedding row for
// (catalog_id, spec_name). Called by the admin reembed
// endpoint before recomputing — the spec FK cascade handles
// spec-deletion cleanup so callers do not need to invoke this
// separately on spec delete.
DeleteOperationEmbeddings(ctx context.Context, catalogID, specName string) error
// SetOperationCount updates api_catalog_specs.operation_count
// on a single row. The embedding worker calls this after a
// successful Upsert so the reconciler's gap predicate (which
// compares operation_count against the embedding row count)
// sees a fully-indexed spec on the next sweep. Without this
// hook, specs whose operation_count column was never stamped
// (legacy rows from before migration 000045) would re-enqueue
// on every reconciler tick forever.
//
// Returns ErrNotFound when (catalogID, specName) does not
// exist. The worker treats that as best-effort and logs only.
SetOperationCount(ctx context.Context, catalogID, specName string, count int) error
}
Store is the persistence interface for catalogs and their component specs. Implementations are expected to enforce the (name, version) uniqueness and the spec-name uniqueness within a catalog at the storage layer; ValidateID / ValidateSpecName / ValidateSourceKind handle input shape upstream.