store

package
v0.3.42 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 29, 2026 License: MIT Imports: 35 Imported by: 0

Documentation

Overview

Package store is the entity-storage catalog provider (design 034).

It is the engine's catalog storage. The storage it replaced shredded a FULLY compiled GraphQL schema into the _schema_* tables; this one stores only the BASE entities that arrive from sources (modules, data objects, their fields, functions, source-defined types and relations) in the CoreDB `catalog` schema, and reconstructs the served GraphQL schema on the fly at read time.

The served schema is composed of three layers:

  • the SYSTEM layer — scalars, introspection (__*), @system types and the base/query/GIS directives. These are assembled at startup from the binary (static.New, same set the old provider persisted under the "_system" catalog) and held in memory. They are NEVER written to catalog.*: the writer skips sdl.IsSystemType and the reader serves them from the static layer.

  • the SOURCE entities — reconstructed from catalog.* on demand (ForName / Types synthesis). Data objects, source types, functions.

  • the SYNTHESIZED roots and derived types — Query/Mutation/Subscription and per-module roots (never stored, always assembled), plus the derived helpers (_X_aggregation, filters, …) generated lazily by name.

Write path (a partial compiler: prefix + validate + extension merge, NO type generation) → collect base entities → Reconcile into catalog.*. Read path is cached with per-data-source invalidation (schemaCache): a reconcile evicts only the changed/removed sources.

All SQL addresses the tables literally as `core.catalog.<table>`: the CoreDB is always attached as `core` (production and tests alike, via the coredb Source) and DML runs from the DuckDB context on both backends (implicit VARCHAR→STRUCT casts on DuckDB, INOUT assignment casts on Postgres), so the prefix is a constant and the statement texts match core-db's pinned probeCatalogStatements verbatim.

Index

Constants

This section is empty.

Variables

View Source 
var ErrNoEmbeddings = errors.New("embeddings not configured (EMBEDDER_URL not set)")

ErrNoEmbeddings is returned by ReindexEmbeddings when the engine runs without an embedder — there are no vectors to recompute.

View Source 
var ErrReadOnly = errors.New("catalog store: read-only mode")

ErrReadOnly is returned by every write entry point when the store was configured read-only (cluster workers read the catalog the writer node populated).

Functions

func AggRefArgs 

func AggRefArgs(filterName string, pos *ast.Position) ast.ArgumentDefinitionList

AggRefArgs returns reference field args on aggregation types: filter + order_by + distinct_on + inner + nested_*.

func AggSubRefArgs 

func AggSubRefArgs(filterName string, pos *ast.Position) ast.ArgumentDefinitionList

AggSubRefArgs returns args for the _aggregation sub-field on aggregation types. Includes filter + order_by + limit/offset + distinct_on + inner + nested_*.

func AggTypeNameAtDepth 

func AggTypeNameAtDepth(objectName string, depth int) string

AggTypeNameAtDepth returns the aggregation type name at a given depth. depth 0: _Type_aggregation depth 1: _Type_aggregation_sub_aggregation depth 2: _Type_aggregation_sub_aggregation_sub_aggregation

func JoinObjectAggArgsWithViewArgs 

func JoinObjectAggArgsWithViewArgs(info *base.ObjectInfo, filterName string, pos *ast.Position) ast.ArgumentDefinitionList

JoinObjectAggArgsWithViewArgs creates args for _join_aggregation type fields, optionally prepending view args for parameterized views.

func JoinObjectQueryArgsWithViewArgs 

func JoinObjectQueryArgsWithViewArgs(info *base.ObjectInfo, filterName string, pos *ast.Position) ast.ArgumentDefinitionList

JoinObjectQueryArgsWithViewArgs creates args for _join type fields, optionally prepending view args for parameterized views.

func PrependViewArgs 

func PrependViewArgs(info *base.ObjectInfo, args ast.ArgumentDefinitionList, pos *ast.Position) ast.ArgumentDefinitionList

PrependViewArgs puts the target's view args in front of an argument list. A no-op for a target that is not a parameterized view.

func QueryArgsWithViewArgs 

func QueryArgsWithViewArgs(info *base.ObjectInfo, filterName string, pos *ast.Position) ast.ArgumentDefinitionList

QueryArgsWithViewArgs returns standard query arguments, optionally prepending an "args" parameter for parameterized views (@args directive).

func SpatialObjectAggArgs 

func SpatialObjectAggArgs(filterName string, pos *ast.Position) ast.ArgumentDefinitionList

SpatialObjectAggArgs creates args for _spatial_aggregation type fields (no limit/offset).

func SpatialObjectQueryArgs 

func SpatialObjectQueryArgs(filterName string, pos *ast.Position) ast.ArgumentDefinitionList

SpatialObjectQueryArgs creates args for _spatial type fields (includes limit/offset).

func SubQueryArgsWithViewArgs 

func SubQueryArgsWithViewArgs(info *base.ObjectInfo, filterName string, pos *ast.Position) ast.ArgumentDefinitionList

SubQueryArgsWithViewArgs is the sub-query (reference) argument set for a target that may be a PARAMETERIZED VIEW, and it takes the view's args exactly as the root query does. A nested field is the ONLY place those arguments can be passed — there is no root query in the path — so without this the view is unreachable through the reference: a back navigation field on a table that a parameterized view references could not be given the view's parameters at all.

func VectorSearchArgs 

func VectorSearchArgs(hasVector, hasEmbeddings bool, pos *ast.Position) ast.ArgumentDefinitionList

VectorSearchArgs is the similarity/semantic argument decorator: hasVector only → similarity; hasEmbeddings → similarity + semantic.

func ViewArgsArgument 

func ViewArgsArgument(info *base.ObjectInfo, pos *ast.Position) *ast.ArgumentDefinition

ViewArgsArgument returns the "args" argument for a field whose TARGET is a parameterized view, or nil when it is not one.

Every field that reaches a parameterized view needs it — a navigation field, its aggregation twins, a @join, a sub-aggregation member — because the view cannot run without its parameters and a nested field is the only place to pass them. Requiredness is the same rule the root query follows: an input with a NonNull member makes args itself NonNull, so "you must parameterize this view" is stated once and enforced everywhere it is reachable.

Types

type CacheConfig 

type CacheConfig struct {
	// MaxEntries is the LRU capacity; 0 selects the default.
	MaxEntries int
	// TTL bounds staleness the invalidation closure cannot see; 0 selects
	// the default.
	TTL time.Duration
	// Disabled turns the cache off entirely (every ForName resolves from
	// CoreDB).
	Disabled bool
}

CacheConfig controls the read-side definition cache (the store's port of the compiled provider's schemaCache).

func DefaultCacheConfig 

func DefaultCacheConfig() CacheConfig

DefaultCacheConfig mirrors the compiled provider's defaults.

type CatalogChecker 

type CatalogChecker = catalog.CatalogChecker

CatalogChecker reports whether a catalog (data source) currently has an active engine — _schema_hard_remove refuses to unregister a loaded source. Implemented by catalog.Service.

type Config 

type Config struct {
	// VecSize is the annotations.vec dimension (fixed at CoreDB init). 0
	// disables vector operations.
	VecSize int
	// IsReadonly rejects all writes (cluster workers read the already-populated
	// catalog schema).
	IsReadonly bool
	// Cache configures the read-side definition cache; the zero value takes
	// the defaults (set Disabled to switch it off).
	Cache CacheConfig
}

Config holds the entity-storage provider configuration.

type Embedder 

type Embedder interface {
	CreateEmbedding(ctx context.Context, input string) (*sources.EmbeddingResult, error)
	CreateEmbeddings(ctx context.Context, inputs []string) (*sources.EmbeddingsResult, error)
}

Embedder creates embedding vectors for annotation seeding at insert time. The engine's embedding source satisfies it structurally. nil when embeddings are not configured; the writer then persists entities without vectors.

type SourceState 

type SourceState struct {
	Name         string
	Version      string
	Capabilities json.RawMessage
	Engine       string
	ReadOnly     bool
	Prefix       string
	AsModule     bool
	IsExtension  bool
	Loaded       bool
	Disabled     bool
	Suspended    bool
	// Dependencies are the source names this source declares @dependency on
	// (extension sources); persisted alongside the rows for the manager's
	// suspend / reactivate cascade.
	Dependencies []string
	// Force skips the version gate: the rows are rewritten even when the
	// stored version matches. The manager uses it to refresh dependents after
	// their base changed (re-resolves virtual attribution) — the dependents'
	// own versions are unchanged.
	Force bool
}

SourceState is the load state of one data source. Version is the catalog content hash — an unchanged version makes writeSource a no-op. Engine is the source's engine type string (re-attached as @catalog(name, engine) on read); ReadOnly is the per-source option gating mutation generation. Prefix and AsModule are the compile options that shape names at read time. IsExtension marks an extension source — generated members of its objects carry @dependency(name: <source>).

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store is the entity-storage catalog provider. It holds the in-memory system layer (never persisted) and talks to the CoreDB `catalog` schema through the pool with direct SQL — the statement forms proven by the provider DML inventory (core-db/hugr_catalog_test.go: probeCatalogStatements).

Method surfaces are added incrementally: M2 adds the writer (MutableProvider / CatalogManager), M3 adds ForName/Types synthesis and the read cache.

func New 

func New(_ context.Context, pool *db.Pool, cfg Config, embedder Embedder) (*Store, error)

New assembles the system layer and returns a Store bound to the CoreDB pool. It performs no DDL — the catalog schema is delivered by coredb.InitSchema (new databases) or the hugr 0.0.19 migration (existing databases).

func (*Store) ActiveCatalogs 

func (s *Store) ActiveCatalogs(ctx context.Context) ([]string, error)

ActiveCatalogs lists the catalogs that should be attached: stored, loaded, enabled and not suspended (the cluster.CatalogProvider surface — workers reconcile their attached sources against it).

func (*Store) AddCatalog 

func (s *Store) AddCatalog(ctx context.Context, name string, cat catsrc.Catalog) error

AddCatalog registers a catalog source and persists its entities. It is version-gated on the STORED version (content hash + compile options): when unchanged the rows are current and only the visibility flags are repaired — a redeploy costs one metadata read. Otherwise the source is partially compiled (VALIDATE + PREPARE against the live store), collected and written wholesale in one transaction. A content rewrite refreshes active dependents (their references may resolve differently against the new base), and any change retries suspended sources whose dependencies may now be satisfied.

func (*Store) CleanOrphanedCatalogs 

func (s *Store) CleanOrphanedCatalogs(ctx context.Context) error

CleanOrphanedCatalogs removes the stored schema of every source that is no longer registered: a metadata row with no core.data_sources row and no live handle in this process. Runtime sources have no data_sources row either, so the live handles are what keeps them — they are all added before startup reaches this point.

func (*Store) ClearSourceVersion 

func (s *Store) ClearSourceVersion(ctx context.Context, name string) error

ClearSourceVersion invalidates a source's STORED version so the next load recompiles and rewrites its rows wholesale instead of taking the version gate (_schema_version_clean). The gate compares a content hash, so a row set that went stale for a reason the hash cannot see — a compiler change, a partial write recovered by hand — can only be forced out this way.

func (*Store) DataObject 

func (s *Store) DataObject(ctx context.Context, name string) *sdl.Object

DataObject resolves a data object by type name (activity-gated), nil for anything else. The definition comes through ForName, NOT the bare reconstruction: ForName is where the description overlay is applied (and the result cached), so the logical model serves the curated text the entity views serve.

func (*Store) DataObjects 

func (s *Store) DataObjects(ctx context.Context, module string) iter.Seq[*sdl.Object]

DataObjects iterates the data objects that are members of a module.

func (*Store) DataSource 

func (s *Store) DataSource(ctx context.Context, name string) *catalog.DataSourceInfo

DataSource resolves ONE active data source; nil when absent or inactive.

func (*Store) DataSources 

func (s *Store) DataSources(ctx context.Context) iter.Seq[*catalog.DataSourceInfo]

DataSources iterates the active data sources, ordered by name.

func (*Store) Definitions 

func (s *Store) Definitions(ctx context.Context) iter.Seq[*ast.Definition]

Definitions enumerates every compiled definition the schema serves, by name. The store synthesizes derived types on demand, so there is no table to list — the set is the reachability closure of the schema from its roots (schemaTypeNames), each name resolved through ForName.

func (*Store) Description 

func (s *Store) Description(_ context.Context) string

Description returns the schema-level description. The storage keeps no root schema description — sources describe themselves through their modules and data objects — so this is empty.

func (*Store) DirectiveDefinitions 

func (s *Store) DirectiveDefinitions(ctx context.Context) iter.Seq2[string, *ast.DirectiveDefinition]

func (*Store) DirectiveForName 

func (s *Store) DirectiveForName(ctx context.Context, name string) *ast.DirectiveDefinition

DirectiveForName / DirectiveDefinitions delegate to the static prelude: the whole directive set (system + hugr) is binary-owned; sources never define directives, so the entity storage holds none.

func (*Store) ExistsCatalog 

func (s *Store) ExistsCatalog(name string) bool

ExistsCatalog reports whether the source is known — as a live handle or as stored metadata (registered by a previous process).

func (*Store) ForName 

func (s *Store) ForName(ctx context.Context, name string) *ast.Definition

ForName resolves a type definition by name (nil when absent): cache hit → singleflight-collapsed resolution through the chain, on a fresh read session. The catalog.Provider surface has no error channel, so a read failure is logged here and served as nil — never cached (absence is a nil WITHOUT an error and is simply not cached either).

A freshly built (non-system) definition is finalised before caching: description defaults for synthetic query members, and the annotation overlay. System definitions come straight from the shared static prelude and are neither finalised nor cached.

func (*Store) Function 

func (s *Store) Function(ctx context.Context, module, name string) *catalog.FunctionEntry

Function resolves a callable member of a module by name ((module, name) is the storage primary key — no root probing needed).

func (*Store) Functions 

func (s *Store) Functions(ctx context.Context, module string) iter.Seq[*catalog.FunctionEntry]

Functions iterates all callable members of a module: function, mutation and subscription kinds in that order, by name within each kind.

func (*Store) GetSchemaVersion 

func (s *Store) GetSchemaVersion(ctx context.Context) (int64, error)

GetSchemaVersion returns the schema version counter (0 when unset).

func (*Store) Implements 

func (s *Store) Implements(ctx context.Context, name string) iter.Seq[*ast.Definition]

Implements yields the interfaces the named type declares.

func (*Store) IncrementSchemaVersion 

func (s *Store) IncrementSchemaVersion(ctx context.Context) (int64, error)

IncrementSchemaVersion bumps the schema version counter and returns the new value. Two-step read + literal write inside one transaction: no in-SQL arithmetic over the JSON value column, so the statement shapes hold on both a native DuckDB CoreDB and an attached PostgreSQL one. The management node is the single writer (same assumption the old provider made).

func (*Store) InvalidateAll 

func (s *Store) InvalidateAll()

InvalidateAll drops every cached definition — the cluster worker's response to a schema-version change (the rows were rewritten by the writer node).

func (*Store) InvalidateCatalog 

func (s *Store) InvalidateCatalog(name string)

InvalidateCatalog drops the cached definitions a source contributed to. The affected-object closure is not available here (the write happened on another node), so the whole source bucket goes, roots included.

func (*Store) IsSuspended 

func (s *Store) IsSuspended(name string) bool

IsSuspended reports whether the source exists and is suspended.

func (*Store) Module 

func (s *Store) Module(ctx context.Context, name string) *catalog.ModuleInfo

Module resolves a module by full dotted name ("" = root); nil when absent or backed by no active source. ONE query: the root kinds come from a flat bool_or aggregate over the module's CLOSURE rows (the hierarchy was flattened at save time — buildModuleClosure), with the mutation kinds gated by data_source_meta.read_only (all-read-only sources → no mutations and no mutation functions).

func (*Store) Modules 

func (s *Store) Modules(ctx context.Context, parent string) iter.Seq[*catalog.ModuleInfo]

Modules iterates the DIRECT child modules of parent — children, activity and root kinds all come from ONE grouped query.

func (*Store) MutationType 

func (s *Store) MutationType(ctx context.Context) *ast.Definition

func (*Store) NameExists 

func (s *Store) NameExists(ctx context.Context, name string) bool

NameExists reports whether ForName would serve the name — the validate-only classification pass (write-time name-collision checks §5.1, cheap negatives): the same per-class EXISTS probes the resolver chain runs, without building any definition. It over-approximates module roots whose kind gates need row flags; a read failure logs and classifies as NOT served.

func (*Store) PossibleTypes 

func (s *Store) PossibleTypes(ctx context.Context, name string) iter.Seq[*ast.Definition]

PossibleTypes yields the concrete members of an interface or union. Unions carry their members inline; interface implementors are found by scanning the enumerable object types. Interfaces and unions are source-defined and rare, and this is a cold validator/introspection path, so the scan is acceptable.

func (*Store) QueryType 

func (s *Store) QueryType(ctx context.Context) *ast.Definition

QueryType / MutationType / SubscriptionType synthesize the top-level roots through ForName: resolveModuleRoot builds each from its module members, so a root with no members resolves to nil.

func (*Store) ReactivateCatalog 

func (s *Store) ReactivateCatalog(ctx context.Context, name string, cat catsrc.Catalog) error

ReactivateCatalog re-activates a previously suspended catalog. The rows were never dropped, so this is the Add path: an unchanged version just clears the suspended flag, a changed one recompiles.

func (*Store) RegisterUDFs 

func (s *Store) RegisterUDFs(ctx context.Context, checker CatalogChecker) error

RegisterUDFs registers the schema-management UDFs on the CoreDB pool. The checker gates _schema_hard_remove; nil skips the gate.

func (*Store) ReindexEmbeddings 

func (s *Store) ReindexEmbeddings(ctx context.Context, dataSource string, batchSize int) (int, error)

ReindexEmbeddings recomputes the embedding vector of every entity in scope (empty dataSource = all) and returns how many rows were rewritten. Embedding happens in batches OUTSIDE any transaction — each batch's vectors are written as they are computed, so a failure part-way leaves the already-processed rows updated (the count reports how far it got) instead of losing the whole run.

func (*Store) Relations 

func (s *Store) Relations(ctx context.Context, object string) iter.Seq[*catalog.RelationInfo]

Relations iterates the logical edges of a data object (see genContext.relations for the synthesis rules).

func (*Store) ReloadCatalog 

func (s *Store) ReloadCatalog(ctx context.Context, name string) error

ReloadCatalog refreshes a source (Reload when supported) and re-runs the Add path — the version gate makes an unchanged catalog a cheap no-op, so there is no separate incremental path: a changed version rewrites the source's rows wholesale.

func (*Store) RemoveCatalog 

func (s *Store) RemoveCatalog(ctx context.Context, name string) error

RemoveCatalog deletes a source's entity rows, metadata, dependencies and seed annotations (the unregister primitive) and drops the source handle. Sources depending on it are suspended first (recursively) — their rows stay and reactivateSuspended restores them when the dependency returns.

func (*Store) ReplacesCatalogOnAdd 

func (s *Store) ReplacesCatalogOnAdd() bool

ReplacesCatalogOnAdd reports the store's write contract: writeSource deletes the source's rows and re-inserts them wholesale inside one transaction, so an entity the source stopped declaring cannot survive an add. A reload therefore needs no destructive pre-step — see catalog.ReplacingCatalogManager.

func (*Store) SetArgumentDescription 

func (s *Store) SetArgumentDescription(ctx context.Context, typeName, fieldName, argName, desc, longDesc string) error

SetArgumentDescription curates a generated field argument's description; evicts the owning type.

func (*Store) SetDataObjectDescription 

func (s *Store) SetDataObjectDescription(ctx context.Context, name, desc, longDesc string) error

SetDataObjectDescription curates a data object's description; evicts that type.

func (*Store) SetDataSourceDescription 

func (s *Store) SetDataSourceDescription(ctx context.Context, name, desc, longDesc string) error

SetDataSourceDescription curates a data source (no ForName surface — data source descriptions reach clients through the entity_data_sources view).

func (*Store) SetDefinitionDescription 

func (s *Store) SetDefinitionDescription(ctx context.Context, typeName, desc, longDesc string) error

SetDefinitionDescription curates a generated type's description; evicts it.

func (*Store) SetFieldDescription 

func (s *Store) SetFieldDescription(ctx context.Context, typeName, fieldName, desc, longDesc string) error

SetFieldDescription curates a generated field's description; evicts its type.

func (*Store) SetFunctionDescription 

func (s *Store) SetFunctionDescription(ctx context.Context, module, name, kind, desc, longDesc string) error

SetFunctionDescription curates ONE function kind (keyed module.name under the kind, parent=module) and evicts the single module root that exposes it — the same name in another root namespace is a different operation and keeps its own description. An empty kind curates the query function.

func (*Store) SetModuleDescription 

func (s *Store) SetModuleDescription(ctx context.Context, module, desc, longDesc string) error

SetModuleDescription curates a module (no ForName surface — module descriptions reach clients through the logical model / entity_modules view).

func (*Store) SetObjectFieldDescription 

func (s *Store) SetObjectFieldDescription(ctx context.Context, owner, field, desc, longDesc string) error

SetObjectFieldDescription curates a data-object field (incl. a relation navigation field); evicts the owning type AND its cached derived types (filters / mutation inputs / aggregations), whose same-named fields inherit the description at reconstruction.

func (*Store) SetSourceTypeDescription 

func (s *Store) SetSourceTypeDescription(ctx context.Context, name, desc, longDesc string) error

SetSourceTypeDescription curates a residual source type's description; evicts it.

func (*Store) SourceTypes 

func (s *Store) SourceTypes(ctx context.Context) iter.Seq2[string, *ast.Definition]

SourceTypes iterates the residual source-defined base types — exactly the content of catalog.types (active sources, by name), with @catalog re-attached the same way ForName serves them.

func (*Store) SubscriptionType 

func (s *Store) SubscriptionType(ctx context.Context) *ast.Definition

func (*Store) SuspendCatalog 

func (s *Store) SuspendCatalog(ctx context.Context, name string) error

SuspendCatalog hides a source without touching its rows — the read-side activity gate excludes suspended sources. A source that was never written is a no-op (mirrors setFlags).

func (*Store) System 

func (s *Store) System() *static.Provider

System returns the in-memory system-type layer. The writer skips sdl.IsSystemType(def) and the reader delegates system-type resolution here (except Query/Mutation/Subscription, which are synthesized from catalog.*).

func (*Store) SystemTypes 

func (s *Store) SystemTypes(ctx context.Context) iter.Seq2[string, *ast.Definition]

SystemTypes iterates the binary-owned static prelude.

func (*Store) Type 

func (s *Store) Type(ctx context.Context, name string) *ast.Definition

Type resolves a type definition by name — the ForName resolver chain.

func (*Store) Types 

func (s *Store) Types(ctx context.Context) iter.Seq2[string, *ast.Definition]

Types enumerates the whole compiled schema as (name, definition) pairs.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.