Help us shape the future of Go! Take the 2026 Go Developer Survey

store

package
v0.3.45 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 9, 2026 License: MIT Imports: 35 Imported by: 0

Documentation

Overview

Package store is the entity-storage catalog provider (design 034).

It is the engine's catalog storage. The storage it replaced shredded a FULLY compiled GraphQL schema into the _schema_* tables; this one stores only the BASE entities that arrive from sources (modules, data objects, their fields, functions, source-defined types and relations) in the CoreDB `catalog` schema, and reconstructs the served GraphQL schema on the fly at read time.

The served schema is composed of three layers:

  • the SYSTEM layer — scalars, introspection (__*), @system types and the base/query/GIS directives. These are assembled at startup from the binary (static.New, same set the old provider persisted under the "_system" catalog) and held in memory. They are NEVER written to catalog.*: the writer skips sdl.IsSystemType and the reader serves them from the static layer.

  • the SOURCE entities — reconstructed from catalog.* on demand (ForName / Types synthesis). Data objects, source types, functions.

  • the SYNTHESIZED roots and derived types — Query/Mutation/Subscription and per-module roots (never stored, always assembled), plus the derived helpers (_X_aggregation, filters, …) generated lazily by name.

Write path (a partial compiler: prefix + validate + extension merge, NO type generation) → collect base entities → Reconcile into catalog.*. Read path is cached with per-data-source invalidation (schemaCache): a reconcile evicts only the changed/removed sources.

All SQL addresses the tables literally as `core.catalog.<table>`: the CoreDB is always attached as `core` (production and tests alike, via the coredb Source) and DML runs from the DuckDB context on both backends (implicit VARCHAR→STRUCT casts on DuckDB, INOUT assignment casts on Postgres), so the prefix is a constant and the statement texts match core-db's pinned probeCatalogStatements verbatim.

Index

Constants

This section is empty.

Variables

View Source 
var ErrNoEmbeddings = errors.New("embeddings not configured (EMBEDDER_URL not set)")

ErrNoEmbeddings is returned by ReindexEmbeddings when the engine runs without an embedder — there are no vectors to recompute.

View Source 
var ErrReadOnly = errors.New("catalog store: read-only mode")

ErrReadOnly is returned by every write entry point when the store was configured read-only (cluster workers read the catalog the writer node populated).

Functions

func AggRefArgs 

func AggRefArgs(filterName string, pos *ast.Position) ast.ArgumentDefinitionList

AggRefArgs returns reference field args on aggregation types: filter + order_by + distinct_on + inner + nested_*.

func AggSubRefArgs 

func AggSubRefArgs(filterName string, pos *ast.Position) ast.ArgumentDefinitionList

AggSubRefArgs returns args for the _aggregation sub-field on aggregation types. Includes filter + order_by + limit/offset + distinct_on + inner + nested_*.

func AggTypeNameAtDepth 

func AggTypeNameAtDepth(objectName string, depth int) string

AggTypeNameAtDepth returns the aggregation type name at a given depth. depth 0: _Type_aggregation depth 1: _Type_aggregation_sub_aggregation depth 2: _Type_aggregation_sub_aggregation_sub_aggregation

func JoinObjectAggArgsWithViewArgs 

func JoinObjectAggArgsWithViewArgs(info *base.ObjectInfo, filterName string, pos *ast.Position) ast.ArgumentDefinitionList

JoinObjectAggArgsWithViewArgs creates args for _join_aggregation type fields, optionally prepending view args for parameterized views.

func JoinObjectQueryArgsWithViewArgs 

func JoinObjectQueryArgsWithViewArgs(info *base.ObjectInfo, filterName string, pos *ast.Position) ast.ArgumentDefinitionList

JoinObjectQueryArgsWithViewArgs creates args for _join type fields, optionally prepending view args for parameterized views.

func PrependViewArgs 

func PrependViewArgs(info *base.ObjectInfo, args ast.ArgumentDefinitionList, pos *ast.Position) ast.ArgumentDefinitionList

PrependViewArgs puts the target's view args in front of an argument list. A no-op for a target that is not a parameterized view.

func QueryArgsWithViewArgs 

func QueryArgsWithViewArgs(info *base.ObjectInfo, filterName string, pos *ast.Position) ast.ArgumentDefinitionList

QueryArgsWithViewArgs returns standard query arguments, optionally prepending an "args" parameter for parameterized views (@args directive).

func SpatialObjectAggArgs 

func SpatialObjectAggArgs(filterName string, pos *ast.Position) ast.ArgumentDefinitionList

SpatialObjectAggArgs creates args for _spatial_aggregation type fields (no limit/offset).

func SpatialObjectQueryArgs 

func SpatialObjectQueryArgs(filterName string, pos *ast.Position) ast.ArgumentDefinitionList

SpatialObjectQueryArgs creates args for _spatial type fields (includes limit/offset).

func SubQueryArgsWithViewArgs 

func SubQueryArgsWithViewArgs(info *base.ObjectInfo, filterName string, pos *ast.Position) ast.ArgumentDefinitionList

SubQueryArgsWithViewArgs is the sub-query (reference) argument set for a target that may be a PARAMETERIZED VIEW, and it takes the view's args exactly as the root query does. A nested field is the ONLY place those arguments can be passed — there is no root query in the path — so without this the view is unreachable through the reference: a back navigation field on a table that a parameterized view references could not be given the view's parameters at all.

func VectorSearchArgs 

func VectorSearchArgs(hasVector, hasEmbeddings bool, pos *ast.Position) ast.ArgumentDefinitionList

VectorSearchArgs is the similarity/semantic argument decorator: hasVector only → similarity; hasEmbeddings → similarity + semantic.

func ViewArgsArgument 

func ViewArgsArgument(info *base.ObjectInfo, pos *ast.Position) *ast.ArgumentDefinition

ViewArgsArgument returns the "args" argument for a field whose TARGET is a parameterized view, or nil when it is not one.

Every field that reaches a parameterized view needs it — a navigation field, its aggregation twins, a @join, a sub-aggregation member — because the view cannot run without its parameters and a nested field is the only place to pass them. Requiredness is the same rule the root query follows: an input with a NonNull member makes args itself NonNull, so "you must parameterize this view" is stated once and enforced everywhere it is reachable.

Types

type CacheConfig 

type CacheConfig struct {
	// MaxEntries is the LRU capacity; 0 selects the default.
	MaxEntries int
	// TTL bounds staleness the invalidation closure cannot see; 0 selects
	// the default.
	TTL time.Duration
	// Disabled turns the cache off entirely (every ForName resolves from
	// CoreDB).
	Disabled bool
}

CacheConfig controls the read-side definition cache (the store's port of the compiled provider's schemaCache).

func DefaultCacheConfig 

func DefaultCacheConfig() CacheConfig

DefaultCacheConfig mirrors the compiled provider's defaults.

type CatalogChecker 

type CatalogChecker = catalog.CatalogChecker

CatalogChecker reports whether a catalog (data source) currently has an active engine — _schema_hard_remove refuses to unregister a loaded source. Implemented by catalog.Service.

type Config 

type Config struct {
	// VecSize is the annotations.vec dimension (fixed at CoreDB init). 0
	// disables vector operations.
	VecSize int
	// IsReadonly rejects all writes (cluster workers read the already-populated
	// catalog schema).
	IsReadonly bool
	// Cache configures the read-side definition cache; the zero value takes
	// the defaults (set Disabled to switch it off).
	Cache CacheConfig
}

Config holds the entity-storage provider configuration.

type Embedder 

type Embedder interface {
	CreateEmbedding(ctx context.Context, input string) (*sources.EmbeddingResult, error)
	CreateEmbeddings(ctx context.Context, inputs []string) (*sources.EmbeddingsResult, error)
}

Embedder creates embedding vectors for annotation seeding at insert time. The engine's embedding source satisfies it structurally. nil when embeddings are not configured; the writer then persists entities without vectors.

type SourceState 

type SourceState struct {
	Name         string
	Version      string
	Capabilities json.RawMessage
	Engine       string
	ReadOnly     bool
	Prefix       string
	AsModule     bool
	IsExtension  bool
	Loaded       bool
	Disabled     bool
	Suspended    bool
	// Dependencies are the source names this source declares @dependency on
	// (extension sources); persisted alongside the rows for the manager's
	// suspend / reactivate cascade.
	Dependencies []string
	// Force skips the version gate: the rows are rewritten even when the
	// stored version matches. The manager uses it to refresh dependents after
	// their base changed (re-resolves virtual attribution) — the dependents'
	// own versions are unchanged.
	Force bool
}

SourceState is the load state of one data source. Version is the catalog content hash — an unchanged version makes writeSource a no-op. Engine is the source's engine type string (re-attached as @catalog(name, engine) on read); ReadOnly is the per-source option gating mutation generation. Prefix and AsModule are the compile options that shape names at read time. IsExtension marks an extension source — generated members of its objects carry @dependency(name: <source>).

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store is the entity-storage catalog provider. It holds the in-memory system layer (never persisted) and talks to the CoreDB `catalog` schema through the pool with direct SQL — the statement forms proven by the provider DML inventory (core-db/hugr_catalog_test.go: probeCatalogStatements).

Method surfaces are added incrementally: M2 adds the writer (MutableProvider / CatalogManager), M3 adds ForName/Types synthesis and the read cache.

func New 

func New(_ context.Context, pool *db.Pool, cfg Config, embedder Embedder) (*Store, error)

New assembles the system layer and returns a Store bound to the CoreDB pool. It performs no DDL — the catalog schema is delivered by coredb.InitSchema (new databases) or the hugr 0.0.19 migration (existing databases).

func (*Store) ActiveCatalogs 

func (s *Store) ActiveCatalogs(ctx context.Context) ([]string, error)

ActiveCatalogs lists the catalogs that should be attached: stored, loaded, enabled and not suspended (the cluster.CatalogProvider surface — workers reconcile their attached sources against it).

func (*Store) AddCatalog 

func (s *Store) AddCatalog(ctx context.Context, name string, cat catsrc.Catalog) error

AddCatalog registers a catalog source and persists its entities. It is version-gated on the STORED version (content hash + compile options): when unchanged the rows are current and only the visibility flags are repaired — a redeploy costs one metadata read. Otherwise the source is partially compiled (VALIDATE + PREPARE against the live store), collected and written wholesale in one transaction. A content rewrite refreshes active dependents (their references may resolve differently against the new base), and any change retries suspended sources whose dependencies may now be satisfied.

func (*Store) CleanOrphanedCatalogs 

func (s *Store) CleanOrphanedCatalogs(ctx context.Context) error

CleanOrphanedCatalogs removes the stored schema of every source that is no longer registered: a metadata row with no core.data_sources row and no live handle in this process. Runtime sources have no data_sources row either, so the live handles are what keeps them — they are all added before startup reaches this point.

func (*Store) ClearSourceVersion 

func (s *Store) ClearSourceVersion(ctx context.Context, name string) error

ClearSourceVersion invalidates a source's STORED version so the next load recompiles and rewrites its rows wholesale instead of taking the version gate (_schema_version_clean). The gate compares a content hash, so a row set that went stale for a reason the hash cannot see — a compiler change, a partial write recovered by hand — can only be forced out this way.

func (*Store) DataObject 

func (s *Store) DataObject(ctx context.Context, name string) *sdl.Object

DataObject resolves a data object by type name (activity-gated), nil for anything else. The definition comes through ForName, NOT the bare reconstruction: ForName is where the description overlay is applied (and the result cached), so the logical model serves the curated text the entity views serve.

func (*Store) DataObjects 

func (s *Store) DataObjects(ctx context.Context, module string) iter.Seq[*sdl.Object]

DataObjects iterates the data objects that are members of a module.

func (*Store) DataSource 

func (s *Store) DataSource(ctx context.Context, name string) *catalog.DataSourceInfo

DataSource resolves ONE active data source; nil when absent or inactive.

func (*Store) DataSources 

func (s *Store) DataSources(ctx context.Context) iter.Seq[*catalog.DataSourceInfo]

DataSources iterates the active data sources, ordered by name.

func (*Store) Definitions 

func (s *Store) Definitions(ctx context.Context) iter.Seq[*ast.Definition]

Definitions enumerates every compiled definition the schema serves, by name. The store synthesizes derived types on demand, so there is no table to list — the set is the reachability closure of the schema from its roots (schemaTypeNames), each name resolved through ForName.

func (*Store) Description 

func (s *Store) Description(_ context.Context) string

Description returns the schema-level description. The storage keeps no root schema description — sources describe themselves through their modules and data objects — so this is empty.

func (*Store) DirectiveDefinitions 

func (s *Store) DirectiveDefinitions(ctx context.Context) iter.Seq2[string, *ast.DirectiveDefinition]

func (*Store) DirectiveForName 

func (s *Store) DirectiveForName(ctx context.Context, name string) *ast.DirectiveDefinition

DirectiveForName / DirectiveDefinitions delegate to the static prelude: the whole directive set (system + hugr) is binary-owned; sources never define directives, so the entity storage holds none.

func (*Store) ExistsCatalog 

func (s *Store) ExistsCatalog(name string) bool

ExistsCatalog reports whether the source is known — as a live handle or as stored metadata (registered by a previous process).

func (*Store) ForName 

func (s *Store) ForName(ctx context.Context, name string) *ast.Definition

ForName resolves a type definition by name (nil when absent): cache hit → singleflight-collapsed resolution through the chain, on a fresh read session. The catalog.Provider surface has no error channel, so a read failure is logged here and served as nil — never cached (absence is a nil WITHOUT an error and is simply not cached either).

A freshly built (non-system) definition is finalised before caching: description defaults for synthetic query members, and the annotation overlay. System definitions come straight from the shared static prelude and are neither finalised nor cached.

func (*Store) Function 

func (s *Store) Function(ctx context.Context, module, name string) *catalog.FunctionEntry

Function resolves a callable member of a module by name ((module, name) is the storage primary key — no root probing needed).

func (*Store) Functions 

func (s *Store) Functions(ctx context.Context, module string) iter.Seq[*catalog.FunctionEntry]

Functions iterates all callable members of a module: function, mutation and subscription kinds in that order, by name within each kind.

func (*Store) GetSchemaVersion 

func (s *Store) GetSchemaVersion(ctx context.Context) (int64, error)

GetSchemaVersion returns the schema version counter (0 when unset).

func (*Store) Implements 

func (s *Store) Implements(ctx context.Context, name string) iter.Seq[*ast.Definition]

Implements yields the interfaces the named type declares.

func (*Store) IncrementSchemaVersion 

func (s *Store) IncrementSchemaVersion(ctx context.Context) (int64, error)

IncrementSchemaVersion bumps the schema version counter and returns the new value. Two-step read + literal write inside one transaction: no in-SQL arithmetic over the JSON value column, so the statement shapes hold on both a native DuckDB CoreDB and an attached PostgreSQL one. The management node is the single writer (same assumption the old provider made).

func (*Store) InvalidateAll 

func (s *Store) InvalidateAll()

InvalidateAll drops every cached definition — the cluster worker's response to a schema-version change (the rows were rewritten by the writer node).

func (*Store) InvalidateCatalog 

func (s *Store) InvalidateCatalog(name string)

InvalidateCatalog drops the cached definitions a source contributed to. The affected-object closure is not available here (the write happened on another node), so the whole source bucket goes, roots included.

func (*Store) IsSuspended 

func (s *Store) IsSuspended(name string) bool

IsSuspended reports whether the source exists and is suspended.

func (*Store) Module 

func (s *Store) Module(ctx context.Context, name string) *catalog.ModuleInfo

Module resolves a module by full dotted name ("" = root); nil when absent or backed by no active source. ONE query: the root kinds come from a flat bool_or aggregate over the module's CLOSURE rows (the hierarchy was flattened at save time — buildModuleClosure), with the mutation kinds gated by data_source_meta.read_only (all-read-only sources → no mutations and no mutation functions).

func (*Store) Modules 

func (s *Store) Modules(ctx context.Context, parent string) iter.Seq[*catalog.ModuleInfo]

Modules iterates the DIRECT child modules of parent — children, activity and root kinds all come from ONE grouped query.

func (*Store) MutationType 

func (s *Store) MutationType(ctx context.Context) *ast.Definition

func (*Store) NameExists 

func (s *Store) NameExists(ctx context.Context, name string) bool

NameExists reports whether ForName would serve the name — the validate-only classification pass (write-time name-collision checks §5.1, cheap negatives): the same per-class EXISTS probes the resolver chain runs, without building any definition. It over-approximates module roots whose kind gates need row flags; a read failure logs and classifies as NOT served.

func (*Store) PossibleTypes 

func (s *Store) PossibleTypes(ctx context.Context, name string) iter.Seq[*ast.Definition]

PossibleTypes yields the concrete members of an interface or union. Unions carry their members inline; interface implementors are found by scanning the enumerable object types. Interfaces and unions are source-defined and rare, and this is a cold validator/introspection path, so the scan is acceptable.

func (*Store) QueryType 

func (s *Store) QueryType(ctx context.Context) *ast.Definition

QueryType / MutationType / SubscriptionType synthesize the top-level roots through ForName: resolveModuleRoot builds each from its module members, so a root with no members resolves to nil.

func (*Store) ReactivateCatalog 

func (s *Store) ReactivateCatalog(ctx context.Context, name string, cat catsrc.Catalog) error

ReactivateCatalog re-activates a previously suspended catalog. The rows were never dropped, so this is the Add path: an unchanged version just clears the suspended flag, a changed one recompiles.

func (*Store) RegisterUDFs 

func (s *Store) RegisterUDFs(ctx context.Context, checker CatalogChecker) error

RegisterUDFs registers the schema-management UDFs on the CoreDB pool. The checker gates _schema_hard_remove; nil skips the gate.

func (*Store) ReindexEmbeddings 

func (s *Store) ReindexEmbeddings(ctx context.Context, dataSource string, batchSize int) (int, error)

ReindexEmbeddings recomputes the embedding vector of every entity in scope (empty dataSource = all) and returns how many rows were rewritten. Embedding happens in batches OUTSIDE any transaction — each batch's vectors are written as they are computed, so a failure part-way leaves the already-processed rows updated (the count reports how far it got) instead of losing the whole run.

func (*Store) Relations 

func (s *Store) Relations(ctx context.Context, object string) iter.Seq[*catalog.RelationInfo]

Relations iterates the logical edges of a data object (see genContext.relations for the synthesis rules).

func (*Store) ReloadCatalog 

func (s *Store) ReloadCatalog(ctx context.Context, name string) error

ReloadCatalog refreshes a source (Reload when supported) and re-runs the Add path — the version gate makes an unchanged catalog a cheap no-op, so there is no separate incremental path: a changed version rewrites the source's rows wholesale.

func (*Store) RemoveCatalog 

func (s *Store) RemoveCatalog(ctx context.Context, name string) error

RemoveCatalog deletes a source's entity rows, metadata, dependencies and seed annotations (the unregister primitive) and drops the source handle. Sources depending on it are suspended first (recursively) — their rows stay and reactivateSuspended restores them when the dependency returns.

func (*Store) ReplacesCatalogOnAdd 

func (s *Store) ReplacesCatalogOnAdd() bool

ReplacesCatalogOnAdd reports the store's write contract: writeSource deletes the source's rows and re-inserts them wholesale inside one transaction, so an entity the source stopped declaring cannot survive an add. A reload therefore needs no destructive pre-step — see catalog.ReplacingCatalogManager.

func (*Store) SetArgumentDescription 

func (s *Store) SetArgumentDescription(ctx context.Context, typeName, fieldName, argName, desc, longDesc string) error

SetArgumentDescription curates a generated field argument's description; evicts the owning type.

func (*Store) SetDataObjectDescription 

func (s *Store) SetDataObjectDescription(ctx context.Context, name, desc, longDesc string) error

SetDataObjectDescription curates a data object's description; evicts that type.

func (*Store) SetDataSourceDescription 

func (s *Store) SetDataSourceDescription(ctx context.Context, name, desc, longDesc string) error

SetDataSourceDescription curates a data source (no ForName surface — data source descriptions reach clients through the core.catalog.active_sources view).

func (*Store) SetDefinitionDescription 

func (s *Store) SetDefinitionDescription(ctx context.Context, typeName, desc, longDesc string) error

SetDefinitionDescription curates a generated type's description; evicts it.

func (*Store) SetFieldDescription 

func (s *Store) SetFieldDescription(ctx context.Context, typeName, fieldName, desc, longDesc string) error

SetFieldDescription curates a generated field's description; evicts its type.

func (*Store) SetFunctionDescription 

func (s *Store) SetFunctionDescription(ctx context.Context, module, name, kind, desc, longDesc string) error

SetFunctionDescription curates ONE function kind (keyed module.name under the kind, parent=module) and evicts the single module root that exposes it — the same name in another root namespace is a different operation and keeps its own description. An empty kind curates the query function.

func (*Store) SetModuleDescription 

func (s *Store) SetModuleDescription(ctx context.Context, module, desc, longDesc string) error

SetModuleDescription curates a module (no ForName surface — module descriptions reach clients through the logical model / core.catalog.modules view).

func (*Store) SetObjectFieldDescription 

func (s *Store) SetObjectFieldDescription(ctx context.Context, owner, field, desc, longDesc string) error

SetObjectFieldDescription curates a data-object field (incl. a relation navigation field); evicts the owning type AND its cached derived types (filters / mutation inputs / aggregations), whose same-named fields inherit the description at reconstruction.

func (*Store) SetSourceTypeDescription 

func (s *Store) SetSourceTypeDescription(ctx context.Context, name, desc, longDesc string) error

SetSourceTypeDescription curates a residual source type's description; evicts it.

func (*Store) SourceTypes 

func (s *Store) SourceTypes(ctx context.Context) iter.Seq2[string, *ast.Definition]

SourceTypes iterates the residual source-defined base types — exactly the content of catalog.types (active sources, by name), with @catalog re-attached the same way ForName serves them.

func (*Store) SubscriptionType 

func (s *Store) SubscriptionType(ctx context.Context) *ast.Definition

func (*Store) SuspendCatalog 

func (s *Store) SuspendCatalog(ctx context.Context, name string) error

SuspendCatalog hides a source without touching its rows — the read-side activity gate excludes suspended sources. A source that was never written is a no-op (mirrors setFlags).

func (*Store) System 

func (s *Store) System() *static.Provider

System returns the in-memory system-type layer. The writer skips sdl.IsSystemType(def) and the reader delegates system-type resolution here (except Query/Mutation/Subscription, which are synthesized from catalog.*).

func (*Store) SystemTypes 

func (s *Store) SystemTypes(ctx context.Context) iter.Seq2[string, *ast.Definition]

SystemTypes iterates the binary-owned static prelude.

func (*Store) Type 

func (s *Store) Type(ctx context.Context, name string) *ast.Definition

Type resolves a type definition by name — the ForName resolver chain.

func (*Store) Types 

func (s *Store) Types(ctx context.Context) iter.Seq2[string, *ast.Definition]

Types enumerates the whole compiled schema as (name, definition) pairs.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.