core

package
v0.11.9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 4, 2026 License: Apache-2.0 Imports: 27 Imported by: 1

Documentation

Overview

Package core is the thin OpenVaultDB layer between the HTTP API and DALgo drivers. Storage access goes through dal.DB natively — inGitDB via dalgo2ingitdb, SQLite via dalgo2sqlite — so ovdb's job is schema-mode enforcement, collection provisioning (via ddl.SchemaModifier), inferred schema observation, and (future) authentication. Reads, writes, updates and queries pass through to the driver.

Index

Constants

This section is empty.

Variables

View Source 
var ErrAlreadyExists = errors.New("record already exists")

ErrAlreadyExists is the insert-conflict sentinel (mapped to HTTP 409).

View Source 
var ErrInvalidDTQL = errors.New("invalid or unsupported DTQL query")

ErrInvalidDTQL identifies invalid or unsupported DTQL query shapes.

View Source 
var ErrInvalidFieldName = errors.New("invalid field name")

ErrInvalidFieldName identifies a write that carries a field name that is not a plain name (mapped to HTTP 400 bad_request). See ValidateFieldName.

View Source 
var ErrInvalidKey = errors.New("invalid key")

ErrInvalidKey identifies a malformed or unsafe key, collection name or parent path (mapped to HTTP 400 invalid_key).

View Source 
var ErrInvalidQuery = errors.New("invalid query")

ErrInvalidQuery identifies a structurally invalid wire query (mapped to HTTP 400).

View Source 
var ErrNotFound = errors.New("record not found")

ErrNotFound is the server-side not-found sentinel (mapped to HTTP 404).

View Source 
var ErrQueryUnsupported = errors.New("structured queries are not supported on this engine")

ErrQueryUnsupported identifies a structured query refused because the mounted storage engine cannot run structured queries safely yet (mapped to HTTP 501 query_unsupported). Use errors.As with *QueryUnsupportedError for the engine name.

View Source 
var ErrUpdateOfMissingRecord = errors.New("cannot update: record not found")

ErrUpdateOfMissingRecord is returned when an update op targets a record that neither exists in the store nor was written earlier in the batch.

Functions

func ParseDTQL added in v0.4.0 

func ParseDTQL(doc []byte) (dal.StructuredQuery, string, error)

ParseDTQL validates the server's bounded single-collection query profile. The returned collection is suitable for checking the token's capabilities. It knows no engine, so it checks field names with the widest rule (the one of quotedNameEngines); the Database that runs the query checks them again with the rule of its own engine, before any adapter is reached.

func ParseKey 

func ParseKey(segments ...string) (*record.Key, error)

ParseKey builds a *record.Key from already-unescaped path segments (alternating collection/id pairs, root first). Segments are validated for path-traversal safety (ValidateSegment) before the key is constructed.

func ParseKeyPath 

func ParseKeyPath(raw string) (*record.Key, error)

ParseKeyPath parses a dal-escaped key path ("collection/id[/sub/id...]") into a dalgo key: segments are split on '/' and percent-unescaped individually, so dal.EscapeID-encoded characters inside IDs survive.

func RootCollection added in v0.6.1 

func RootCollection(key *record.Key) string

RootCollection returns the root collection name of key: the collection a per-collection capability scopes, and the top-level directory/table the driver writes under.

func SQLiteLogicalName added in v0.11.8 

func SQLiteLogicalName(name string) (string, bool)

SQLiteLogicalName returns the public identifier inside a SQL-quoted schema key of a SQLite manifest, where a doubled quote stands for one literal quote. It is the one definition the guard (declaredCollections) and the mount (the recordsets it registers with the driver) share.

func ValidateCollectionName added in v0.6.1 

func ValidateCollectionName(name string) error

ValidateCollectionName applies ValidateSegment to a collection name taken from a request body (query collection, DTQL source).

func ValidateFieldName added in v0.11.4 

func ValidateFieldName(name string) error

ValidateFieldName checks one field name from a request body against the strict rule. The rule a mount applies depends on its engine.

func ValidateFieldNames added in v0.11.8 

func ValidateFieldNames(names []string) error

ValidateFieldNames checks every name with ValidateFieldName, in order; no names is fine. The error wraps ErrInvalidFieldName.

func ValidateSegment added in v0.6.1 

func ValidateSegment(s string) error

ValidateSegment checks one decoded key segment (collection name or record id) for path-traversal safety on every engine. A segment must be non-empty, must not be "." or "..", must not contain control characters (U+0000–U+001F, U+007F), and — because IDs may legitimately carry an escaped '/' — none of its '/'- or '\'-separated components may be "." or "..": file-backed engines join segments into filesystem paths, so "../../secrets" inside an id would escape its collection.

Types

type Database 

type Database struct {
	Manifest *manifest.Manifest
	// contains filtered or unexported fields
}

Database is one mounted logical database: a DALgo driver plus mode enforcement.

func Open 

func Open(m *manifest.Manifest, db dal.DB, supportedModes []schema.Mode, cataloguePath string, policies ...access.Policy) (*Database, error)

Open validates driver/schema-mode compatibility and prepares the database: declared collections are provisioned through the driver's ddl.SchemaModifier, and (for partial/schemaless modes) the inferred schema catalogue is loaded from cataloguePath.

func OpenWithPolicyController added in v0.5.0 

func OpenWithPolicyController(m *manifest.Manifest, db dal.DB, modes []schema.Mode, cataloguePath string, controller *policystore.Controller) (*Database, error)

OpenWithPolicyController mounts an immutable owner generation provider. The controller's publication APIs remain trusted owner-administration APIs.

func (*Database) Apply 

func (d *Database) Apply(ctx context.Context, ops []Op, message string) (int, error)

Apply validates a batch of ops (pre-flight, since inGitDB cannot roll back files already written) and then passes them through to the driver in order inside one dal.RunReadwriteTransaction — for inGitDB that is at most one git commit per batch, with message as the commit message.

func (*Database) CanQuery added in v0.11.4 

func (d *Database) CanQuery() bool

CanQuery reports whether structured queries (/query, /dtql) are allowed on this mount. It is the same allow-list guardQuery enforces, so database metadata can advertise exactly what the guard will accept.

func (*Database) Close 

func (d *Database) Close() error

Close releases the engine resources behind the database: it propagates to the underlying DALgo driver when that driver implements io.Closer (SQLite, PostgreSQL, MySQL) and runs callbacks registered with OnClose. It is idempotent; later calls return the first call's result. Callers must stop using the database before closing it (the server drains in-flight requests first, see server.Unmount).

func (*Database) CollectionForeignKeys added in v0.11.0 

func (d *Database) CollectionForeignKeys(ctx context.Context, collection string) ([]dbschema.ForeignKeyDef, error)

CollectionForeignKeys returns foreign keys discovered by the storage provider. A provider without schema introspection contributes no keys; manifest-declared references are handled separately by the caller.

func (*Database) Collections 

func (d *Database) Collections(ctx context.Context) ([]string, error)

Collections lists collections known to the driver.

func (*Database) Coordinator added in v0.5.0 

func (d *Database) Coordinator() *access.EnforcementCoordinator

func (*Database) DB 

func (d *Database) DB() dal.DB

DB exposes the underlying DALgo driver.

func (*Database) Execute 

func (d *Database) Execute(ctx context.Context, q Query) ([]Record, error)

Execute translates the wire query to dal.StructuredQuery and runs it on the driver. Result keys are full paths from the database root: records of a nested collection carry their parent key.

func (*Database) ExecuteDTQL 

func (d *Database) ExecuteDTQL(ctx context.Context, doc []byte) ([]Record, error)

ExecuteDTQL runs DTQL through the same secured DALgo handle as record reads.

func (*Database) ExecuteDTQLQuery added in v0.4.0 

func (d *Database) ExecuteDTQLQuery(ctx context.Context, query dal.StructuredQuery) ([]Record, error)

ExecuteDTQLQuery executes an already parsed query after validating its shape.

func (*Database) Exists 

func (d *Database) Exists(ctx context.Context, key *record.Key) (bool, error)

Exists reports whether the record exists. Like Get, it refuses an undeclared collection on a SQL engine before the adapter is called.

func (*Database) Get 

func (d *Database) Get(ctx context.Context, key *record.Key) (map[string]any, error)

Get returns record data or ErrNotFound. A collection the database does not declare is ErrNotFound on an engine whose adapter builds SQL, without an adapter call (GuardKey).

func (*Database) GuardCollection added in v0.11.8 

func (d *Database) GuardCollection(name string) error

GuardCollection refuses, before any adapter call, a collection the database does not declare when its adapter builds SQL (sqlite, postgres, mysql, and any engine not known to be a document engine). dalgo2sql writes the collection name into the text of key reads and writes, and ValidateCollectionName is only a path-safety rule that accepts quotes, spaces and semicolons, so the set of collections the mount declared when it opened is the allow-list. Names match exactly, case included. Document engines keep their own rule: any collection that passes ValidateCollectionName. The error wraps ErrNotFound.

func (*Database) GuardKey added in v0.11.8 

func (d *Database) GuardKey(key *record.Key) error

GuardKey refuses, before any adapter call, a key the adapter of a SQL engine cannot address safely: a key with a parent, because a SQL mount has no subcollections (dalgo2sql maps such a key to a recordset named <leaf>_<parent> that no mount registers, and its delete statement names only the leaf table, so the capability checked on the root collection would not be the table written), and a collection the database does not declare (see GuardCollection). Document engines address nested keys natively and are not refused here. A nil key names nothing and is left to the caller. The error wraps ErrNotFound (HTTP 404 not_found).

func (*Database) HasAccessPolicies added in v0.5.0 

func (d *Database) HasAccessPolicies() bool

func (*Database) ID 

func (d *Database) ID() string

ID returns the database id.

func (*Database) InferredSnapshot 

func (d *Database) InferredSnapshot() *inferred.Snapshot

InferredSnapshot returns the inferred schema catalogue view, or nil for strict databases.

func (*Database) OnClose added in v0.6.0 

func (d *Database) OnClose(fn func() error)

OnClose registers fn to run when the database is closed, after resources registered later (reverse order). Mounts use it for resources the dal.DB driver does not own, e.g. a Firestore client.

func (*Database) PolicyLayers added in v0.5.0 

func (d *Database) PolicyLayers(ctx context.Context) []PolicyLayer

func (*Database) PublishPolicies added in v0.5.0 

func (d *Database) PublishPolicies(ctx context.Context, expected string, documents []access.DTQLDocument) (policystore.Snapshot, error)

func (*Database) ReloadPolicies added in v0.5.0 

func (d *Database) ReloadPolicies(ctx context.Context) (policystore.Snapshot, error)

ReloadPolicies reloads this owner's committed generation. It is an embedded owner-administration API; no data endpoint grants this authority.

func (*Database) SelectAccessSample added in v0.5.0 

func (d *Database) SelectAccessSample(ctx context.Context, query dal.StructuredQuery, n int, requester access.Principal) ([]Record, []dal.OrderExpression, error)

SelectAccessSample selects only the readable intersection. Its adapters use different key expressions; neither stored document id fields nor post-page sorting substitute for canonical record identity.

func (*Database) SetAfterWrite 

func (d *Database) SetAfterWrite(fn func(ctx context.Context) error)

SetAfterWrite registers a hook invoked after each successfully applied write batch. Used by pkg/mount to wire git push policies for inGitDB.

func (*Database) StreamDTQLSnapshot added in v0.8.0 

func (d *Database) StreamDTQLSnapshot(ctx context.Context, query dal.StructuredQuery, emit func(Record) error) error

StreamDTQLSnapshot executes one complete query through one DALgo reader. Local OVDB writes cannot interleave with the capture. The caller must bound the emitted result and persist it before exposing any page token.

func (*Database) ValidateFieldPath added in v0.11.8 

func (d *Database) ValidateFieldPath(path []string) error

ValidateFieldPath checks the path of an update from a write body. The first segment is a field of the record, so it must be a plain name (ValidateFieldName) on every engine. The segments after it are map keys: on a SQL engine (or one nobody classified) they are held to the same rule, to fail closed; on a document engine they never reach SQL and are data (Sneat's linkage writes ["related", ext, collection, "id@spaceID"]), so only a segment that is empty, blank or carries a control character is refused (an empty one panics in update.ByFieldPath inside the transaction). An empty path is refused. The error wraps ErrInvalidFieldName.

type Filter 

type Filter struct {
	Field string `json:"field"`
	Op    string `json:"op"`
	Value any    `json:"value"`
}

Filter is one field condition. Op is one of: ==, <, <=, >, >=, in, array-contains, array-contains-any.

type ModeCompatibilityError 

type ModeCompatibilityError struct {
	Engine    string
	Requested schema.Mode
	Supported []schema.Mode
}

ModeCompatibilityError is the loud failure for a schema mode a driver does not support.

func (*ModeCompatibilityError) Error 

func (e *ModeCompatibilityError) Error() string

type Op 

type Op struct {
	Op      string         `json:"op"` // set | insert | update | delete
	Key     *record.Key    `json:"-"`
	KeyPath string         `json:"key"`
	Data    map[string]any `json:"data,omitempty"`
	Updates []UpdateOp     `json:"updates,omitempty"`
}

Op is one operation of a write batch, in wire format (see docs/api.md).

type OrderBy 

type OrderBy struct {
	Field string `json:"field"`
	Desc  bool   `json:"desc,omitempty"`
}

OrderBy is one ordering key.

type PolicyLayer added in v0.5.0 

type PolicyLayer struct {
	Kind     string
	Enabled  bool
	Policies []access.Policy
	Err      error
}

PolicyLayer is a trusted in-process snapshot, never an HTTP DTO. Every enabled participant remains represented even if another participant denies. Hosts must separately authorize disclosure before exposing any of its facts.

type Profile added in v0.11.7 

type Profile struct {
	Kind ProfileKind
	// Sources lists every collection read in the query, in document order,
	// including reads inside derived sources and subquery expressions.
	Sources []ProfileSource
	// HasSubquery reports a derived source, scalar subquery, EXISTS predicate
	// or query-valued comparison operand anywhere in the query.
	HasSubquery bool
	// HasAggregation reports GROUP BY, HAVING or an aggregate function in the
	// query's own columns or ordering.
	HasAggregation bool
}

Profile is the classification of a DTQL query.

func ClassifyDTQL added in v0.11.7 

func ClassifyDTQL(query dal.StructuredQuery) (Profile, error)

ClassifyDTQL decides which profile query belongs to and refuses everything outside both. It is a security boundary: a shape it does not recognise is refused with ErrInvalidDTQL, never passed through.

The relational rules are checked for every document, single-collection ones included, over the whole query tree and the subqueries in it. A document that ParseDTQL accepts is therefore refused here when it carries a money configuration, the one such shape: a schema-qualified root, a scan root and a database on the root are refused by ParseDTQL as well. Names are checked next, with the query guard's rules (see validateRelationalNames), so a name that ParseDTQL refuses is not an accepted relational document either.

A query is single-collection only when validateDTQL accepts it and it also names no database and has no subquery anywhere (validateDTQL checks names inside conditions but does not refuse a subquery there, so a subquery in WHERE would otherwise hide a second source). Every other query is relational.

type ProfileKind added in v0.11.7 

type ProfileKind string

ProfileKind names the DTQL profile a query belongs to. 

const (
	// ProfileSingleCollection is today's profile: one plain root collection
	// of the database the request is addressed to, accepted by validateDTQL.
	ProfileSingleCollection ProfileKind = "single-collection"
	// ProfileRelational is the bounded relational profile: joins, GROUP BY,
	// HAVING, aggregates, column aliases and subqueries over root collections.
	ProfileRelational ProfileKind = "relational"
)

type ProfileSource added in v0.11.7 

type ProfileSource struct {
	// Database is the mounted database id the source names, or empty when the
	// source does not name one.
	Database string
	// Collection is the root collection name.
	Collection string
	// Alias is the alias the query gives the source, or empty.
	Alias string
}

ProfileSource is one collection read a query makes. A query that reads the same collection twice lists it twice, because each reference is a read.

type Query 

type Query struct {
	Collection string    `json:"collection"`
	Parent     string    `json:"parent,omitempty"` // dal-escaped parent key path for scoped subcollection queries
	Where      []Filter  `json:"where,omitempty"`
	OrderBy    []OrderBy `json:"orderBy,omitempty"`
	Limit      int       `json:"limit,omitempty"`
	KeysOnly   bool      `json:"keysOnly,omitempty"`
}

Query is a structured query in the JSON wire format (see docs/api.md). Filters are AND-ed. It translates 1:1 onto dal.StructuredQuery and executes natively on the DALgo driver — ovdb does not evaluate queries itself.

func (Query) Target added in v0.6.1 

func (q Query) Target() (parent *record.Key, rootCollection string, err error)

Target validates the query's collection name and parent path and returns the parsed parent key (nil for a root-collection query) and the root collection that scopes it: the parent's root when a parent is given.

type QueryUnsupportedError added in v0.11.4 

type QueryUnsupportedError struct {
	// Engine is the storage engine of the mount, as written in its manifest.
	Engine string
}

QueryUnsupportedError is the typed refusal of a structured query on an engine that is not yet cleared for queries.

func (*QueryUnsupportedError) Error added in v0.11.4 

func (e *QueryUnsupportedError) Error() string

func (*QueryUnsupportedError) Is added in v0.11.4 

func (e *QueryUnsupportedError) Is(target error) bool

Is makes errors.Is(err, ErrQueryUnsupported) true.

type Record 

type Record struct {
	Key  *record.Key
	Data map[string]any
}

Record is one query result.

type UpdateOp 

type UpdateOp struct {
	FieldName       string   `json:"fieldName,omitempty"`
	FieldPath       []string `json:"fieldPath,omitempty"`
	Value           any      `json:"value,omitempty"`
	Delete          bool     `json:"delete,omitempty"`
	Transform       string   `json:"transform,omitempty"`
	ServerTimestamp bool     `json:"serverTimestamp,omitempty"`
}

UpdateOp is one field-level update operation, in wire format.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.