Documentation
¶
Overview ¶
Package datastore emulates the altengine datastore data plane: a per-namespace JSON document store with a bounded, index-served query/aggregate/transaction engine. It keeps one SQLite database per (instance, namespace) so queries, keyset cursors, and unique-index semantics behave the same as the hosted service.
Index ¶
- Constants
- func DecodeNs(seg string) string
- func ToFieldGroups(m map[string][][]identity.Filter) map[string][][]Filter
- func ToGroups(gs [][]identity.Filter) [][]Filter
- func ValidateAggregateRequest(v any) error
- func ValidateConfig(cfg map[string]any) error
- func ValidateQueryRequest(v any) error
- type AggGroup
- type AggregateRequest
- type AggregateResult
- type AutoIndexInfo
- type Filter
- type Handler
- type IndexRow
- type Join
- type LiveConfig
- type LiveDoc
- type LivePublisher
- type Manager
- func (m *Manager) Drop(instanceID, namespace string) (bool, error)
- func (m *Manager) DropInstance(instanceID string)
- func (m *Manager) NamespaceEntries(instanceID string) []NamespaceEntry
- func (m *Manager) Namespaces(instanceID string) []string
- func (m *Manager) Open(instanceID, namespace, autoID string) (*Store, error)
- type Metric
- type NamespaceEntry
- type Order
- type PutDoc
- type QueryRequest
- type QueryResult
- type Store
- func (s *Store) Aggregate(collection string, req AggregateRequest, autoIndex bool, orGroups [][]Filter) (*AggregateResult, error)
- func (s *Store) BatchGet(collection string, keys []any) ([]StoredDoc, error)
- func (s *Store) BatchGetScoped(collection string, keys []any, groups [][]Filter, ...) ([]StoredDoc, error)
- func (s *Store) Collections(after string, limit int) ([]string, *string, error)
- func (s *Store) CreateIndex(collection string, fields []string, unique bool) (int, error)
- func (s *Store) Delete(collection string, keys []any) (int, error)
- func (s *Store) DeleteScoped(collection string, keys []any, match [][]Filter) (int, error)
- func (s *Store) DropIndex(collection string, id int64) (bool, error)
- func (s *Store) Get(collection, key string) (*StoredDoc, error)
- func (s *Store) ListIndexes(collection string) ([]IndexRow, error)
- func (s *Store) Put(collection string, docs []PutDoc) ([]string, int, error)
- func (s *Store) PutScoped(collection string, docs []PutDoc, create, update identity.WritePolicy) ([]string, []LiveDoc, error)
- func (s *Store) Query(collection string, req QueryRequest, autoIndex bool, readGroups [][]Filter) (*QueryResult, error)
- func (s *Store) RunTransaction(ops []TxnOp) ([]any, int, error)
- func (s *Store) RunTransactionChanges(ops []TxnOp, collect bool) ([]any, int, []WriteChange, error)
- type StoredDoc
- type TriggerSink
- type TxnOp
- type WriteChange
Constants ¶
const ( CollectionsPage = 200 CollectionsMax = 1000 )
Collection listing page sizes, as hosted.
Variables ¶
This section is empty.
Functions ¶
func DecodeNs ¶
DecodeNs is decodeNs, exported for other packages that route on a {ns} segment.
It is exported because the ADMIN routes need it too and did not have it: they passed the raw segment to Open(), so browsing the default namespace opened a namespace literally named "_default" — a different, always-empty store. The data browser showed no collections, and a stray "_default" appeared in the namespace list beside the real one.
func ToFieldGroups ¶
ToFieldGroups converts a rule-engine field->groups map (masks / write gates) into datastore form.
func ValidateAggregateRequest ¶
ValidateAggregateRequest rejects an aggregate body whose shape the engine cannot walk.
func ValidateConfig ¶
ValidateConfig is the datastore instance's config check on a write, with the hosted rules.
func ValidateQueryRequest ¶
ValidateQueryRequest rejects a query body whose shape the engine cannot walk.
Types ¶
type AggregateRequest ¶
type AggregateResult ¶
type AggregateResult struct {
Groups []AggGroup `json:"groups"`
AutoIndexed *AutoIndexInfo `json:"auto_indexed,omitempty"`
}
type AutoIndexInfo ¶
type Handler ¶
type Handler struct {
Reg *control.Registry
Auth *auth.Store
Mgr *Manager
// Ident resolves end-user identity tokens (nil = API keys only).
Ident *identity.Service
// Live publishes change events for the datastore→channel live bridge (nil = off).
Live LivePublisher
// Triggers receives committed writes for the instance's trigger rules (nil = off).
Triggers TriggerSink
}
Handler serves the datastore data plane.
func NewHandler ¶
NewHandler builds a datastore handler.
func (*Handler) WithIdentity ¶
WithIdentity enables end-user identity tokens (and their row rules) on this handler.
func (*Handler) WithLive ¶
func (h *Handler) WithLive(p LivePublisher) *Handler
WithLive enables the datastore→channel live bridge.
func (*Handler) WithTriggers ¶ added in v0.2.35
func (h *Handler) WithTriggers(t TriggerSink) *Handler
WithTriggers enables write triggers.
type LiveConfig ¶
type LiveConfig struct {
ChannelInstance string
Collections map[string]string // collection -> keyBy field path ("" = no partition)
}
LiveConfig is an instance's `live` config blob.
type LiveDoc ¶
type LiveDoc struct {
Key string
Data json.RawMessage
}
LiveDoc is a committed document change, carried to the live bridge.
type LivePublisher ¶
type LivePublisher interface {
Publish(instanceID, channel string, data json.RawMessage) (int, error)
}
LivePublisher publishes one message to a channel instance.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager owns the per-(instance,namespace) SQLite databases.
func NewManager ¶
NewManager creates a datastore manager. dir "" (or memory) keeps everything in RAM.
func (*Manager) Drop ¶
Drop deletes a namespace: closes its handle, forgets it, and (in file mode) removes the database file. Returns whether the namespace existed.
func (*Manager) DropInstance ¶
DropInstance closes and removes every namespace database an instance owns (used when the instance is deleted).
func (*Manager) NamespaceEntries ¶
func (m *Manager) NamespaceEntries(instanceID string) []NamespaceEntry
NamespaceEntries is Namespaces with each one's creation time.
The time lives in the namespace's own database, so it survives a restart. A database that is on disk and has not been opened this session is opened to read it, which is the only way to answer — and at an emulator's scale costs nothing worth saving.
func (*Manager) Namespaces ¶
Namespaces lists namespaces known for an instance (opened this session, plus, in file mode, any *.db already on disk).
type NamespaceEntry ¶
type NamespaceEntry struct {
Namespace string `json:"namespace"`
Created int64 `json:"created_at"`
}
NamespaceEntry is one namespace and when it was created — the hosted listing's row.
type PutDoc ¶
type PutDoc struct {
Key any `json:"key"`
Data json.RawMessage `json:"data"`
}
type QueryRequest ¶
type QueryResult ¶
type QueryResult struct {
Documents []StoredDoc `json:"documents,omitempty"`
Keys []string `json:"keys,omitempty"`
Cursor *string `json:"cursor"`
// Metering plumbing — the rows the query read, behind its billed reads. Never part of a
// response, as hosted.
RowsRead int `json:"-"`
AutoIndexed *AutoIndexInfo `json:"auto_indexed,omitempty"`
}
func (*QueryResult) Reads ¶
func (r *QueryResult) Reads() int
Reads is the billed read count behind RowsRead, which the console is shown.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is a handle to one namespace's database plus its instance settings.
func (*Store) Aggregate ¶
func (s *Store) Aggregate(collection string, req AggregateRequest, autoIndex bool, orGroups [][]Filter) (*AggregateResult, error)
Aggregate runs a grouped metrics query. Group dimensions are guarded like sort keys. orGroups carries the caller's row-level read rules as OR-of-AND groups: 0/1 group ANDs into the where, ≥2 groups inject an inline `AND ((g1) OR (g2))` — aggregates count each row once, so no merge/de-dup is needed (unlike a query). Each branch is independently index-served.
func (*Store) BatchGet ¶
BatchGet returns the found documents for the given keys (order unspecified).
func (*Store) BatchGetScoped ¶
func (s *Store) BatchGetScoped(collection string, keys []any, groups [][]Filter, fieldReads map[string][][]Filter) ([]StoredDoc, error)
BatchGetScoped is BatchGet with the caller's row-level read groups AND per-field read masks applied server-side: a row the caller may not read is omitted (never leaked), and a masked field is projected out of the rows they can read.
func (*Store) Collections ¶
Collections lists a page of the collection names in the namespace, in name order, after `after`. The cursor is the last name served, or nil when there are no more.
A skip-scan, as hosted: each step is one seek to the first name after the previous one, so a page costs one seek per collection on it however many documents each holds.
func (*Store) CreateIndex ¶
CreateIndex declares an index and, when unique, creates a backing SQLite unique index. Returns rows written (the one-time build cost, approximated by the collection size).
func (*Store) DeleteScoped ¶
DeleteScoped removes documents whose EXISTING row satisfies the delete rule's match. A row that doesn't match is refused outright (rather than silently skipped) so a client gets a clear 403 instead of a confusing "deleted: 0".
func (*Store) PutScoped ¶
func (s *Store) PutScoped(collection string, docs []PutDoc, create, update identity.WritePolicy) ([]string, []LiveDoc, error)
PutScoped inserts or replaces documents under a create/update rule policy, returning the committed keys and their final (post-stamp) bodies.
func (*Store) Query ¶
func (s *Store) Query(collection string, req QueryRequest, autoIndex bool, readGroups [][]Filter) (*QueryResult, error)
Query runs a QueryRequest. autoIndex controls whether an INDEX_REQUIRED failure auto-creates the suggested index and retries. readGroups carries the caller's row-level read rules as OR-of-AND groups: 0/1 group ANDs into the query, ≥2 groups run the multi-query (UNION) merge — each branch independently index-served.
func (*Store) RunTransaction ¶
RunTransaction applies operations atomically within the namespace.
func (*Store) RunTransactionChanges ¶ added in v0.2.35
RunTransactionChanges is RunTransaction that, with collect, also reports what each write did to each document — for a trigger watching the collections it touches. A delete of a document that did not exist is no change.
type StoredDoc ¶
type StoredDoc struct {
Key string `json:"key"`
Data json.RawMessage `json:"data"`
Created int64 `json:"created,omitempty"`
Updated int64 `json:"updated,omitempty"`
Joins map[string]any `json:"joins,omitempty"`
}
StoredDoc is the wire shape of a document.
func MaskFields ¶
MaskFields projects restricted fields OUT of docs the caller may read at the row level: for each field in fieldReads, remove it from a doc that doesn't satisfy that field's mask groups (nil groups = always visible). Returns NEW StoredDocs only where a field was actually removed — the input docs are never mutated. Fields are top-level; each field's mask is evaluated against the doc's ORIGINAL data (removals are decided before any deletion is applied).
type TriggerSink ¶ added in v0.2.35
type TriggerSink interface {
// DatastoreWatches reports whether any rule could match a write to this collection — asked
// before the write, so a collection nobody watches pays nothing for the bookkeeping.
DatastoreWatches(in *control.Instance, namespace, collection string) bool
// DatastoreWrite hands over a committed write's changes. It must not block.
DatastoreWrite(in *control.Instance, namespace string, changes []WriteChange, writerDepth int)
}
TriggerSink receives committed writes for an instance's trigger rules.
type TxnOp ¶
type TxnOp struct {
Op string `json:"op"`
Collection string `json:"collection"`
Key any `json:"key"`
Data json.RawMessage `json:"data"`
Set map[string]any `json:"set"`
Increment map[string]float64 `json:"increment"`
Remove []string `json:"remove"`
Upsert bool `json:"upsert"`
Exists bool `json:"exists"`
}
type WriteChange ¶ added in v0.2.35
type WriteChange struct {
Collection string
Key string
Op string // create | update | delete
Doc json.RawMessage
Before json.RawMessage
}
WriteChange is what one write did to one document. Before is the document it replaced, where the write already reads it (row-scoped writes, transactions).