Documentation
¶
Overview ¶
Package runtime is the small public package imported by sqletch-generated code. It composes verified constant fragments into SQL deterministically — the byte-for-byte mirror of the compiler's verification renderer (premise P2); a shared conformance test in the compiler pins the equality.
Nothing here parses SQL or touches user data: composition is table-driven selection and concatenation, and user values travel exclusively through bind parameters.
API contract (v1) ¶
Two audiences share this package:
The USER API — what application code is expected to touch: Tree and its constructors (And, Or, Unscoped; the typed per-predicate constructors are generated into your package), TreeCaps, and the sentinel errors ErrFilterRequired, ErrChooseRequired, ErrOrderKey, ErrTreeTooLarge, ErrTreePredicate, ErrTreeArity. These follow Go API compatibility for all v1 releases.
The observability surface — Observer, CacheStats, ShapeUse, ShapeSpaceInfo and the corresponding ComposedCache methods (design doc 18) — is USER API too.
The GENERATED-CODE CONTRACT — Frag, ShapeKey, Bind, Compose and friends, ComposedCache, Expanded. These are public only because generated code lives outside this module. They also follow Go API compatibility within v1, but their semantics are pinned to the sqletch compiler: after upgrading sqletch, re-run `sqletch generate` so generated code and runtime agree (the conformance tests hold per version pair, not across them). Constructing Frag tables by hand is UNSUPPORTED.
Index ¶
- Constants
- Variables
- func BuildArgs(argIdx []int16, vals []any) []any
- func ChooseOrdinal(v, numNamed int, hasDefault bool) (uint8, error)
- func Compose(frags []Frag, key ShapeKey) (string, []int16)
- func ComposeStyle(style Style, frags []Frag, key ShapeKey) (string, []int16)
- func Lookup(shapes map[string]Expanded, key ShapeKey) (string, []int16, error)
- func OrderSeq[T ~int](vals []T, numKeys int) ([]uint8, error)
- func ResolveArgs(binds []Bind, vals, treeArgs []any) []any
- func TreeArgs(t Tree) []any
- type Bind
- type CacheStats
- type Case
- type ComposedCache
- func (c *ComposedCache) Get(queryName string, frags []Frag, key ShapeKey) (string, []int16)
- func (c *ComposedCache) GetBindsStyle(style Style, queryName string, frags []Frag, key ShapeKey) (string, []Bind, error)
- func (c *ComposedCache) GetStyle(style Style, queryName string, frags []Frag, key ShapeKey) (string, []int16)
- func (c *ComposedCache) GetTree(queryName string, frags []Frag, key ShapeKey, tree Tree, caps TreeCaps) (string, []Bind, error)
- func (c *ComposedCache) GetTreeStyle(style Style, queryName string, frags []Frag, key ShapeKey, tree Tree, ...) (string, []Bind, error)
- func (c *ComposedCache) SetMaxBytes(n int64)
- func (c *ComposedCache) SetObserver(o Observer)
- func (c *ComposedCache) Stats() CacheStats
- func (c *ComposedCache) TopShapes(n int) []ShapeUse
- type Expanded
- type Frag
- type Kind
- type Observer
- type Sep
- type ShapeKey
- type ShapeSpaceInfo
- type ShapeUse
- type Span
- type Style
- type Tree
- type TreeCaps
Constants ¶
const ( // MaxOrderKeys: sequence elements pack as key<<1|desc into a uint8, // and the duplicate-key mask below is 64 bits wide. MaxOrderKeys = 64 // MaxChooseOrdinals: ShapeKey.Choices holds one uint8 per @choose // block, counting the @default body. MaxChooseOrdinals = 255 // MaxInArity bounds an @in list on expanding dialects. Bind.Elem is // an int16 holding a 1-based element index, so past this the index // wraps: negative reads as "bind the value whole", and far enough // round it lands on a different element. Unlike the limits above // this one is on CALLER data, not the template, so it is enforced // during composition. Engines cap placeholders well below it // anyway (SQLite's default is 32766). MaxInArity = 32767 )
Structural limits of the ShapeKey encoding. The compiler refuses templates that exceed them (SQLETCH010), so generated code can never reach the checks below; they exist because silent truncation here composes a DIFFERENT query's SQL — a wrong case, a wrong sort column — with no error anywhere. internal/codegen pins these against the scanner's copies.
const MaxTreeArgs = 32767
MaxTreeArgs bounds the predicate arguments one tree may contribute. Bind indices are int16 and the composer accumulates one per argument across the whole tree, so past this the base wraps negative and ResolveArgs indexes out of range. TreeCaps has no upper bound of its own — a project may configure MaxNodes freely — so this is checked against the value, not the caps.
Variables ¶
var ( // ErrFilterRequired is returned when a @filter-tree! argument is the // zero Tree; deliberate unscoped access must use the generated // Unscoped constructor. ErrFilterRequired = errors.New("sqletch: required @filter-tree argument is the zero Tree (use the generated Unscoped() for deliberate opt-out)") ErrTreeTooLarge = errors.New("sqletch: filter tree exceeds the configured caps") ErrTreePredicate = errors.New("sqletch: filter tree references an unknown predicate") // ErrTreeArity is returned when a leaf carries a different number of // arguments than its predicate's parameters. The composer flattens // leaf arguments into one preorder TreeArgs space and each leaf owns // exactly the slice its predicate consumes, so an under-supplied // leaf would bind a NEIGHBORING leaf's value (or index out of range // at query time) and an over-supplied one would silently drop // values; both are construction bugs, rejected before any SQL is // built. Generated predicate constructors always match — this guards // hand-written NewLeaf calls. ErrTreeArity = errors.New("sqletch: filter tree leaf argument count does not match its predicate's parameters") )
var DefaultTreeCaps = TreeCaps{MaxNodes: 32, MaxDepth: 8}
var ErrChooseRequired = errors.New("sqletch: required @choose parameter has its zero value")
ErrChooseRequired is returned before any SQL is sent when a required @choose parameter carries its zero value.
var ErrOrderKey = errors.New("sqletch: invalid @order-by key selection")
ErrOrderKey is returned when an @order-by selection references a key out of range or repeats a key.
var ErrShapeKeyLimit = errors.New("sqletch: construct exceeds the shape-key encoding limit")
ErrShapeKeyLimit reports a construct too large for the shape key's encoding. Reaching it means codegen and the scanner disagree.
var ErrShapeNotExpanded = errors.New("sqletch: shape missing from the static expansion table")
ErrShapeNotExpanded is returned when a statically expanded query is asked for a shape key absent from its table — impossible unless the generated code is stale.
Functions ¶
func BuildArgs ¶
BuildArgs selects the bind values for a shape from the flattened params-struct values. Pointer values pass through as-is (active guards guarantee non-nil; the driver encodes pointers natively).
func ChooseOrdinal ¶
ChooseOrdinal maps a generated enum value to the composer's case ordinal. With a @default, enum 0 selects the default (ordinal numNamed); named cases are 1..numNamed. Without a default, 0 is an error.
func Compose ¶
Compose walks the fragment table in source order and emits the SQL of the shape plus the bind order: argIdx[n] is the params-struct value index bound to placeholder $(n+1). Placeholders are numbered in first-occurrence order per shape. Queries with a @filter-tree use ComposeTree instead.
func ComposeStyle ¶
ComposeStyle is Compose with an explicit placeholder style.
func OrderSeq ¶
OrderSeq converts a generated sort-key slice into the composer's sequence, validating range and rejecting duplicate keys (the same key in both directions makes no sense either).
func ResolveArgs ¶
ResolveArgs materializes a bind plan into driver arguments.
Types ¶
type Bind ¶
type Bind struct {
FromTree bool
Idx int16
// Elem selects within a slice value: 0 binds the value whole; k>0
// binds element k-1 (@in arity expansion on Tier 2 dialects).
Elem int16
}
Bind is one entry of a shape's bind plan: the placeholder $(i+1) takes vals[Idx] (struct source) or treeArgs[Idx] (tree source). Plans contain positions only — never values — so they are cacheable.
func ComposeTree ¶
ComposeTree composes a shape that may include one @filter-tree block. The zero Tree renders TRUE; a zero tree for a required block is rejected by the generated code before reaching here.
type CacheStats ¶
type CacheStats struct {
// Hits counts accesses served from a memoized entry. Summed from
// per-entry counters plus the folded counts of evicted entries, so
// eviction never loses history; hit rate = Hits/(Hits+Misses).
//
// Hit counting starts at the first SetObserver, Stats, or
// TopShapes call (sticky): a cache that is never observed pays
// nothing for the counter on its lock-free hit path. Scrape-based
// consumers read deltas between scrapes, which the warm-up does
// not distort past the very first interval.
Hits uint64
// Misses counts compositions performed, including raced duplicates
// whose entry was discarded (the work happened).
Misses uint64
// Inserts counts entries created — the driver-side proxy for
// distinct SQL texts handed to the driver (each is what a
// per-connection statement cache would prepare).
Inserts uint64
// Evictions counts second-chance evictions. A nonzero rate on a
// steady workload means the shape set outgrows Capacity.
Evictions uint64
Entries int // resident entries
Capacity int
// SQLBytes is Σ len(sql) over resident entries. The lock-free read
// path's lagging snapshot may briefly retain up to about twice
// this (see the ComposedCache.fast comment).
SQLBytes int64
}
CacheStats is a point-in-time snapshot of the cache's counters, taken under the cache mutex — a scrape-time cost, never a hot-path one. Counters are cumulative since the cache was created.
type ComposedCache ¶
type ComposedCache struct {
// contains filtered or unexported fields
}
ComposedCache memoizes composed SQL per (query, shape), bounded by a capacity with approximate-LRU eviction. Hits compare the full key, never just its string form.
Reads take no lock. Composition is deterministic and entries are immutable once published, so hits are served from an atomically published snapshot of the entry map; only misses — bounded in a healthy workload by the number of shapes an application uses, which is typically tiny — take the mutex.
Eviction is second-chance (CLOCK) rather than exact LRU: a lock-free hit records recency by setting a bit on its own entry instead of reordering shared state, and eviction skips once over entries whose bit is set. Recency is therefore approximate, which bounds memory exactly as strict LRU would while letting hits scale across cores.
func NewComposedCache ¶
func NewComposedCache(capacity int) *ComposedCache
func (*ComposedCache) GetBindsStyle ¶
func (c *ComposedCache) GetBindsStyle(style Style, queryName string, frags []Frag, key ShapeKey) (string, []Bind, error)
GetBindsStyle is GetStyle returning the full bind plan — needed when binds select slice elements (@in arity expansion).
func (*ComposedCache) GetStyle ¶
func (c *ComposedCache) GetStyle(style Style, queryName string, frags []Frag, key ShapeKey) (string, []int16)
GetStyle is Get with an explicit placeholder style.
func (*ComposedCache) GetTree ¶
func (c *ComposedCache) GetTree(queryName string, frags []Frag, key ShapeKey, tree Tree, caps TreeCaps) (string, []Bind, error)
GetTree is the @filter-tree variant: the tree's structural encoding becomes part of the cache key (values never do).
func (*ComposedCache) GetTreeStyle ¶
func (c *ComposedCache) GetTreeStyle(style Style, queryName string, frags []Frag, key ShapeKey, tree Tree, caps TreeCaps) (string, []Bind, error)
GetTreeStyle is GetTree with an explicit placeholder style.
func (*ComposedCache) SetMaxBytes ¶
func (c *ComposedCache) SetMaxBytes(n int64)
SetMaxBytes bounds the cache's approximate retained bytes (Σ composed SQL + bind plans + arg indices over resident entries) in addition to the entry-count capacity, evicting least-recently-used entries when the total would exceed it. A value <= 0 disables the byte bound (count cap only). Like SetObserver, call it before the cache serves traffic. The default is [defaultCacheMaxBytes]; it exists so a caller-controlled @in arity cannot pin unbounded memory behind a modest entry count.
func (*ComposedCache) SetObserver ¶
func (c *ComposedCache) SetObserver(o Observer)
SetObserver installs an observer receiving one ObserveCompose per cache access. The install is atomic, so calling it after the cache has begun serving traffic is safe (a racing hot-path read sees either the old or the new observer, never a torn value). It also enables hit counting (see Stats). Passing nil removes the observer.
func (*ComposedCache) Stats ¶
func (c *ComposedCache) Stats() CacheStats
Stats returns cumulative counters and current occupancy. It takes the cache mutex and walks resident entries: call it at scrape time, not on the query path.
func (*ComposedCache) TopShapes ¶
func (c *ComposedCache) TopShapes(n int) []ShapeUse
TopShapes returns the n most-used resident shapes, ordered by hits descending, then query, key, and SQL length ascending — a total order, so equal state yields byte-identical output. Like Stats, it is a scrape-time call.
type Expanded ¶
Expanded is one statically expanded shape: SQL and bind order were precomputed at generate time (via Compose, so byte-identical to what runtime composition would produce).
type Frag ¶
type Frag struct {
Kind Kind
Text string
ParamSpans []Span // :name token positions within Text
ParamIdx []int16 // flattened params-struct index per span
GuardMask uint64 // Guarded: all these bits must be set
Sep Sep
Cases []Case // Choose: named cases, then the default (if any)
// OrderBy: the keys
Default *Case // OrderBy: the @default clause body (may be nil)
}
Frag is one compile-time-constant fragment. Emitted by sqletch generate; never constructed by hand.
type Observer ¶
type Observer interface {
// ObserveCompose fires once per cache access: hit reports whether
// a memoized entry was served (true) or composition ran (false).
// Under a race, two compositions of one shape may both report
// hit=false while a single entry is inserted — compose events can
// exceed insertions, never undercount work. The key's slices are
// shared with the caller: copy them before retaining the key.
ObserveCompose(query string, key ShapeKey, hit bool)
// ObserveExec fires from generated code after a database call
// completes: the call's context (for trace correlation and metric
// exemplars), its duration, its row count (rows returned for
// queries, rows affected for execs; -1 when the count was unknown
// — a driver error or an aborted scan), and the driver error if
// any. It receives the shape key as its canonical encoding, not a
// ShapeKey: generated code builds its key on the stack, and
// passing that through an interface call would heap-allocate the
// key's slices on every call, observed or not — the encoding is
// built inside the observer guard instead, so only observed calls
// pay for it. (ObserveCompose can pass the cache's retained key
// for free, which is why the two differ. ObserveCompose carries no
// context because the cache API takes none.)
ObserveExec(ctx context.Context, query, shapeKey string, d time.Duration, rows int64, err error)
// ObserveReject fires from generated code when a call is refused
// before any SQL is sent: [ErrChooseRequired], [ErrOrderKey],
// [ErrFilterRequired], [ErrTreeTooLarge], [ErrTreePredicate],
// [ErrShapeKeyLimit]. Classify with errors.Is, never by message.
ObserveReject(ctx context.Context, query string, err error)
}
Observer receives runtime events from the composed-SQL cache and from generated code (design doc 18). Implementations must be safe for concurrent use and must return quickly — events fire on the query path. A nil observer costs one predictable branch per event site; format-specific export (OpenTelemetry, Prometheus) belongs in adapter modules, never here.
type ShapeKey ¶
type ShapeKey struct {
Guards uint64
Choices []uint8
// Orders holds one key sequence per @order-by block (elements are
// key<<1|desc). nil inner = maximal/all keys (verification only);
// empty = default-or-omit.
Orders [][]uint8
// Trees holds the canonical structural encoding of each
// @filter-tree value (values excluded) — the cache-key component
// for tree-shaped queries.
Trees []string
// Arities holds the element count of each @in slice, in template
// order — a shape dimension on expanding dialects only (empty on
// PostgreSQL, whose `= ANY` binds the slice whole).
Arities []int32
}
ShapeKey identifies one concrete query shape.
type ShapeSpaceInfo ¶
type ShapeSpaceInfo struct {
// Enumerable counts the reachable shapes of the enumerable
// dimensions (guard sets × @choose ordinals × @order-by
// selections), saturating at MaxUint64.
Enumerable uint64
// Exact is false when the true count exceeded Enumerable's range
// (large @order-by permutation spaces).
Exact bool
// Unbounded marks dimensions the count deliberately excludes —
// @filter-tree structure, and @in arity on expanding dialects —
// under which used/reachable coverage is a floor, never a ratio.
Unbounded bool
}
ShapeSpaceInfo describes one query's reachable shape space, computed at generate time and emitted into the generated package's ShapeSpace registry. It lets an exporter pre-register per-query series and compute used-vs-reachable coverage without parsing anything.
type ShapeUse ¶
type ShapeUse struct {
Query string
Key string // canonical ShapeKey encoding
// Hits counts accesses served by the resident entry; it resets if
// the shape is evicted and later recomposed.
Hits uint64
SQLBytes int // len of the composed SQL
}
ShapeUse is one row of the per-shape usage ranking.
type Span ¶
type Span struct{ Start, End int32 }
Span marks a :name parameter token inside a fragment's text.
type Style ¶
type Style uint8
Style is the dialect placeholder emission mode of a generated query, fixed at generation time.
type Tree ¶
type Tree struct {
// contains filtered or unexported fields
}
Tree is a runtime-composed boolean combination over a query's closed predicate vocabulary. Values are opaque to the composer: they travel exclusively as bind parameters. Construct trees only through the generated per-query predicate constructors plus And/Or/Unscoped.
Tree is a value type, not a pointer, so that `nil` is not a Tree — passing it to a required @filter-tree! argument does not compile:
cannot use nil as runtime.Tree value in argument to FilterUsers
That matters because `nil` is the shape a forgotten scope takes. The only zero Tree that survives compilation is one written out as `runtime.Tree{}`, which nobody types by accident; it is still refused at runtime with ErrFilterRequired, and Unscoped() remains the way to say "no scope" on purpose.
func NewLeaf ¶
NewLeaf is called by generated predicate constructors; user code never calls it directly. args must be exactly the predicate's parameters in declaration order — composition rejects any count mismatch with ErrTreeArity (generated constructors always match).
func Unscoped ¶
func Unscoped() Tree
Unscoped is the explicit, greppable opt-out of a required @filter-tree!: it renders as TRUE.
func (Tree) Encode ¶
Encode is the canonical structural encoding (values excluded) used in cache keys: leaves "p<idx>", nodes "&(...)"/"|(...)", TRUE "T".
The walk is unbounded recursion, so callers must have passed the tree through checkCaps first — the cache entry point does, and generated code only reaches Encode after a successful compose. Encode cannot enforce that itself: it does not receive the caps, and rejecting on a built-in bound would collapse the encoding of legitimate trees in projects that raised filter_tree_caps, which is a cache-key collision (distinct trees sharing composed SQL).