Documentation
¶
Overview ¶
Package cache holds the committed oracle cache and the catalog snapshot model. In P3 only the catalog data model is needed (the resolver and star expansion consume it); the oracle fills it from a live database in P4. See docs/design/04-type-oracle.md §3.
Index ¶
- Constants
- func EncodeCatalog(cat *Catalog) ([]byte, error)
- func EncodeEnv(e *Env) ([]byte, error)
- func EncodeOracle(e *OracleEntry) ([]byte, error)
- func Fingerprint(dialectName, serverVersion string, files []SchemaFile) string
- func NumericVersionPrefix(raw string) string
- func OracleFileName(r OracleRef) string
- func ReadFileCapped(path string) ([]byte, error)
- func WriteFileAtomic(path string, data []byte, perm os.FileMode) error
- type Catalog
- type Column
- type EntryColumn
- type EntryType
- type Env
- type OracleEntry
- type OracleRef
- type SchemaFile
- type Store
- func (s *Store) CheckWritable(refs []OracleRef) error
- func (s *Store) LoadCatalog(fp string) (*Catalog, bool)
- func (s *Store) LoadEnv(fp string) (*Env, bool)
- func (s *Store) LoadOracle(r OracleRef, fp, renderedSQL string) (*OracleEntry, bool)
- func (s *Store) SaveCatalog(cat *Catalog) error
- func (s *Store) SaveEnv(e *Env) error
- func (s *Store) SaveOracle(r OracleRef, e *OracleEntry) error
- func (s *Store) Sweep(live map[OracleRef]bool) error
- func (s *Store) Walk(fn func(ref OracleRef, path string) error) error
- type SymlinkError
- type Table
Constants ¶
const ( CatalogFile = "catalog.json" EnvFile = "env.json" // OracleDir holds one file per verified rendering. OracleDir = "oracle" )
CatalogFile and EnvFile are the store's two singleton files. Neither name carries the schema fingerprint: it is compared from INSIDE the file, so a schema change modifies these files rather than renaming them, and one committed cache describes one schema state (docs/design/21-cache-layout.md D1).
const FormatVersion = 2
FormatVersion is the on-disk cache format. Every written file carries it; loads treat any other value — including its absence in pre-1.0 caches — as a miss, so a format change can never misread an old entry: the pipeline falls back to the database and rewrites.
v2 is the doc-21 layout: named paths instead of fingerprint-derived hashes. The bump is what migrates an existing committed cache — every v1 file is a miss, the first `generate` rewrites the tree, and the same run's sweep removes what the old layout left behind.
const MaxFileBytes = 64 << 20
MaxFileBytes bounds every cache file sqletch reads. Cache file names are derived from the config and the templates, hence fully attacker-computable (doc 21 made them more predictable, not less): a cloned repo can plant a file at the exact hit path, so an unbounded os.ReadFile would OOM before any key check runs. 64 MiB dwarfs any real catalog/oracle entry while capping the blast radius (mirrors the LSP body cap).
Variables ¶
This section is empty.
Functions ¶
func EncodeCatalog ¶
EncodeCatalog returns the exact canonical bytes SaveCatalog writes. It stamps FormatVersion. Anything that compares against a committed catalog file byte-wise (the oracle corpus harness) must serialize through here, never through its own marshaling.
func EncodeEnv ¶
EncodeEnv returns the exact canonical bytes SaveEnv writes, stamping FormatVersion.
func EncodeOracle ¶
func EncodeOracle(e *OracleEntry) ([]byte, error)
EncodeOracle returns the exact canonical bytes SaveOracle writes, stamping FormatVersion — the byte form the corpus harness compares.
func Fingerprint ¶
func Fingerprint(dialectName, serverVersion string, files []SchemaFile) string
Fingerprint is the offline-computable schema identity: sha256 over (dialect, pinned server version, ordered schema inputs).
func NumericVersionPrefix ¶
NumericVersionPrefix reduces a server-reported version string to the value drift detection compares: its leading dotted-numeric run.
Servers spell the same version differently depending on how they were built — PostgreSQL reports "16.4 (Debian 16.4-1.pgdg120+1)" on the Debian images and a bare "16.4" on Alpine, MySQL appends "-log". Comparing raw strings would report a base-image change as an environment drift, which is noise, not signal.
func OracleFileName ¶
OracleFileName is the ref's store-relative, slash-separated path.
Every component is folded to [A-Za-z0-9_-]. The grammar already restricts query names (SQLETCH002/003) and construct parameters to that alphabet and Slug folds the target, so the folding here is defence in depth: these strings become filesystem paths, and no spelling of them may climb out of the cache directory.
func ReadFileCapped ¶
ReadFileCapped reads path but refuses more than MaxFileBytes, so an attacker-planted giant file at a computable cache path cannot OOM the process. It reads at most MaxFileBytes+1 and rejects if that ceiling is reached, so the bound holds even if the file grows after an initial stat (no size TOCTOU).
func WriteFileAtomic ¶
WriteFileAtomic writes data to path atomically without ever following a symlink at path or at a predictable temp name. A cloned repo can pre-plant `<path>.tmp` (a computable name) as a symlink to a secret or config file; writing through it and renaming over the target would corrupt an arbitrary location. os.CreateTemp opens with O_CREATE|O_EXCL and a RANDOM suffix, so it neither follows nor collides with any planted link; the final os.Rename replaces a symlink sitting at path with our regular file rather than writing through it.
Types ¶
type Catalog ¶
type Catalog struct {
Format int `json:"format"`
SchemaFP string `json:"schema_fp"`
Tables []Table `json:"tables"`
}
Catalog is an offline snapshot of the schema portions sqletch needs: relation and column existence, types, NOT NULL, and defaults.
func (*Catalog) Lookup ¶
Lookup finds a table by unqualified name, preferring the "public" schema on ties. Returns nil when absent.
func (*Catalog) LookupQualified ¶
LookupQualified finds a table by an explicit schema qualifier, or falls back to Lookup's unqualified resolution when schema is empty. An explicitly qualified name never falls back: resolving it to a same-named table of another schema is exactly the confusion the nullability analysis must not inherit.
type EntryColumn ¶
type EntryColumn struct {
Name string `json:"name"`
OID uint32 `json:"oid"`
TypeName string `json:"type_name"`
// TypeSchema/TypeKind/TypeLabels mirror EntryType's
// user-defined-type fields for the column's type.
TypeSchema string `json:"type_schema,omitempty"`
TypeKind string `json:"type_kind,omitempty"`
TypeLabels []string `json:"type_labels,omitempty"`
SrcRel uint32 `json:"src_rel,omitempty"`
SrcAtt int16 `json:"src_att,omitempty"`
}
EntryColumn records what the ORACLE said about a result column — and nothing derived: nullability verdicts are recomputed from the catalog and the parse tree on every run (design 05 §4), so they never enter these byte-pinned files. (A vestigial always-false "nullable" field was removed 2026-08 without a FormatVersion bump: old entries still decode — v1 tolerates the extra field — and re-derivation gates were regenerated.)
type EntryType ¶
type EntryType struct {
OID uint32 `json:"oid"`
Name string `json:"name"`
Schema string `json:"schema,omitempty"`
Kind string `json:"kind,omitempty"`
Labels []string `json:"labels,omitempty"`
}
EntryType / EntryColumn mirror dialect.TypeRef / dialect.ColumnDesc without importing the dialect package (cache is a leaf; dialect imports cache for the Catalog model).
The user-defined-type fields (design 26 §4.2) are omitempty: an entry whose types are all built-in serializes exactly as before they existed, and no MySQL/SQLite entry ever sets them.
type Env ¶
type Env struct {
Format int `json:"format"`
SchemaFP string `json:"schema_fp"`
// Dialect and OracleBackend are recorded for forensics only. The
// dialect is already fingerprint input, and the backend is
// deliberately NOT compared: server and native backends are
// required to produce byte-identical output, so a backend
// difference is either a no-op or a sqletch bug for the corpus
// gates to catch — never a reason to fail a user's build.
Dialect string `json:"dialect"`
OracleBackend string `json:"oracle_backend"`
// ServerVersion is the compared value: the leading dotted-numeric
// run of what the server reported (see NumericVersionPrefix).
ServerVersion string `json:"server_version"`
// ServerVersionRaw is the full reported string, kept so the
// diagnostic can name the actual builds involved.
ServerVersionRaw string `json:"server_version_raw"`
}
Env is the sidecar record of the environment a committed cache was generated in.
It is deliberately NOT a cache key. The schema fingerprint must stay offline-computable (spec requirement), so the version of a server we have not contacted cannot enter it; and the bytes of catalog and oracle entries are pinned byte-identical across oracle backends by internal/corpus, so nothing backend- or connection-specific may enter those files either. The sidecar therefore lives beside them, keyed by the same fingerprint, and is read only by runs that contact a server anyway — where it answers one question the cache otherwise cannot: "was what I am connected to the same thing that produced these entries?" (docs/design/04-type-oracle.md §3.1).
Only facts that are semantically part of the oracle's answer belong here. Host names, user names, and timestamps do not: they would churn the committed diff on every developer's machine and break the project's determinism invariant.
type OracleEntry ¶
type OracleEntry struct {
Format int `json:"format"`
SchemaFP string `json:"schema_fp"`
RenderedSQL string `json:"rendered_sql"`
Params []EntryType `json:"params"`
Columns []EntryColumn `json:"columns"`
}
OracleEntry is one cached Describe result, self-describing with its full keys.
type OracleRef ¶ added in v0.0.3
type OracleRef struct {
Target string // config.ResolvedTarget.Slug(); may contain '/'
Query string
Shape string // ast.Rendering.Shape
}
OracleRef is an entry's place in the committed tree: the target slug, the query name, and the rendering's shape name (doc 21 §3).
It is a NAME, never a key. LoadOracle still compares the fingerprint and the rendered SQL stored inside the file, so a file sitting at a ref's path that does not match them is a miss and gets rewritten — exactly as a stale hash-named entry was under v1. Nothing downstream may treat "found at this path" as evidence of what an entry is.
type SchemaFile ¶
SchemaFile is one ordered schema input contributing to the fingerprint.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is the committed, offline-usable cache of oracle results and catalog snapshots. A path is an index, never identity: every entry stores its full inputs and loads compare them byte-wise (store-and-compare; design 04 §3, layout in design 21).
func (*Store) CheckWritable ¶ added in v0.0.9
CheckWritable reports, without writing anything, the first symlinked directory on the way to OracleDir or to any of refs' entries (a *SymlinkError), so a caller can refuse BEFORE the first write and leave the committed tree untouched. Missing directories are fine (the write creates them), and so is a symlink at an entry's own path: the atomic rename replaces it rather than writing through it.
The writes enforce the same rule themselves; this is the early, all-or-nothing form of it.
func (*Store) LoadCatalog ¶
LoadCatalog returns the snapshot for fp, or ok=false on miss or key mismatch.
func (*Store) LoadEnv ¶
LoadEnv returns the recorded generation environment for fp.
Every way of not having a usable record — absent file, unreadable file, format drift, key mismatch — returns ok=false, which callers must treat as "no record yet", never as a failure. Caches committed before this sidecar existed have no record, and adopting the connected server on the next write is the correct migration.
func (*Store) LoadOracle ¶
func (s *Store) LoadOracle(r OracleRef, fp, renderedSQL string) (*OracleEntry, bool)
LoadOracle returns the cached Describe result for (fp, renderedSQL) from the entry named by ref, comparing the stored full keys (the path is where to look, never what the file is).
func (*Store) SaveCatalog ¶
func (*Store) SaveOracle ¶
func (s *Store) SaveOracle(r OracleRef, e *OracleEntry) error
func (*Store) Sweep ¶ added in v0.0.3
Sweep deletes everything in the store that live does not account for: entries whose ref is absent, and the root files superseded layouts wrote. The two singletons are the store's own and are never removed.
It touches only files the store itself writes — regular `.json` files under OracleDir, plus legacyRootFile matches at the root — so a `.gitignore`, a README, a sibling tool's data file, and every other subdirectory survive. A planted symlink is left alone rather than followed. That discipline matters because the cache directory is configured: pointed at a directory somebody else also owns, a broader sweep would eat their files.
Directories emptied by the sweep are removed, so a renamed target or query leaves no husk.
func (*Store) Walk ¶ added in v0.0.3
Walk calls fn for every oracle entry in the store, with the ref that names it and its absolute path, in a deterministic order (lexical, parents before children — filepath.WalkDir's).
The store owns its own tree: anything that discovers entries rather than addressing one — the oracle corpus, a test — goes through here instead of re-deriving the layout. Files that are not entry paths (a README, an older layout's leftovers) are skipped; an absent tree is not an error.
type SymlinkError ¶ added in v0.0.9
type SymlinkError struct {
Dir string // the cache directory
Rel string // the offending component, store-relative, slash form
}
SymlinkError is the store's refusal to write through a symbolic link that sits on a DIRECTORY between the cache root and an entry. The cache tree is committed, hence repo-controlled: a planted `oracle -> /elsewhere` (or a symlinked slug/query directory) would otherwise have every miss-fill create and overwrite files outside the project, which Walk and Sweep — they never follow links — could not even see to clean up. The cli layer reports it as SQLETCH306.
func (*SymlinkError) Error ¶ added in v0.0.9
func (e *SymlinkError) Error() string
type Table ¶
type Table struct {
Schema string `json:"schema"`
Name string `json:"name"`
OID uint32 `json:"oid"`
// HasChildren marks a plain-inheritance parent (PostgreSQL:
// relhassubclass on relkind 'r'). Children may DROP an inherited
// NOT NULL (proven on PG 16), so a parent scan can return NULL
// where attnotnull says otherwise — the analyzer must not narrow
// such tables unless the reference is `FROM ONLY`. Partitioned
// parents ('p') are exempt: partitions cannot drop inherited NOT
// NULL (42P16). omitempty keeps every inheritance-free catalog
// byte-identical.
HasChildren bool `json:"has_children,omitempty"`
// IsView marks a relation that is a VIEW rather than a base table.
// SQLite's column-origin attribution (sqlite3_column_origin_name)
// resolves a view's result columns THROUGH to the view's base
// tables, whose declared NOT NULL the view's (invisible, possibly
// null-extending) body need not preserve — so a base table appearing
// directly in FROM must not be allowed to vouch for a column that
// actually flows through a view. The nullability analyzer treats any
// view in play as a wholesale narrowing kill-switch. The PostgreSQL
// (relkind v/m) and MySQL server (table_type VIEW) snapshots record it
// too (their view-provenance fix), and design 20 relies on it on
// every dialect: a view's column NOT NULL does not carry its base
// column's constraint, so a write into a view never derives a
// nullable parameter. The native MySQL catalog refuses CREATE VIEW.
// omitempty keeps view-free catalogs byte-identical.
IsView bool `json:"is_view,omitempty"`
Cols []Column `json:"cols"`
}