README
¶
dalgo2sqlite
SQLite-specific DALgo driver. Wraps github.com/dal-go/dalgo2sql to provide the dal.DB surface, and adds SQLite-native implementations of:
dbschema.Adapter— schema introspection viasqlite_masterandpragma_table_infoddl.Applier— SQLite-flavoredCREATE TABLE/CREATE INDEX/DROP TABLEetc.dal.ConcurrencyAware— advertisesConcurrency() = 1for write paths (SQLite is single-writer)
Used by consumers (e.g. datatug-cli's db copy) that need schema-modification and concurrency hints through the unified DALgo abstraction without hand-rolling engine-specific SQL.
Our approach to development
We build with our own tooling:
- SpecScore — specify requirements as
SpecScore.mdartifacts - SpecStudio — author & manage specs across their lifecycle
- inGitDB — store structured data in Git where applicable
- DALgo — data access layer for Go
- cover100.dev — drive toward 100% test coverage
- DataTug — query & explore data
SQLite driver: modernc.org/sqlite (pure Go, CGO_ENABLED=0)
This package uses modernc.org/sqlite — a pure-Go transpilation of the
SQLite C library — instead of the former github.com/mattn/go-sqlite3 cgo
binding. The migration was done because:
- The cgo driver required a C toolchain in every build environment (CI, containers, cross-compilation targets).
- It prevented single static binaries for downstream consumers such as
datatug-cli. modernc.org/sqliteexposes the samedatabase/sqlinterface (driver name"sqlite"instead of"sqlite3"), sodalgo2sqlcontinues to work unchanged.- All existing SQLite features relied on here —
sqlite_masterintrospection,PRAGMAtable/index queries, transactional DDL — behave identically under the pure-Go driver.
The cgo_enabled: true flag can be removed from this repo's CI workflow and
from any downstream workflow that no longer needs cgo for other reasons.
TugQL JOIN acceptance
The checked-in testdata/joins directory contains a small Chinook-shaped
SQLite seed, the canonical nested Invoice/Customer/Employee TugQL query, and
expected rows. The test loads the seed into a temporary database, runs the
query through both the database and read-transaction entry points, and checks
ordered results and LEFT JOIN null extension. A second query carries independent
algorithm preferences on outer, nested, and sibling JOINs; SQLite's native
execution may ignore those physical preferences and must return the same rows.
The tests also check the canonical fixture hashes, their DALgo source commit,
and the expected diagnostics for malformed JOIN references.
Run the complete acceptance journey from this repository's root:
go test ./end2end -run 'TestTugQL(NestedJoinChinook|JoinFixtureManifest|JoinNegativeFixtures)' -count=1
Documentation
¶
Overview ¶
Package dalgo2sqlite is the SQLite-specific DALgo driver.
It composes github.com/dal-go/dalgo2sql for the dal.DB read/write surface (transactions, recordset reader, Get/Set/Insert/Delete) and adds SQLite-native implementations of:
- dbschema.SchemaReader for schema introspection via sqlite_master and PRAGMA queries
- ddl.SchemaModifier for SQLite-flavored CREATE / DROP / ALTER
- ddl.TransactionalDDL (always true — SQLite supports transactional DDL)
- dal.ConcurrencyAware returning false (SQLite serializes writers)
Index ¶
- Constants
- func IsAlreadyExists(err error) bool
- type Database
- func (d *Database) Adapter() dal.Adapter
- func (d *Database) AlterCollection(ctx context.Context, name string, ops ...ddl.AlterOp) error
- func (d *Database) CheckSourceConstraints(ctx context.Context) (resultErr error)
- func (d *Database) Close() error
- func (d *Database) CreateCollection(ctx context.Context, c dbschema.CollectionDef, opts ...ddl.Option) error
- func (d *Database) Delete(ctx context.Context, key *dalrecord.Key) error
- func (d *Database) DeleteMulti(ctx context.Context, keys []*dalrecord.Key) error
- func (d *Database) DescribeCollection(ctx context.Context, ref *dal.CollectionRef) (*dbschema.CollectionDef, error)
- func (d *Database) DropCollection(ctx context.Context, name string, opts ...ddl.Option) error
- func (d *Database) ExecuteQueryToRecordsReader(ctx context.Context, query dal.Query) (dal.RecordsReader, error)
- func (d *Database) ExecuteQueryToRecordsetReader(ctx context.Context, query dal.Query, opts ...recordset.Option) (dal.RecordsetReader, error)
- func (d *Database) Exists(ctx context.Context, key *dalrecord.Key) (bool, error)
- func (d *Database) Get(ctx context.Context, record dalrecord.Record) error
- func (d *Database) GetMulti(ctx context.Context, records []dalrecord.Record) error
- func (d *Database) ID() string
- func (d *Database) Insert(ctx context.Context, record dalrecord.Record, opts ...dal.InsertOption) error
- func (d *Database) LenientSchema() *Database
- func (d *Database) ListCollections(ctx context.Context, parent *record.Key) ([]dal.CollectionRef, error)
- func (d *Database) ListConstraints(ctx context.Context, ref *dal.CollectionRef) ([]dbschema.ConstraintDef, error)
- func (d *Database) ListIndexes(ctx context.Context, ref *dal.CollectionRef) ([]dbschema.IndexDef, error)
- func (d *Database) ListReferrers(ctx context.Context, ref *dal.CollectionRef) ([]dbschema.Referrer, error)
- func (d *Database) ListSourceViews(ctx context.Context) ([]dbschema.SourceViewDef, error)
- func (d *Database) ListViews(ctx context.Context) ([]dal.CollectionRef, error)
- func (d *Database) OpenSourceRows(ctx context.Context, ref *dal.CollectionRef) (dbschema.SourceRowCursor, error)
- func (d *Database) RunReadonlyTransaction(ctx context.Context, f dal.ROTxWorker, opts ...dal.TransactionOption) error
- func (d *Database) RunReadwriteTransaction(ctx context.Context, f dal.RWTxWorker, opts ...dal.TransactionOption) error
- func (d *Database) Schema() dal.Schema
- func (d *Database) Set(ctx context.Context, record dalrecord.Record) error
- func (d *Database) SetMulti(ctx context.Context, records []dalrecord.Record) error
- func (d *Database) SupportsConcurrentConnections() bool
- func (d *Database) SupportsTransactionalDDL() bool
- func (d *Database) UnnamedObjects(ctx context.Context) ([]string, error)
- func (d *Database) Update(ctx context.Context, key *dalrecord.Key, updates []update.Update, ...) error
- func (d *Database) UpdateMulti(ctx context.Context, keys []*dalrecord.Key, updates []update.Update, ...) error
- func (d *Database) UpdateRecord(ctx context.Context, record dalrecord.Record, updates []update.Update, ...) error
- func (d *Database) Upsert(ctx context.Context, record dalrecord.Record) error
Constants ¶
const Version = "0.1.0"
Version is the dalgo2sqlite package version. Updated by hand on each release; consumed by Adapter.Version().
Variables ¶
This section is empty.
Functions ¶
func IsAlreadyExists ¶ added in v0.1.6
IsAlreadyExists reports whether err — the raw error modernc.org/sqlite (the pure-Go SQLite driver this adapter registers; see database.go) returns for a failed INSERT — represents a duplicate-key violation: a duplicate primary key or a duplicate value in a UNIQUE index.
It is dalgo2sqlite's implementation of the github.com/dal-go/dalgo2sql.DbOptions.IsAlreadyExists hook, which dalgo2sql itself cannot supply because detecting a duplicate key is driver-specific. NewDatabase and NewDatabaseWithOptions wire it in by default whenever the caller-supplied DbOptions leaves IsAlreadyExists nil (see database.go), so ordinary use needs no extra configuration. It is exported so a caller assembling a custom dalgo2sql.DbOptions can still reference it directly — e.g. to compose it with another classifier, or to confirm what it matches.
It matches only on errors.As(err, *sqlite.Error) and that error's extended result code — never on message text, which is not a stable contract across modernc.org/sqlite versions or SQLite builds. The primary result code SQLITE_CONSTRAINT (19) is deliberately not matched: it also covers NOT NULL, CHECK, and FOREIGN KEY violations, none of which are duplicate keys, so matching it would misclassify them.
Types ¶
type Database ¶
type Database struct {
dal.NoConcurrency // SupportsConcurrentConnections() = false
dal.DB // delegate for the dal.DB surface
// contains filtered or unexported fields
}
Database is the dalgo2sqlite driver instance. It implements dal.DB by embedding a dal.DB obtained from dalgo2sql.NewDatabase, and adds SQLite-specific dbschema, ddl, and concurrency surfaces.
The embedded dal.DB (rather than a named field) is what lets Database satisfy dal.DB itself: dal.DB is sealed by an unexported marker method, and embedding is the only way for that method to be promoted onto a decorating type — see dal.NewDB's doc comment.
Construct via NewDatabase. Database values are safe for concurrent use only insofar as SQLite itself is — readers can be concurrent under WAL mode; writers serialize.
func NewDatabase ¶
NewDatabase opens (or creates) the SQLite file at dbPath using modernc.org/sqlite (pure Go, CGO_ENABLED=0), pings to surface malformed-file errors at construction time, wraps the *sql.DB via dalgo2sql.NewDatabase for the dal.DB surface, and returns a *Database that satisfies dal.DB + dal.ConcurrencyAware.
Use NewDatabaseWithOptions when you need to supply per-collection primary-key metadata (required for Insert/Get/Delete with map[string]any data).
func NewDatabaseWithOptions ¶ added in v0.0.17
func (*Database) AlterCollection ¶
AlterCollection applies ops in order inside a single transaction. Partial failures roll back and leave the collection untouched.
func (*Database) CheckSourceConstraints ¶ added in v0.3.0
CheckSourceConstraints asks SQLite to verify the source's table/index/CHECK integrity and foreign keys with SQLite's own affinity and collation rules. It checks stored rows; it does not claim the export target enforces them.
func (*Database) Close ¶
Close closes the underlying *sql.DB. After Close the Database value is unusable; further method calls will fail with an error from database/sql.
func (*Database) CreateCollection ¶
func (d *Database) CreateCollection(ctx context.Context, c dbschema.CollectionDef, opts ...ddl.Option) error
CreateCollection creates a table and its inline indexes transactionally. On any error, the transaction rolls back and no schema state remains.
func (*Database) DeleteMulti ¶
func (*Database) DescribeCollection ¶
func (d *Database) DescribeCollection(ctx context.Context, ref *dal.CollectionRef) (*dbschema.CollectionDef, error)
DescribeCollection is implemented in T15.
func (*Database) DropCollection ¶
DropCollection drops the table; SQLite cascades to its indexes.
func (*Database) ExecuteQueryToRecordsReader ¶
func (*Database) ExecuteQueryToRecordsetReader ¶
func (*Database) LenientSchema ¶ added in v0.4.8
LenientSchema returns a Database over the same connection whose schema reads describe a file the way SQLite itself lists it, for a reader that shows a database to a person (a scan, a viewer) rather than copies it:
- DescribeCollection describes a view too, by the columns SQLite lists for it;
- a column whose declared type has no portable type (none at all, or one such as JSON or UUID) is dbschema.Null instead of failing the description, and its declared type stays in SourceDefinition;
- generated columns and the hidden columns of a virtual table are left out of the description, as PRAGMA table_info leaves them out, instead of failing it;
- a table or a view with an empty name is left out of ListCollections and ListViews (UnnamedObjects lists them), where the strict reader refuses it;
- an index on a column the description leaves out is left out of Indexes and ListIndexes; SourceDefinition.Indexes keeps it.
The Database it is called on is unchanged: its schema reads still refuse what a lossless export cannot represent. The two share one connection pool, so closing either closes both: close the Database that owns the pool, not the lenient one.
func (*Database) ListCollections ¶
func (d *Database) ListCollections(ctx context.Context, parent *record.Key) ([]dal.CollectionRef, error)
ListCollections returns the user-defined tables in alphabetical order. The parent *record.Key is ignored — SQLite has no catalog/schema hierarchy.
func (*Database) ListConstraints ¶
func (d *Database) ListConstraints(ctx context.Context, ref *dal.CollectionRef) ([]dbschema.ConstraintDef, error)
ListConstraints returns a best-effort survey of constraints on the table:
- The primary-key constraint (one row if any PK columns exist)
- Foreign-key constraints from PRAGMA foreign_key_list
- UNIQUE constraints, each named as the index SQLite made for it (origin "u" in PRAGMA index_list): ConstraintDef holds no columns, so a constraint's columns are those of the index of its name in ListIndexes
CHECK clauses and inline NOT NULL constraints are NOT enumerated (SQLite doesn't expose CHECK source portably). Callers read those from DescribeCollection.Fields.
func (*Database) ListIndexes ¶
func (d *Database) ListIndexes(ctx context.Context, ref *dal.CollectionRef) ([]dbschema.IndexDef, error)
ListIndexes is implemented in T16.
func (*Database) ListReferrers ¶
func (d *Database) ListReferrers(ctx context.Context, ref *dal.CollectionRef) ([]dbschema.Referrer, error)
ListReferrers performs an O(N) scan: for each other user-defined table, query PRAGMA foreign_key_list and check whether any row references ref.Name.
func (*Database) ListSourceViews ¶ added in v0.3.0
ListSourceViews lists view definitions once per database. View SQL is descriptive metadata; views are not materialized as collection records.
func (*Database) ListViews ¶ added in v0.4.8
ListViews returns the views of the database in alphabetical order; a view with an empty name is treated as ListCollections treats a table with one. A view is not among the collections ListCollections returns; on a Database made by LenientSchema, DescribeCollection describes one.
func (*Database) OpenSourceRows ¶ added in v0.3.0
func (d *Database) OpenSourceRows(ctx context.Context, ref *dal.CollectionRef) (dbschema.SourceRowCursor, error)
OpenSourceRows streams every native table row with its SQLite storage classes. A declared NUMERIC/DECIMAL value stored as REAL is rendered as the shortest decimal string that round-trips to its stored binary64 value; SQLite cannot recover decimal digits that were lost on initial insertion.
func (*Database) RunReadonlyTransaction ¶
func (d *Database) RunReadonlyTransaction(ctx context.Context, f dal.ROTxWorker, opts ...dal.TransactionOption) error
func (*Database) RunReadwriteTransaction ¶
func (d *Database) RunReadwriteTransaction(ctx context.Context, f dal.RWTxWorker, opts ...dal.TransactionOption) error
func (*Database) SupportsConcurrentConnections ¶ added in v0.1.0
SupportsConcurrentConnections reports SQLite's own concurrency behaviour (always false — see dal.NoConcurrency), not dalgo2sql's. An explicit method is required here: dal.NoConcurrency and the embedded dal.DB (whose Backend requirement embeds dal.ConcurrencyAware) both declare this method at the same promotion depth, which Go otherwise treats as an ambiguous selector.
func (*Database) SupportsTransactionalDDL ¶
SupportsTransactionalDDL reports that SQLite supports transactional DDL — every CREATE / DROP / ALTER statement can be wrapped in a BEGIN/COMMIT and is rolled back atomically on commit failure.
func (*Database) UnnamedObjects ¶ added in v0.4.8
UnnamedObjects lists the kind ("table" or "view") of each table and view of the database whose name is empty, which SQLite allows and no reference can hold: the lenient reader leaves them out of its listings, and its caller can say so.
func (*Database) UpdateMulti ¶
func (*Database) UpdateRecord ¶
func (d *Database) UpdateRecord(ctx context.Context, record dalrecord.Record, updates []update.Update, preconditions ...dal.Precondition) error
UpdateRecord is not supported at the database level by dalgo2sql; use Update with an explicit key instead, or call UpdateRecord inside a RunReadwriteTransaction where the transaction object does support it.
Source Files