Documentation
¶
Index ¶
- Variables
- func IsTransactionConflict(err error) bool
- func New(profile Profile, options ...Option) dal.DB
- func NewBranchingProvider() branching.Provider
- func NewDB(options ...Option) dal.DBdeprecated
- func WithCollection[T any](name string, newRecord func() *T, opts ...CollectionOption) collectionDef
- type CollectionOption
- type ColumnOption
- type ColumnStrategy
- type Option
- func WithFirestoreProfile() Option
- func WithInterleavedReadsAndWritesInTransaction() Option
- func WithNoReadsAfterWritesInTransaction() Optiondeprecated
- func WithOptimisticConcurrency() Optiondeprecated
- func WithSchema(allowUndefinedCollections bool, collections ...collectionDef) Option
- func WithSingleWriterTransactions() Option
- func WithoutSchemaRefBreaking() Option
- type Profile
- type SlotSet
Constants ¶
This section is empty.
Variables ¶
var ErrNestedTransaction = errors.New("dalgo2memory: nested transactions are not supported: Firestore rejects a transaction started inside another transaction's callback")
ErrNestedTransaction is returned by RunReadonlyTransaction and RunReadwriteTransaction when either is called with a context that is already running inside another transaction's callback, in any combination of read-only and read-write.
This is Firestore parity, not an invented restriction. The real client (cloud.google.com/go/firestore, transaction.go) keeps one transactionInProgressKey and one RunTransaction method underneath both Client.RunTransaction (read-write) and the ReadOnly() option (read-only):
if ctx.Value(transactionInProgressKey{}) != nil {
return errNestedTransaction
}
That check runs before RunTransaction even looks at whether either side is read-only, so all four nestings — read-write-in-read-write, read-only-in-read-write, read-write-in-read-only, read-only-in-read-only — are rejected identically. dalgo2memory mirrors that scope exactly, rather than only guarding the read-write-in-read-write case, so code that runs against this adapter cannot rely on a nesting shape that would fail against real Firestore.
Independently of parity, dalgo2memory has its own reason to refuse read-write-in-read-write specifically: the default whole-database-lock RunReadwriteTransaction (runLockedReadwriteTransaction) holds db.mu for its callback's entire duration, so a nested RunReadwriteTransaction call would deadlock trying to re-acquire it (sync.RWMutex is not reentrant). The opt-in WithOptimisticConcurrency path (runOptimisticReadwriteTransaction) holds no such lock, so the same nested call would instead silently succeed as an independent, concurrently-committing transaction — legalizing exactly the pattern real Firestore rejects outright. This guard closes both wrong outcomes — a hang and a silent behavior change — before either can happen, for every mode combination.
A rejected nested attempt does not poison the outer transaction: this error is returned directly by the nested RunReadonlyTransaction / RunReadwriteTransaction call, before any transaction state for the nested attempt is created, exactly like errNestedTransaction is returned by the real client's RunTransaction with no effect on the outer *Transaction's fields. A callback that treats the error as fatal and returns it aborts the outer transaction (its writes are discarded, same as any other callback error); a callback that swallows it and returns nil lets the outer transaction commit its own work normally.
To intentionally start a genuinely independent transaction from inside a callback — rather than nesting one inside it — begin it with dal.GetNonTransactionalContext(ctx) in place of ctx: dal core's sanctioned escape for exactly this. It works here because it returns a context from before dalgo2memory (or whatever wraps it) ever set this key, so the guard sees no marker. It is only available when something upstream wrapped the context with dal.NewContextWithTransaction before handing it to this backend (e.g. an access.NewDB-style decorator) — dalgo2memory itself performs no such wrapping, so with nothing upstream doing it, GetNonTransactionalContext has nothing to return. Note that the escape only sidesteps THIS guard: started against the default whole-database-lock mode while the outer callback's db.mu is still held, an "independent" transaction on the same *database still deadlocks on that pre-existing, unrelated lock — the escape is only deadlock-free run against a WithOptimisticConcurrency database, or once the outer transaction has returned.
var ErrReadAfterWriteInTransaction = errors.New("firestore: read after write in transaction")
ErrReadAfterWriteInTransaction matches the ordering error returned by Firestore transactions when a read follows a queued write.
var ErrTransactionConflict = errors.New("dalgo2memory: transaction conflict: another transaction committed a write to a key this transaction read or wrote")
ErrTransactionConflict is returned by RunReadwriteTransaction, for a database created with WithOptimisticConcurrency, when another transaction committed a write to a key this transaction read or wrote before this transaction reached its own commit. A caller that wants to retry should test for it with IsTransactionConflict rather than a direct comparison, since the error returned always wraps additional diagnostic context.
Before adding this, both the dal and record packages (dalgo's own error vocabulary) were searched for an existing conflict / retryable / aborted error type or predicate to reuse instead — dalgo2mysql, dalgo2sql, dalgo2postgres and dalgo2sqlite were checked too, in case a convention already existed for a backend that has real serialization failures. None of them define one. This sentinel-plus-predicate pair follows the same idiom this package already uses for ErrReadAfterWriteInTransaction (a checked sentinel) and that the record package uses for ErrRecordNotFound/IsNotFound (a predicate function) — see IsTransactionConflict.
Functions ¶
func IsTransactionConflict ¶ added in v0.65.0
IsTransactionConflict reports whether err is, or wraps, ErrTransactionConflict. A caller of a WithOptimisticConcurrency database's RunReadwriteTransaction should use this to decide whether a failed transaction is safe to retry.
func New ¶ added in v0.74.0
New creates an in-memory DALgo database emulating the named backend.
The profile is required and nil panics: an in-memory test double is always standing in for something, and which something changes what the tests prove — see Profile's doc comment for why there is deliberately no default.
The backend is wrapped by dal.NewDB, so writes through the returned DB run the framework write pipeline — record validation and before-save hooks — before reaching the in-memory store.
func NewBranchingProvider ¶ added in v0.63.2
NewBranchingProvider returns the optional database branching capability for dalgo2memory's default serialized storage engine.
Columnar and custom storage engines are intentionally outside this capability. Capture reports them as branching.UnsupportedError instead of publishing an incomplete checkpoint.
func NewDB
deprecated
NewDB creates an in-memory DALgo database emulating Firestore.
Deprecated: use New with an explicit Profile instead. NewDB keeps the FirestoreProfile default for compatibility with existing callers, but a platform-independent library should not imply any vendor's transaction semantics without the call site naming it; NewDB will be removed once known consumers have migrated.
func WithCollection ¶
func WithCollection[T any](name string, newRecord func() *T, opts ...CollectionOption) collectionDef
WithCollection registers a collection backed by the concrete record type T.
If newRecord is nil, a zero value (new(T)) is used to materialize each record read by a query. Provide a factory to populate default field values instead.
Trailing CollectionOption arguments select a per-collection storage engine; with none, the collection uses the default Serialized engine.
Types ¶
type CollectionOption ¶
type CollectionOption func(*collectionDef)
CollectionOption configures a collection definition produced by WithCollection — currently the per-collection storage-engine selection. Pass it as a trailing argument to WithCollection.
func WithColumnarStorage ¶
func WithColumnarStorage(opts ...ColumnOption) CollectionOption
WithColumnarStorage selects the columnar storage engine for a schema-registered WithCollection[T] collection, with optional per-column strategies and a per-collection ref-breaking override. Selecting columnar storage for a schemaless or non-struct collection fails with a descriptive error when the collection is used.
func WithSerializedStorage ¶
func WithSerializedStorage() CollectionOption
WithSerializedStorage selects the Serialized storage engine for a collection. It is the default engine, so this option states the default explicitly; an option-less collection behaves identically.
type ColumnOption ¶
type ColumnOption func(*columnarConfig)
ColumnOption configures a single aspect of a columnar collection: it either supplies a ColumnStrategy for a named column (WithColumnStrategy) or sets the per-collection ref-breaking override (WithColumnarRefBreaking). It is passed to WithColumnarStorage. Exported so an out-of-core package can return one carrying its own ColumnStrategy without dalgo2memory importing it.
func WithColumnStrategy ¶
func WithColumnStrategy(name string, strategy ColumnStrategy) ColumnOption
WithColumnStrategy supplies a ColumnStrategy for the named column of a columnar collection. Columns without an explicit strategy use the default typed-slice strategy.
func WithColumnarRefBreaking ¶
func WithColumnarRefBreaking(refBreaking bool) ColumnOption
WithColumnarRefBreaking sets the per-collection ref-breaking override for a columnar collection, taking precedence over the schema-wide default (WithoutSchemaRefBreaking). Pass true to force faithful storage, false to store reference-bearing columns without the serialization round-trip.
func WithDeclaredColumn ¶
func WithDeclaredColumn[T any](name string) ColumnOption
WithDeclaredColumn declares a columnar column by name for a map-backed (map[string]any) collection, stored in a strongly-typed []T slice. At least one declared column is required to select columnar storage for a map-backed collection; undeclared fields are kept in a parallel leftover map. On a struct collection a declared column is accepted but redundant (the struct path reflects over the record type instead). When the same name is declared more than once, the last declaration wins.
type ColumnStrategy ¶
type ColumnStrategy interface {
// SetValue records that the column holds value at the given slot. The engine
// calls it on every write (insert/overwrite/update) for the column.
SetValue(slot int, value any)
// ClearValue records that the slot no longer holds a value for the column.
// The engine calls it on delete and during compaction rebuilds.
ClearValue(slot int)
// EqualSlots returns the live slots whose column equals value, with ok=true.
// Returning ok=false signals "no opinion": the engine falls back to scanning.
// Equality is the adapter's value equality (Go == on comparable decoded
// values, as in matchesWhere).
EqualSlots(value any) (slots SlotSet, ok bool)
}
ColumnStrategy backs a single column of a columnar collection. It is exported so an out-of-core package (e.g. a bitmap index) can supply a strategy via WithColumnStrategy without dalgo2memory importing it.
The engine remains the source of truth for stored values (its typed column slices); a strategy is an index kept in sync through the write side, used to accelerate the equality read side.
type Option ¶
type Option func(*database)
Option configures an in-memory database created by NewDB.
func WithFirestoreProfile ¶ added in v0.73.0
func WithFirestoreProfile() Option
WithFirestoreProfile names the backend a plain NewDB() emulates: Firestore. It re-asserts the full Firestore-faithful transaction bundle — strict read-before-write ordering, snapshot reads with genuine contention, atomic buffered commits, and bounded auto-retry of conflicts — and is therefore an affirming no-op on a plain NewDB(), useful in two ways: as documentation in a test that wants to SAY what it emulates rather than rely on defaults, and as a last-wins reset after earlier options in the same list.
Profiles name real backends rather than exposing isolation levels as free dials, because a test double configured into a combination no real backend has emulates nothing — see this package's option naming throughout. A SQL profile family (per-transaction isolation levels, read-your-writes, interleaved ordering) is planned to join it once the interleaved mode gains query overlay support; until then WithInterleavedReadsAndWritesInTransaction and WithSingleWriterTransactions are the SQL-flavoured building blocks.
func WithInterleavedReadsAndWritesInTransaction ¶ added in v0.67.0
func WithInterleavedReadsAndWritesInTransaction() Option
WithInterleavedReadsAndWritesInTransaction opts a database out of the default Firestore-compatible transaction-ordering check: with it, a read-write transaction may freely read a key after the transaction has written (to that key or any other), the same way a SQL database's session-local transaction behaves.
Use this when the in-memory database stands in for a backend whose transactions genuinely permit interleaving reads and writes — a SQL-style adapter such as dalgo2mysql, dalgo2postgres, or dalgo2sqlite, or a test double standing in for one of them. Do NOT reach for this just to make a failing test pass: if the code under test also runs against Firestore (dalgo2firestore) or another backend with the same read-after-write restriction, a test failing with ErrReadAfterWriteInTransaction is reporting a real ordering bug in the code under test — the same class of bug that shipped a production 500 in sneat-core-modules's set_user_country, undetected because its unit tests ran against the old permissive default. Silencing that signal with this option would hide the bug again, not fix it.
func WithNoReadsAfterWritesInTransaction
deprecated
added in
v0.63.0
func WithNoReadsAfterWritesInTransaction() Option
WithNoReadsAfterWritesInTransaction enables Firestore-compatible transaction ordering for this in-memory database. In a read-write transaction, every read after the first successful write returns ErrReadAfterWriteInTransaction.
Deprecated: this is now NewDB's default behavior, so calling this option is redundant on a plain NewDB(). It still works — it re-asserts strict ordering, which only matters when combined with an earlier WithInterleavedReadsAndWritesInTransaction() in the same option list, since options apply in order and the last one wins. New code should omit it and rely on the default; existing callers (sneat-co/chessraiders among them) are unaffected and may drop the call at their own pace.
func WithOptimisticConcurrency
deprecated
added in
v0.65.0
func WithOptimisticConcurrency() Option
WithOptimisticConcurrency selects optimistic-concurrency read-write transactions for this in-memory database.
Deprecated: this is now NewDB's default behavior, so calling this option is redundant on a plain NewDB(). It still works — it re-asserts optimistic concurrency, which only matters when combined with an earlier WithSingleWriterTransactions() in the same option list, since options apply in order and the last one wins. New code should omit it and rely on the default (or name it via WithFirestoreProfile); existing callers (sneat-co/chessraiders among them) are unaffected and may drop the call at their own pace. This follows the exact deprecation precedent of WithNoReadsAfterWritesInTransaction when strict ordering became the default.
The paragraphs below describe the machinery, which is now simply how a plain NewDB() behaves. Before this was the default, a test that claimed to prove a real concurrency guarantee — "two concurrent claims of a unique slug, exactly one wins", or "two concurrent bookings for the last remaining place, one is refused" — passed trivially against the old whole-database lock, because the two transactions it spawned could never actually run at the same time. It proved nothing about a database, like Firestore, whose transactions really do contend. Production code in this ecosystem relies on exactly this guarantee (see sneat-co/bookius's facade4bookius/booking.go, which does a Get-then-Insert inside RunReadwriteTransaction for slug uniqueness and for capacity), which is why contention is now the default.
In this mode, transactions may run concurrently: each buffers the keys it reads and the writes it makes locally, touching no shared storage until it commits (when its callback returns nil). At that point it fails with ErrTransactionConflict (test with IsTransactionConflict) if another transaction has committed a write to any key this one read or wrote since it first touched that key — whether this one only read the key or wrote it too. Buffering writes rather than applying them immediately is what makes the LOSING side of a race get the conflict error specifically, rather than some other error that happens to depend on write ordering: see optimisticState's doc comment in optimistic.go for the full reasoning.
A query (ExecuteQueryToRecordsReader / ExecuteQueryToRecordsetReader) inside such a transaction is supported and participates in the transaction's snapshot and conflict detection at COLLECTION granularity: the query registers its collection, aborts with ErrTransactionConflict if the collection was committed to after this transaction's snapshot, and the commit revalidates it — which is what makes phantom inserts conflict instead of slipping past the per-key read set (see optimisticState.observeCollectionAtSnapshot). Joins are the one refused shape (Firestore has none to stay faithful to). Point reads and writes by key (Get, Exists, Set, Insert, Update, Delete and their -Multi forms) are fully supported.
This is deliberately adapter-local rather than part of dalgotest's shared conformance suite: the suite proves record-validation invariants every dal.DB adapter can be held to uniformly, but optimistic-concurrency contention is not such a capability — a real backend like Firestore or a SQL database already has its own genuine transactional contention, and forcing every adapter to grow an equivalent option and test hook just to stay in the suite would be scope the other adapters never asked for.
Both modes buffer a transaction's writes and apply them only once its callback returns nil, so a failed transaction discards its writes exactly as Firestore does regardless of this choice (see runLockedReadwriteTransaction).
func WithSchema ¶
WithSchema registers per-collection record types so that queries return records populated into the concrete Go type of the collection.
allowUndefinedCollections controls what happens when a query targets a collection that is not part of the schema: when false (the default intent) such a query returns an error; when true it falls back to the schemaless behavior (map[string]any / keys-only records).
func WithSingleWriterTransactions ¶ added in v0.73.0
func WithSingleWriterTransactions() Option
WithSingleWriterTransactions opts a database out of the default contention-capable transaction machinery: RunReadwriteTransaction takes a whole-database lock for the callback's entire duration, so read-write transactions are fully serialized — a single writer at a time, the way SQLite's database-level write lock behaves. This was NewDB's default before contention was; atomicity is unaffected (writes are buffered and discarded on a failed callback in both modes), and transaction options like dal.TxWithAttempts are silently ignored, since a conflict can never occur.
Use this when the in-memory database stands in for a backend that genuinely serializes writers, or for a test that deliberately choreographs step-by-step transaction ordering and needs transactions never to abort. Do NOT reach for it just to make a failing concurrent test pass: if the code under test also runs against Firestore, a test failing with ErrTransactionConflict under the default is exercising real contention that production sees too — silencing it here hides the signal, the same trap WithInterleavedReadsAndWritesInTransaction's doc comment warns about for ordering.
func WithoutSchemaRefBreaking ¶
func WithoutSchemaRefBreaking() Option
WithoutSchemaRefBreaking disables columnar ref-breaking schema-wide: columnar collections store reference-bearing column values without the serialization round-trip unless a collection re-enables it (see WithColumnarRefBreaking). It has no effect on the always-faithful Serialized engine. The default is faithful (ref-breaking on).
type Profile ¶ added in v0.74.0
type Profile func(*database)
Profile names the real backend this in-memory database stands in for, and selects that backend's transaction-semantics bundle wholesale. It is a distinct type rather than an Option so the compiler enforces that every call to New names its backend: dalgo is platform-independent — it has adapters for Firestore, Datastore, MySQL, Postgres, SQLite and more — so no single vendor's transaction semantics is a neutral default for the test double, and a database configured into a combination no real backend has emulates nothing. Construct one with FirestoreProfile or SingleWriterProfile; a SQL profile family (per-transaction isolation levels, read-your-writes) will join them once the interleaved mode's query support exists — profile names are never published before their semantics.
Options passed to New apply AFTER the profile, in order, last-wins — so the intent-named options remain available as narrow, explicit modifiers on top of a named backend's bundle.
func FirestoreProfile ¶ added in v0.74.0
func FirestoreProfile() Profile
FirestoreProfile emulates Google Cloud Firestore's transaction semantics: serializable isolation with snapshot reads (a fractured view aborts at the read), genuine contention between concurrent read-write transactions, atomic buffered commits, strict read-before-write ordering (a rejected read also poisons the commit), transactional queries with phantom protection, nested transactions rejected, and bounded auto-retry of conflicts with final-attempt lock escalation — matching the Firestore Go client's own silent retry of aborted transactions.
func SingleWriterProfile ¶ added in v0.74.0
func SingleWriterProfile() Profile
SingleWriterProfile emulates a backend that serializes writers behind a database-level write lock, the way SQLite behaves: RunReadwriteTransaction holds a whole-database lock for the callback's entire duration, so read-write transactions can never contend and never abort. Atomicity and strict read-before-write ordering are retained; transaction options such as dal.TxWithAttempts are silently ignored, since a conflict cannot occur. Choose it for backends that genuinely serialize writers, or for tests that deliberately choreograph step-by-step transaction ordering — never to silence contention failures in code that also runs against Firestore.
type SlotSet ¶
type SlotSet map[int]struct{}
SlotSet is a set of per-row slot indices. The columnar engine assigns each live record a stable slot shared across all of a collection's column slices; a ColumnStrategy's equality read side returns the slots whose column equals a queried value. A set (rather than a slice) is returned so that, when the adapter grows multi-predicate AND WHERE, per-predicate sets can be intersected.
Source Files