dalgo2ingitdb

package module
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 15, 2026 License: MIT Imports: 29 Imported by: 0

Documentation

Index

Constants

View Source
const DatabaseID = "dalgo2ingitdb"

DatabaseID is the name reported by Database.ID() and used as the Adapter name.

View Source
const IDColumn = "$id"

IDColumn is the reserved recordset column that carries each record's key. It is not a declared schema column; the Starlark evaluator strips it from the stored field map so formula inputs match the eager ApplyFormulasToRead pipeline exactly. The "$id" spelling mirrors the dal pseudo-field convention and cannot collide with a real column (Starlark identifiers cannot contain "$", so no formula can reference it).

Variables

View Source
var ErrCollectionPathConflict = errors.New("dalgo2ingitdb: root-collections.yaml entry conflicts with auto-registration")

ErrCollectionPathConflict is returned by CreateCollection when <projectPath>/.ingitdb/root-collections.yaml already contains an entry for the collection name with a non-default path value. Callers either remove the existing entry or pick a different collection name.

Functions

func AccessValue

func AccessValue(row recordset.Row, rs recordset.Recordset, collectionID, recordKey, colName string, colDef *ingitdb.ColumnDef) (any, error)

AccessValue reads colName from a recordset row — the single coerce-on-access path every read consumer (select/delete/update/TUI) uses.

For a computed (formula) column the raw evaluator result is coerced to the column's declared ColumnType via coerceFormulaResult, so typed results stay identical to the eager ApplyFormulasToRead pipeline. A stored column's value (colDef nil or no Formula) is returned unchanged — the eager pipeline never coerced stored values, so neither do we.

Because computation is lazy, evaluation happens here, on access. Any evaluator or coercion error is wrapped with the collection, record key, and column, so the failure is fail-loud and names its source (matching the eager pipeline's error format).

func AllColumnNames

func AllColumnNames(rs recordset.Recordset) []string

AllColumnNames returns every column name of rs except the reserved IDColumn.

func ApplyFormulasToRead

func ApplyFormulasToRead(data map[string]any, cols map[string]*ingitdb.ColumnDef, collectionID, recordKey string) (map[string]any, error)

ApplyFormulasToRead computes the value of every computed column (one with a non-empty Formula) and adds it to the returned map, coerced to the column's declared type. The stored fields in data are used as the formula's variable bindings.

When the collection has no computed columns the input map is returned unchanged (not cloned); otherwise a clone is returned with the computed values added, so the input is never mutated. Computed columns are evaluated in deterministic (sorted) order so that, when more than one formula errors, the surfaced error is reproducible.

A runtime evaluation error, or a result that cannot be coerced to the declared type, aborts with an error naming the collection, record key, and column. No partial result is returned in that case.

func BuildRecordset

func BuildRecordset(colDef *ingitdb.CollectionDef, records []KeyedStored) recordset.Recordset

BuildRecordset assembles a recordset.Recordset for a collection: a reserved "$id" column carrying each record key, one ordinary column per stored (non-formula) column carrying the per-record stored value, and one recordset.NewComputedColumn per formula column bound to a Starlark-backed evaluator. Computed values are never evaluated here — they resolve lazily, at most once per row, when a consumer reads them.

func CollectionForKey

func CollectionForKey(def *ingitdb.Definition, id string) (*ingitdb.CollectionDef, string, error)

CollectionForKey finds the collection and record key for a given ID string.

The id format is "{collectionID}/{recordKey}" where collection IDs use "." for namespaces. "/" is reserved for separating collection ID from record key path segments. The longest matching collection prefix wins.

func NewDatabase

func NewDatabase(projectPath string, reader ingitdb.CollectionsReader) (dal.DB, error)

NewDatabase constructs a Database rooted at projectPath. The reader is used to load the project Definition at the start of each transaction and inside DB-level record-access methods. Returns an error if projectPath is empty or does not exist; the constructor does NOT load any collection definitions.

func NewRecordsetReader

func NewRecordsetReader(rs recordset.Recordset) dal.RecordsetReader

NewRecordsetReader returns a dal.RecordsetReader that walks the rows of rs.

func RowData

func RowData(row recordset.Row, rs recordset.Recordset, collectionID, recordKey string, colDef *ingitdb.CollectionDef, names []string) (map[string]any, error)

RowData reads the named columns of a recordset row through AccessValue and returns them as a map, omitting nil values so the result matches the ragged record map the eager pipeline produced (absent fields were never keys). Only the requested columns are read, so an unreferenced computed column is never evaluated; a referenced computed column that errors surfaces fail-loud.

func RowKey

func RowKey(row recordset.Row, rs recordset.Recordset) string

RowKey returns the record key carried by a row's reserved IDColumn. The IDColumn is present in every recordset BuildRecordset produces, so the lookup cannot fail; a missing or non-string value yields the empty string.

func StoredColumnNames

func StoredColumnNames(rs recordset.Recordset) []string

StoredColumnNames returns the stored (non-computed) column names of rs except the reserved IDColumn. Used by write consumers that must persist stored fields only, never computed values.

func ValidateDelete

func ValidateDelete(def *ingitdb.Definition, parentCollection, parentKey string) error

ValidateDelete enforces parent-side referential integrity before a record is removed (or its key renamed, which manifests as removal of the old key): no stored or computed foreign key in any collection may still reference it.

This is the shared entry point for any DALgo driver's delete path, so enforcement is identical across drivers.

func ValidateWrite

func ValidateWrite(def *ingitdb.Definition, operation, collectionID string, colDef *ingitdb.CollectionDef, recordKey string, data map[string]any) error

ValidateWrite enforces every write-time rule before a record is inserted or set, against the on-disk state described by def:

  • a computed column's value must not be supplied (it is derived, not stored),
  • every stored foreign key must resolve to an existing parent record, and
  • every computed foreign key, evaluated from the record's stored fields, must resolve to an existing parent record.

operation labels the caller ("Insert" or "Set") for error messages. collectionID and colDef identify the collection being written; recordKey and data are the record's key and field values.

This is the shared entry point for any DALgo driver's read-write transaction, so enforcement is identical whether a record is written through the concurrent-safe driver or the filesystem driver.

Types

type CSVParseOptions

type CSVParseOptions struct {
	// KeyColumn, if non-empty, names the column to use as the record
	// key (overrides $id/id auto-resolution).
	KeyColumn string
	// Fields, if non-empty, replaces the header row: the first stdin
	// line is treated as data and these names are used for column
	// mapping.
	Fields []string
}

CSVParseOptions controls CSV-specific behavior.

type Database

type Database struct {
	// dal.NoConcurrency makes SupportsConcurrentConnections() report false.
	//
	// An inGitDB database is a git working tree. We do take gofrs/flock
	// advisory locks per file (shared for reads, exclusive for writes) as
	// defence-in-depth, but that is NOT a basis to advertise safe concurrent
	// connections, because:
	//   - flock is ADVISORY on Unix: it only binds processes that also call
	//     flock. A plain `git`, an editor, or `rm` ignores it entirely — and
	//     on Unix can even unlink a file out from under a held lock. It is
	//     mandatory only on Windows (LockFileEx), so the protection is not
	//     cross-platform.
	//   - locks are PER FILE, so a change spanning multiple files (e.g. a
	//     collection's definition.yaml plus root-collections.yaml, or a
	//     subsequent git commit) is not atomic as a unit.
	// The honest cross-platform contract is therefore single-writer: callers
	// MUST NOT open concurrent writing connections against the same tree.
	dal.NoConcurrency
	// contains filtered or unexported fields
}

Database is the dal.DB implementation for inGitDB projects on the local filesystem. It implements the schema-management capability interfaces (dbschema.SchemaReader, ddl.SchemaModifier, ddl.TransactionalDDL), the dal.DB record-access methods, and reports dal.NoConcurrency — concurrent connections are NOT advertised as safe (see the field comment for why).

Record access loads the project Definition once per transaction via the injected CollectionsReader; individual file operations take a shared (read) or exclusive (write) advisory lock on the affected file. ExecuteQueryToRecordsetReader is not yet implemented and returns dal.ErrNotSupported.

func (*Database) Adapter

func (db *Database) Adapter() dal.Adapter

Adapter returns the dalgo adapter descriptor.

func (*Database) AlterCollection

func (db *Database) AlterCollection(ctx context.Context, name string, ops ...ddl.AlterOp) error

AlterCollection applies AlterOp values in order. Operations mutate an in-memory ingitdb.CollectionDef; after each op the updated definition is flushed to disk. Failure mid-sequence returns *ddl.PartialSuccessError with applied / failed / not-attempted lists.

func (*Database) CreateCollection

func (db *Database) CreateCollection(_ context.Context, c dbschema.CollectionDef, opts ...ddl.Option) error

CreateCollection writes <projectPath>/<c.Name>/.collection/definition.yaml from the dbschema.CollectionDef. Validates name and field types before any filesystem write. With ddl.IfNotExists, an existing collection is a no-op; without it, an existing collection is an error.

After the definition.yaml write succeeds, the collection is registered in <projectPath>/.ingitdb/root-collections.yaml (REQ:auto-register-in-root-collections) so the validator-backed CollectionsReader picks it up. Registry conflicts (an existing entry mapping the same name to a non-default path) return ErrCollectionPathConflict; the definition.yaml is left in place — see AC:create-collection-rejects-registry-conflict for the recovery story.

func (*Database) DescribeCollection

func (db *Database) DescribeCollection(_ context.Context, ref *dal.CollectionRef) (*dbschema.CollectionDef, error)

DescribeCollection reads and parses the collection's definition.yaml under a shared lock, then maps the ingitdb columns to dbschema fields via type_mapping. PrimaryKey is synthesized as [pkFieldName] because inGitDB uses the record's filesystem key as the de-facto PK.

func (*Database) DropCollection

func (db *Database) DropCollection(_ context.Context, name string, opts ...ddl.Option) error

DropCollection removes <projectPath>/<name>/ from disk. The directory must contain a .collection/definition.yaml as a safety check — this guards against accidentally deleting non-collection directories. With ddl.IfExists, a missing collection is a no-op.

func (*Database) ExecuteQueryToRecordsReader

func (db *Database) ExecuteQueryToRecordsReader(ctx context.Context, query dal.Query) (dal.RecordsReader, error)

ExecuteQueryToRecordsReader runs a structured query against a single collection. See readonlyTx.ExecuteQueryToRecordsReader for supported query features.

func (*Database) ExecuteQueryToRecordsetReader

func (db *Database) ExecuteQueryToRecordsetReader(_ context.Context, _ dal.Query, _ ...recordset.Option) (dal.RecordsetReader, error)

ExecuteQueryToRecordsetReader is not implemented yet; callers should use ExecuteQueryToRecordsReader instead.

func (*Database) Exists

func (db *Database) Exists(ctx context.Context, key *dal.Key) (bool, error)

Exists reports whether the record identified by key exists on disk.

func (*Database) Get

func (db *Database) Get(ctx context.Context, record dal.Record) error

Get loads a single record. See readonlyTx.Get for semantics.

func (*Database) GetMulti

func (db *Database) GetMulti(ctx context.Context, records []dal.Record) error

GetMulti loads multiple records.

func (*Database) ID

func (db *Database) ID() string

ID returns the driver identifier.

func (*Database) ListCollections

func (db *Database) ListCollections(_ context.Context, _ *dal.Key) ([]dal.CollectionRef, error)

ListCollections walks the project directory looking for directories that contain a .collection/definition.yaml file. The parent argument is ignored (inGitDB has no catalog hierarchy). Results are sorted alphabetically by name; names use "/" as the separator for nested collection paths relative to projectPath.

func (*Database) ListConstraints

func (db *Database) ListConstraints(_ context.Context, _ *dal.CollectionRef) ([]dbschema.ConstraintDef, error)

ListConstraints returns a synthesized single-element slice describing the primary-key constraint. ingitdb does not store other constraint kinds in definition.yaml. The richer PK column information lives on `DescribeCollection.PrimaryKey`; dbschema.ConstraintDef is intentionally minimal (Name + Type only).

func (*Database) ListIndexes

func (db *Database) ListIndexes(_ context.Context, _ *dal.CollectionRef) ([]dbschema.IndexDef, error)

ListIndexes returns a non-nil empty slice and nil error. inGitDB has no per-collection index declarations today.

func (*Database) ListReferrers

func (db *Database) ListReferrers(_ context.Context, _ *dal.CollectionRef) ([]dbschema.Referrer, error)

ListReferrers returns *dbschema.NotSupportedError — inGitDB has no structural foreign-key declarations; ColumnDef.ForeignKey is a free-text hint, not a navigable reference.

func (*Database) RunReadonlyTransaction

func (db *Database) RunReadonlyTransaction(ctx context.Context, f dal.ROTxWorker, options ...dal.TransactionOption) error

RunReadonlyTransaction loads the project Definition and invokes the worker with a readonly transaction. The Definition is captured at the start of the transaction; subsequent on-disk schema changes are not observed within the transaction.

func (*Database) RunReadwriteTransaction

func (db *Database) RunReadwriteTransaction(ctx context.Context, f dal.RWTxWorker, options ...dal.TransactionOption) error

RunReadwriteTransaction loads the project Definition and invokes the worker with a read-write transaction. inGitDB does not guarantee atomicity across multiple file writes within a transaction; each individual file write is locked exclusively, but a worker that fails after writing some files leaves those writes in place.

func (*Database) Schema

func (db *Database) Schema() dal.Schema

Schema returns nil — inGitDB does not yet expose a dal.Schema view of its collection definitions. Callers needing schema introspection should use dbschema.SchemaReader instead.

func (*Database) SupportsTransactionalDDL

func (db *Database) SupportsTransactionalDDL() bool

SupportsTransactionalDDL satisfies ddl.TransactionalDDL by reporting that this driver does NOT guarantee all-or-nothing for multi-op AlterCollection calls. A failure mid-sequence leaves earlier ops applied; the caller receives a *ddl.PartialSuccessError.

type KeyedStored

type KeyedStored struct {
	Key    string
	Stored map[string]any
}

KeyedStored pairs a record key with its locale-normalized stored fields. The stored map holds only stored (non-computed) values; computed columns are resolved lazily through the recordset, never baked in here.

type ParsedRecord

type ParsedRecord struct {
	// Position is 1-based: line number for jsonl/csv, document index
	// for yaml/ingr. For csv with a header row, Position 2 is the
	// first data record.
	Position int
	// Key is the resolved record key (from $id, id, or --key-column).
	Key string
	// Data is the record's structured fields with the key field stripped.
	Data map[string]any
}

ParsedRecord is one record extracted from a batch stream.

func ParseBatchCSV

func ParseBatchCSV(r io.Reader, opts CSVParseOptions) ([]ParsedRecord, error)

ParseBatchCSV reads RFC 4180 CSV from r and returns one ParsedRecord per data row. Key resolution precedence is:

  1. opts.KeyColumn if set (rejected before reading rows if column missing).
  2. column named "$id" if present.
  3. column named "id" if present (auto-mapped).
  4. otherwise error.

When both "$id" and "id" columns exist without opts.KeyColumn, "$id" wins; "id" is kept as a data field. The resolved key column's value is stripped from Data.

If opts.Fields is non-empty, those names override the header row: the first stdin line is treated as data, and Position is 1-based against data rows. Otherwise Position is 1-based against source lines, so the header is line 1 and the first data row is line 2.

func ParseBatchINGR

func ParseBatchINGR(r io.Reader) ([]ParsedRecord, error)

ParseBatchINGR reads an INGR multi-record stream from r and returns one ParsedRecord per record. The key is read from the reserved $ID column (INGR's key field; note the uppercase). $ID is stripped from the returned Data map. Position is the 1-based record index.

func ParseBatchJSONL

func ParseBatchJSONL(r io.Reader) ([]ParsedRecord, error)

ParseBatchJSONL reads NDJSON from r and returns one ParsedRecord per non-blank line. Each record MUST have a top-level $id; the $id is stripped from the returned Data map. Blank lines are skipped but counted for the Position field.

func ParseBatchYAMLStream

func ParseBatchYAMLStream(r io.Reader) ([]ParsedRecord, error)

ParseBatchYAMLStream reads a YAML multi-document stream from r and returns one ParsedRecord per non-nil document. Each record MUST have a top-level $id; $id is stripped from the returned Data map. Position is the 1-based document index.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL