dalgo2ingitdb

package
v1.24.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 3, 2026 License: MIT Imports: 32 Imported by: 0

Documentation

Index

Constants

View Source 
const DatabaseID = "dalgo2ingitdb"

DatabaseID is the name reported by Database.ID() and used as the Adapter name.

View Source 
const IDColumn = "$id"

IDColumn is the reserved recordset column that carries each record's key. It is not a declared schema column; the Starlark evaluator strips it from the stored field map so formula inputs match the eager ApplyFormulasToRead pipeline exactly. The "$id" spelling mirrors the dal pseudo-field convention and cannot collide with a real column (Starlark identifiers cannot contain "$", so no formula can reference it).

Variables

View Source 
var ErrCollectionPathConflict = errors.New("dalgo2ingitdb: root-collections.yaml entry conflicts with auto-registration")

ErrCollectionPathConflict is returned by CreateCollection when <projectPath>/.ingitdb/root-collections.yaml already contains an entry for the collection name with a non-default path value. Callers either remove the existing entry or pick a different collection name.

Functions

func AccessValue added in v1.22.0 

func AccessValue(row recordset.Row, rs recordset.Recordset, collectionID, recordKey, colName string, colDef *ingitdb.ColumnDef) (any, error)

AccessValue reads colName from a recordset row — the single coerce-on-access path every read consumer (select/delete/update/TUI) uses.

For a computed (formula) column the raw evaluator result is coerced to the column's declared ColumnType via coerceFormulaResult, so typed results stay identical to the eager ApplyFormulasToRead pipeline. A stored column's value (colDef nil or no Formula) is returned unchanged — the eager pipeline never coerced stored values, so neither do we.

Because computation is lazy, evaluation happens here, on access. Any evaluator or coercion error is wrapped with the collection, record key, and column, so the failure is fail-loud and names its source (matching the eager pipeline's error format).

func AllColumnNames added in v1.22.0 

func AllColumnNames(rs recordset.Recordset) []string

AllColumnNames returns every column name of rs except the reserved IDColumn.

func ApplyFormulasToRead added in v1.20.0 

func ApplyFormulasToRead(data map[string]any, cols map[string]*ingitdb.ColumnDef, collectionID, recordKey string) (map[string]any, error)

ApplyFormulasToRead computes the value of every computed column (one with a non-empty Formula) and adds it to the returned map, coerced to the column's declared type. The stored fields in data are used as the formula's variable bindings.

When the collection has no computed columns the input map is returned unchanged (not cloned); otherwise a clone is returned with the computed values added, so the input is never mutated. Computed columns are evaluated in deterministic (sorted) order so that, when more than one formula errors, the surfaced error is reproducible.

A runtime evaluation error, or a result that cannot be coerced to the declared type, aborts with an error naming the collection, record key, and column. No partial result is returned in that case.

func ApplyLocaleToRead added in v0.12.0 

func ApplyLocaleToRead(data map[string]any, cols map[string]*ingitdb.ColumnDef) map[string]any

ApplyLocaleToRead transforms record data from file representation to application representation. For each column that has a Locale value set (e.g. column "title" with locale "en"), the paired map column (e.g. "titles") is inspected: the locale entry is extracted and exposed as the shortcut column ("title"), and that locale key is removed from the pair map to avoid duplication. The caller receives e.g. {"title": "Work", "titles": {"ru": "Работа"}}.

func ApplyLocaleToWrite added in v0.12.0 

func ApplyLocaleToWrite(data map[string]any, cols map[string]*ingitdb.ColumnDef) map[string]any

ApplyLocaleToWrite normalises record data before writing to file. For each column that has a Locale value set (e.g. column "title" with locale "en"):

  • The shortcut column ("title") is stored as-is in the file.
  • If the paired map column ("titles") contains an entry for the primary locale key ("en"), that entry is promoted to the shortcut column and removed from the map, so the value is never duplicated across both fields.
  • If the paired map becomes empty after removing the primary locale entry, it is dropped from the result to avoid writing a redundant empty map.

func BuildRecordset added in v1.22.0 

func BuildRecordset(colDef *ingitdb.CollectionDef, records []KeyedStored) recordset.Recordset

BuildRecordset assembles a recordset.Recordset for a collection: a reserved "$id" column carrying each record key, one ordinary column per stored (non-formula) column carrying the per-record stored value, and one recordset.NewComputedColumn per formula column bound to a Starlark-backed evaluator. Computed values are never evaluated here — they resolve lazily, at most once per row, when a consumer reads them.

func CollectionForKey added in v0.9.0 

func CollectionForKey(def *ingitdb.Definition, id string) (*ingitdb.CollectionDef, string, error)

CollectionForKey finds the collection and record key for a given ID string.

The id format is "{collectionID}/{recordKey}" where collection IDs use "." for namespaces. "/" is reserved for separating collection ID from record key path segments. The longest matching collection prefix wins.

func EncodeListOfRecordsContent added in v1.17.0 

func EncodeListOfRecordsContent(rows []map[string]any, format ingitdb.RecordFormat, columnsOrder []string) ([]byte, error)

EncodeListOfRecordsContent serializes list rows back to the declared format, preserving record (insertion) order. Within each record, keys are emitted in columnsOrder first, then remaining keys alphabetically. JSONL writes one compact JSON object per line; JSON writes a pretty array; YAML writes a top-level sequence. csv and ingr keep their dedicated encoders.

func EncodeMapOfRecordsContent added in v0.57.0 

func EncodeMapOfRecordsContent(data map[string]map[string]any, format ingitdb.RecordFormat, recordsetName string, columnsOrder []string) ([]byte, error)

EncodeMapOfRecordsContent serializes a map of ID-keyed records into the declared format. For yaml/json/toml, the data is written as a top-level mapping (record ID -> field map). For INGR, the data is flattened into a list of records (each gets `$ID` injected as the first column) and written via the ingr-io writer.

recordsetName is used as the INGR header title (typically the collection ID). It is ignored for non-INGR formats. columnsOrder controls the column order in the INGR header; when empty, columns are emitted in alphabetical order with `$ID` first.

func EncodeRecordContentForCollection added in v1.2.0 

func EncodeRecordContentForCollection(value any, colDef *ingitdb.CollectionDef) ([]byte, error)

EncodeRecordContentForCollection serializes record content using the collection's declared format. It is the write-side counterpart of ParseRecordContentForCollection and the only path callers should use when writing records that may be in a format that requires schema access (csv today; possibly others later).

For csv, value MUST be []map[string]any — see encodeCSVForCollection. For the other six formats, value is passed through to marshalForFormat unchanged; callers that want a single-record map[string]any can keep using the schema-agnostic marshalForFormat directly.

func NewDatabase added in v1.8.0 

func NewDatabase(projectPath string, reader ingitdb.CollectionsReader) (dal.DB, error)

NewDatabase constructs a Database rooted at projectPath. The reader is used to load the project Definition at the start of each transaction and inside DB-level record-access methods. Returns an error if projectPath is empty or does not exist; the constructor does NOT load any collection definitions.

func NewRecordsetReader added in v1.22.0 

func NewRecordsetReader(rs recordset.Recordset) dal.RecordsetReader

NewRecordsetReader returns a dal.RecordsetReader that walks the rows of rs.

func ParseListOfRecordsContent added in v1.17.0 

func ParseListOfRecordsContent(content []byte, format ingitdb.RecordFormat) ([]map[string]any, error)

ParseListOfRecordsContent parses a list-of-records file into ordered row maps. It handles a top-level YAML sequence, a top-level JSON array, and a JSONL stream (one JSON object per non-empty line). Empty content yields no rows. csv and ingr keep their dedicated parsers and are not handled here.

func ParseMapOfRecordsContent added in v0.40.0 

func ParseMapOfRecordsContent(content []byte, format ingitdb.RecordFormat) (map[string]map[string]any, error)

ParseMapOfRecordsContent parses content containing a map of ID-keyed records.

For yaml/json/toml, the file's top-level structure is a mapping from record ID to a per-record field map; we re-shape it into map[string]map[string]any.

For INGR, the file is a list of records where the reserved `$ID` column holds each record's key; we read the list and re-index by `$ID`. Records missing `$ID`, or with duplicate `$ID` values, are rejected as malformed.

func ParseRecordContent added in v0.12.0 

func ParseRecordContent(content []byte, format ingitdb.RecordFormat) (map[string]any, error)

ParseRecordContent parses record content in YAML or JSON format. For markdown-format collections use ParseRecordContentForCollection, which also has access to the column schema and content-field name.

func ParseRecordContentForCollection added in v0.55.0 

func ParseRecordContentForCollection(content []byte, colDef *ingitdb.CollectionDef) (map[string]any, error)

ParseRecordContentForCollection parses record content using the collection's declared format. For YAML and JSON this is equivalent to ParseRecordContent. For markdown records it parses YAML frontmatter, filters to columns declared in the schema, and exposes the body under the configured content_field column name.

func ResolveListRecordKey added in v1.17.0 

func ResolveListRecordKey(row map[string]any, colDef *ingitdb.CollectionDef) (string, bool)

ResolveListRecordKey resolves a list row's record key, in order: the collection's declared primary_key (composite values joined), else a "$id" field, else an "id" field. ok is false when none is available, meaning the list has no usable record key.

func RowData added in v1.22.0 

func RowData(row recordset.Row, rs recordset.Recordset, collectionID, recordKey string, colDef *ingitdb.CollectionDef, names []string) (map[string]any, error)

RowData reads the named columns of a recordset row through AccessValue and returns them as a map, omitting nil values so the result matches the ragged record map the eager pipeline produced (absent fields were never keys). Only the requested columns are read, so an unreferenced computed column is never evaluated; a referenced computed column that errors surfaces fail-loud.

func RowKey added in v1.22.0 

func RowKey(row recordset.Row, rs recordset.Recordset) string

RowKey returns the record key carried by a row's reserved IDColumn. The IDColumn is present in every recordset BuildRecordset produces, so the lookup cannot fail; a missing or non-string value yields the empty string.

func StoredColumnNames added in v1.22.0 

func StoredColumnNames(rs recordset.Recordset) []string

StoredColumnNames returns the stored (non-computed) column names of rs except the reserved IDColumn. Used by write consumers that must persist stored fields only, never computed values.

func ValidateDelete added in v1.21.1 

func ValidateDelete(def *ingitdb.Definition, parentCollection, parentKey string) error

ValidateDelete enforces parent-side referential integrity before a record is removed (or its key renamed, which manifests as removal of the old key): no stored or computed foreign key in any collection may still reference it.

This is the shared entry point for any DALgo driver's delete path, so enforcement is identical across drivers.

func ValidateWrite added in v1.21.1 

func ValidateWrite(def *ingitdb.Definition, operation, collectionID string, colDef *ingitdb.CollectionDef, recordKey string, data map[string]any) error

ValidateWrite enforces every write-time rule before a record is inserted or set, against the on-disk state described by def:

  • a computed column's value must not be supplied (it is derived, not stored),
  • every stored foreign key must resolve to an existing parent record, and
  • every computed foreign key, evaluated from the record's stored fields, must resolve to an existing parent record.

operation labels the caller ("Insert" or "Set") for error messages. collectionID and colDef identify the collection being written; recordKey and data are the record's key and field values.

This is the shared entry point for any DALgo driver's read-write transaction, so enforcement is identical whether a record is written through the concurrent-safe driver or the filesystem driver.

Types

type CSVParseOptions added in v1.7.0 

type CSVParseOptions struct {
	// KeyColumn, if non-empty, names the column to use as the record
	// key (overrides $id/id auto-resolution).
	KeyColumn string
	// Fields, if non-empty, replaces the header row: the first stdin
	// line is treated as data and these names are used for column
	// mapping.
	Fields []string
}

CSVParseOptions controls CSV-specific behavior.

type Database added in v1.8.0 

type Database struct {
	// dal.ConcurrencyAvailable: gofrs/flock provides cross-platform file
	// locking (syscall.Flock on Unix, LockFileEx on Windows), so two DB
	// handles against the same project directory can operate concurrently
	// without data races.
	dal.ConcurrencyAvailable
	// contains filtered or unexported fields
}

Database is the dal.DB implementation for inGitDB projects on the local filesystem. It implements the schema-management capability interfaces (dbschema.SchemaReader, ddl.SchemaModifier, ddl.TransactionalDDL), the dal.DB record-access methods, and reports dal.ConcurrencyAvailable via the embedded helper struct — concurrent connections are safe because every read and write path goes through gofrs/flock file locking.

Record access loads the project Definition once per transaction via the injected CollectionsReader; individual file operations take a shared (read) or exclusive (write) advisory lock on the affected file. ExecuteQueryToRecordsetReader is not yet implemented and returns dal.ErrNotSupported.

func (*Database) Adapter added in v1.8.0 

func (db *Database) Adapter() dal.Adapter

Adapter returns the dalgo adapter descriptor.

func (*Database) AlterCollection added in v1.8.0 

func (db *Database) AlterCollection(ctx context.Context, name string, ops ...ddl.AlterOp) error

AlterCollection applies AlterOp values in order. Operations mutate an in-memory ingitdb.CollectionDef; after each op the updated definition is flushed to disk. Failure mid-sequence returns *ddl.PartialSuccessError with applied / failed / not-attempted lists.

func (*Database) CreateCollection added in v1.8.0 

func (db *Database) CreateCollection(_ context.Context, c dbschema.CollectionDef, opts ...ddl.Option) error

CreateCollection writes <projectPath>/<c.Name>/.collection/definition.yaml from the dbschema.CollectionDef. Validates name and field types before any filesystem write. With ddl.IfNotExists, an existing collection is a no-op; without it, an existing collection is an error.

After the definition.yaml write succeeds, the collection is registered in <projectPath>/.ingitdb/root-collections.yaml (REQ:auto-register-in-root-collections) so the validator-backed CollectionsReader picks it up. Registry conflicts (an existing entry mapping the same name to a non-default path) return ErrCollectionPathConflict; the definition.yaml is left in place — see AC:create-collection-rejects-registry-conflict for the recovery story.

func (*Database) DescribeCollection added in v1.8.0 

func (db *Database) DescribeCollection(_ context.Context, ref *dal.CollectionRef) (*dbschema.CollectionDef, error)

DescribeCollection reads and parses the collection's definition.yaml under a shared lock, then maps the ingitdb columns to dbschema fields via type_mapping. PrimaryKey is synthesized as [pkFieldName] because inGitDB uses the record's filesystem key as the de-facto PK.

func (*Database) DropCollection added in v1.8.0 

func (db *Database) DropCollection(_ context.Context, name string, opts ...ddl.Option) error

DropCollection removes <projectPath>/<name>/ from disk. The directory must contain a .collection/definition.yaml as a safety check — this guards against accidentally deleting non-collection directories. With ddl.IfExists, a missing collection is a no-op.

func (*Database) ExecuteQueryToRecordsReader added in v1.8.0 

func (db *Database) ExecuteQueryToRecordsReader(ctx context.Context, query dal.Query) (dal.RecordsReader, error)

ExecuteQueryToRecordsReader runs a structured query against a single collection. See readonlyTx.ExecuteQueryToRecordsReader for supported query features.

func (*Database) ExecuteQueryToRecordsetReader added in v1.8.0 

func (db *Database) ExecuteQueryToRecordsetReader(_ context.Context, _ dal.Query, _ ...recordset.Option) (dal.RecordsetReader, error)

ExecuteQueryToRecordsetReader is not implemented yet; callers should use ExecuteQueryToRecordsReader instead.

func (*Database) Exists added in v1.8.0 

func (db *Database) Exists(ctx context.Context, key *dal.Key) (bool, error)

Exists reports whether the record identified by key exists on disk.

func (*Database) Get added in v1.8.0 

func (db *Database) Get(ctx context.Context, record dal.Record) error

Get loads a single record. See readonlyTx.Get for semantics.

func (*Database) GetMulti added in v1.8.0 

func (db *Database) GetMulti(ctx context.Context, records []dal.Record) error

GetMulti loads multiple records.

func (*Database) ID added in v1.8.0 

func (db *Database) ID() string

ID returns the driver identifier.

func (*Database) ListCollections added in v1.8.0 

func (db *Database) ListCollections(_ context.Context, _ *dal.Key) ([]dal.CollectionRef, error)

ListCollections walks the project directory looking for directories that contain a .collection/definition.yaml file. The parent argument is ignored (inGitDB has no catalog hierarchy). Results are sorted alphabetically by name; names use "/" as the separator for nested collection paths relative to projectPath.

func (*Database) ListConstraints added in v1.8.0 

func (db *Database) ListConstraints(_ context.Context, _ *dal.CollectionRef) ([]dbschema.ConstraintDef, error)

ListConstraints returns a synthesized single-element slice describing the primary-key constraint. ingitdb does not store other constraint kinds in definition.yaml. The richer PK column information lives on `DescribeCollection.PrimaryKey`; dbschema.ConstraintDef is intentionally minimal (Name + Type only).

func (*Database) ListIndexes added in v1.8.0 

func (db *Database) ListIndexes(_ context.Context, _ *dal.CollectionRef) ([]dbschema.IndexDef, error)

ListIndexes returns a non-nil empty slice and nil error. inGitDB has no per-collection index declarations today.

func (*Database) ListReferrers added in v1.8.0 

func (db *Database) ListReferrers(_ context.Context, _ *dal.CollectionRef) ([]dbschema.Referrer, error)

ListReferrers returns *dbschema.NotSupportedError — inGitDB has no structural foreign-key declarations; ColumnDef.ForeignKey is a free-text hint, not a navigable reference.

func (*Database) RunReadonlyTransaction added in v1.8.0 

func (db *Database) RunReadonlyTransaction(ctx context.Context, f dal.ROTxWorker, options ...dal.TransactionOption) error

RunReadonlyTransaction loads the project Definition and invokes the worker with a readonly transaction. The Definition is captured at the start of the transaction; subsequent on-disk schema changes are not observed within the transaction.

func (*Database) RunReadwriteTransaction added in v1.8.0 

func (db *Database) RunReadwriteTransaction(ctx context.Context, f dal.RWTxWorker, options ...dal.TransactionOption) error

RunReadwriteTransaction loads the project Definition and invokes the worker with a read-write transaction. inGitDB does not guarantee atomicity across multiple file writes within a transaction; each individual file write is locked exclusively, but a worker that fails after writing some files leaves those writes in place.

func (*Database) Schema added in v1.8.0 

func (db *Database) Schema() dal.Schema

Schema returns nil — inGitDB does not yet expose a dal.Schema view of its collection definitions. Callers needing schema introspection should use dbschema.SchemaReader instead.

func (*Database) SupportsTransactionalDDL added in v1.8.0 

func (db *Database) SupportsTransactionalDDL() bool

SupportsTransactionalDDL satisfies ddl.TransactionalDDL by reporting that this driver does NOT guarantee all-or-nothing for multi-op AlterCollection calls. A failure mid-sequence leaves earlier ops applied; the caller receives a *ddl.PartialSuccessError.

type KeyedStored added in v1.22.0 

type KeyedStored struct {
	Key    string
	Stored map[string]any
}

KeyedStored pairs a record key with its locale-normalized stored fields. The stored map holds only stored (non-computed) values; computed columns are resolved lazily through the recordset, never baked in here.

type ParsedRecord added in v1.5.0 

type ParsedRecord struct {
	// Position is 1-based: line number for jsonl/csv, document index
	// for yaml/ingr. For csv with a header row, Position 2 is the
	// first data record.
	Position int
	// Key is the resolved record key (from $id, id, or --key-column).
	Key string
	// Data is the record's structured fields with the key field stripped.
	Data map[string]any
}

ParsedRecord is one record extracted from a batch stream.

func ParseBatchCSV added in v1.7.0 

func ParseBatchCSV(r io.Reader, opts CSVParseOptions) ([]ParsedRecord, error)

ParseBatchCSV reads RFC 4180 CSV from r and returns one ParsedRecord per data row. Key resolution precedence is:

  1. opts.KeyColumn if set (rejected before reading rows if column missing).
  2. column named "$id" if present.
  3. column named "id" if present (auto-mapped).
  4. otherwise error.

When both "$id" and "id" columns exist without opts.KeyColumn, "$id" wins; "id" is kept as a data field. The resolved key column's value is stripped from Data.

If opts.Fields is non-empty, those names override the header row: the first stdin line is treated as data, and Position is 1-based against data rows. Otherwise Position is 1-based against source lines, so the header is line 1 and the first data row is line 2.

func ParseBatchINGR added in v1.6.0 

func ParseBatchINGR(r io.Reader) ([]ParsedRecord, error)

ParseBatchINGR reads an INGR multi-record stream from r and returns one ParsedRecord per record. The key is read from the reserved $ID column (INGR's key field; note the uppercase). $ID is stripped from the returned Data map. Position is the 1-based record index.

func ParseBatchJSONL added in v1.5.0 

func ParseBatchJSONL(r io.Reader) ([]ParsedRecord, error)

ParseBatchJSONL reads NDJSON from r and returns one ParsedRecord per non-blank line. Each record MUST have a top-level $id; the $id is stripped from the returned Data map. Blank lines are skipped but counted for the Position field.

func ParseBatchYAMLStream added in v1.6.0 

func ParseBatchYAMLStream(r io.Reader) ([]ParsedRecord, error)

ParseBatchYAMLStream reads a YAML multi-document stream from r and returns one ParsedRecord per non-nil document. Each record MUST have a top-level $id; $id is stripped from the returned Data map. Position is the 1-based document index.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.