Documentation
¶
Overview ¶
Package duckdb is lake-orm's embedded DuckDB driver. It takes a *sql.DB constructed against go-duckdb and implements the drivers.Driver interface by translating between lake-tagged Go structs and DuckDB SQL rows.
This is the "no network, no JVM, no container" path — useful for:
- Unit + integration tests that want a real engine without Docker or k8s.
- Local development iteration faster than docker-compose up.
- Single-process analytics workloads that fit on one box.
What this driver is NOT:
- A distributed engine. DuckDB is embedded single-process.
- Feature-matched with Spark. Features like Iceberg merge-on- read, Delta deletion vectors, Hive-style writes depend on DuckDB's first-party extensions; this driver only relies on DuckDB's core SQL + parquet support by default.
CGO requirement. go-duckdb links against DuckDB's C++ library via CGO. Builds with `CGO_ENABLED=0` will fail. Pre-compiled DuckDB binaries cover linux-amd64 / linux-arm64 / darwin-amd64 / darwin-arm64 / windows-amd64 out of the box; exotic targets (distroless without CGO, some musl variants) may need custom builds.
Driver family (four now):
- drivers/spark — generic Spark Connect
- drivers/databricksconnect — Databricks Connect (OAuth M2M)
- drivers/databricks — Databricks native (BYO *sql.DB)
- drivers/duckdb — embedded DuckDB (this package)
All four implement drivers.Driver + drivers.Convertible. ORM code (Insert, Query[T], Migrate) stays portable across them.
Index ¶
- type Driver
- func (d *Driver) Close() error
- func (d *Driver) Collect(ctx context.Context, source drivers.Source, out any) error
- func (d *Driver) DB() *sql.DB
- func (d *Driver) Exec(ctx context.Context, sqlStr string, args ...any) (drivers.ExecResult, error)
- func (d *Driver) Execute(ctx context.Context, plan drivers.ExecutionPlan) (drivers.Result, drivers.Finalizer, error)
- func (d *Driver) First(ctx context.Context, source drivers.Source, out any) error
- func (d *Driver) FromRow(sqlStr string, args ...any) drivers.Source
- func (d *Driver) FromRows(rows *sql.Rows) drivers.Source
- func (d *Driver) FromSQL(sqlStr string, args ...any) drivers.Source
- func (d *Driver) FromTable(name string) drivers.Source
- func (d *Driver) Name() string
- func (d *Driver) Stream(ctx context.Context, source drivers.Source, sample any) iter.Seq2[any, error]
- type Option
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Driver ¶
type Driver struct {
// contains filtered or unexported fields
}
Driver is lake-orm's DuckDB driver. Exported so callers can reach the per-driver conversion helpers (FromSQL, FromRows, FromTable, FromRow) and the raw *sql.DB (DB) via a Client.Driver() type-assertion.
func New ¶
New returns a *Driver backed by the supplied *sql.DB. The *sql.DB is owned by the caller: Driver.Close does NOT close it. This matches the database/sql lifecycle most consumers already implement.
Typical construction (embedded in-memory DuckDB):
import _ "github.com/marcboeker/go-duckdb/v2"
sqlDB, _ := sql.Open("duckdb", "")
drv := duckdb.New(sqlDB)
client, _ := lakeorm.Open(drv, dialect, store)
For a persistent file-backed database pass a path:
sql.Open("duckdb", "analytics.ddb")
func (*Driver) Close ¶
Close implements drivers.Driver. The *sql.DB is owned by the caller and is NOT closed here — they constructed it, they close it on their own shutdown path.
func (*Driver) Collect ¶
Collect runs source and appends every decoded row into out (must be a *[]T).
func (*Driver) DB ¶
DB returns the underlying *sql.DB. Escape hatch for callers who need to drop to raw database/sql operations the drivers.Driver interface doesn't expose (PrepareContext, transactions, `INSTALL iceberg;`-style extension setup, etc.).
func (*Driver) Execute ¶
func (d *Driver) Execute(ctx context.Context, plan drivers.ExecutionPlan) (drivers.Result, drivers.Finalizer, error)
Execute implements drivers.Driver.
KindDDL / KindSQL route through (*sql.DB).ExecContext. KindDirectIngest walks plan.Rows via reflection and fires one prepared INSERT per row — fine for an embedded engine where there's no network round-trip per statement. KindParquetIngest is not reachable from the duckdb Dialect (which never routes to it); if another dialect emits one at this driver it errors.
func (*Driver) FromRow ¶
FromRow is the single-row variant of FromSQL. Structurally identical — the "single row" semantic is enforced by the Convertible.First call at the other end, which stops iterating after the first row and returns errors.ErrNoRows when the source yielded zero rows.
func (*Driver) FromRows ¶
FromRows builds a Source that hands back an already-opened *sql.Rows. Use when the caller has run QueryContext themselves and wants the typed decode path over the result. Cleanup is the caller's responsibility — the Source returns nil cleanup so the Convertible impl doesn't close rows the caller still owns.
func (*Driver) FromSQL ¶
FromSQL builds a Source that runs sql with args via (*sql.DB).QueryContext and hands back the resulting *sql.Rows. The cleanup hook calls rows.Close.
Source Files