fdb.dev

fdb-go - FoundationDB Ecosystem for Go

A native FoundationDB client, a testcontainer, the Record Layer and an embedded SQL engine for Go, without cgo. Requires a FoundationDB 7.3 cluster.

go get fdb.dev@master

Or run the examples from a clone, against a disposable cluster in Docker. Build the fdb CLI from the checkout:

go build -o .tmpbuild/fdb ./cmd/fdb &&
export PATH="$PWD/.tmpbuild:$PATH" &&
FDB_CLUSTER_FILE=$(fdb dev up) &&
export FDB_CLUSTER_FILE &&
go run ./example/sql &&
go run ./example

fdb dev down removes the cluster. Tagged installation with go install fdb.dev/cmd/fdb@<tag> requires a release containing cmd/fdb; the installer also requires matching fdb_ release archives. Use the checkout build until that release is available. See the CLI guide for configuration and commands.

Pre-1.0; expect API changes. Status and known gaps. Independent project, not affiliated with Apple or the FoundationDB project.

Native Go Client

fdb.dev/pkg/fdbgo/fdb implements FoundationDB's 7.3 client protocol without cgo or libfdb_c, with an API modeled on Apple's Go binding. Differential tests compare results, stored bytes and error codes against libfdb_c 7.3.77 on every pull request.

import "fdb.dev/pkg/fdbgo/fdb"

fdb.MustAPIVersion(730)
db, err := fdb.OpenDatabase(clusterFile)
if err != nil {
	return err
}
defer db.Close()

v, err := db.Transact(func(tx fdb.WritableTransaction) (any, error) {
	tx.Set(fdb.Key("hello"), []byte("world"))
	return tx.Get(fdb.Key("hello")).Get()
})

Transact retries retryable errors; keep external side effects outside its callback. Supported options.

One transaction per operation, Go client vs. Apple's binding over libfdb_c, same process and cluster:

Median of 5, master 9a9a448, Ryzen 9 3900X, single-node FDB 7.3.77 (ssd engine) in local Docker. Allocation counts cover the Go heap only, not libfdb_c's. Raw results and commands. An earlier, larger comparison measured an older client revision.

Testcontainer

fdb.dev/pkg/testcontainers/foundationdb starts FoundationDB in Docker and returns a configured, ready cluster. Single node or several (RunCluster); storage engine, redundancy and knobs are options; Pause simulates an unreachable cluster.

import foundationdbtc "fdb.dev/pkg/testcontainers/foundationdb"

func TestThing(t *testing.T) {
	ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
	defer cancel()

	c, err := foundationdbtc.Run(ctx, "") // foundationdb/foundationdb:7.3.77
	if err != nil {
		t.Fatal(err)
	}
	t.Cleanup(func() { c.Terminate(context.Background()) })

	clusterFile, err := c.ClusterFilePath(ctx)
	if err != nil {
		t.Fatal(err)
	}
	fdb.MustAPIVersion(730)
	db, err := fdb.OpenDatabase(clusterFile)
	// ...
}

Run returns a usable database in 4.6 s (median of 5, image already pulled, same machine). Raw results.

Record Layer

A Go port of FoundationDB's Record Layer: protobuf records, secondary indexes, schema evolution and resumable cursors. Go and Java Record Layer 4.14.2.0 can share record stores in the supported formats; see compatibility and the Java conformance tests.

Go

fdb.dev/pkg/recordlayer. Records are your protobuf messages; as in Java, a union message lists the record types:

message Order {
  optional int64 order_id = 1;
  optional int32 price = 2;
}

message RecordTypeUnion {
  option (com.apple.foundationdb.record.record).usage = UNION;
  optional Order _Order = 1;
}

builder := recordlayer.NewRecordMetaDataBuilder().SetRecords(shoppb.File_shop_proto)
builder.GetRecordType("Order").SetPrimaryKey(recordlayer.Field("order_id"))
builder.AddIndex("Order", recordlayer.NewIndex("price_idx", recordlayer.Field("price")))
metadata, err := builder.Build()
if err != nil {
	return err
}

rdb := recordlayer.NewFDBDatabase(db)
_, err = rdb.Run(ctx, func(rtx *recordlayer.FDBRecordContext) (any, error) {
	store, err := recordlayer.NewStoreBuilder().
		SetContext(rtx).
		SetMetaDataProvider(metadata).
		SetSubspace(subspace.FromBytes([]byte("app"))).
		CreateOrOpen()
	if err != nil {
		return nil, err
	}
	if _, err := store.SaveRecord(&shoppb.Order{OrderId: proto.Int64(1), Price: proto.Int32(25)}); err != nil {
		return nil, err
	}
	orders, err := recordlayer.GetTypedRecordStore[*shoppb.Order](store, "Order")
	if err != nil {
		return nil, err
	}
	return orders.LoadRecord(tuple.Tuple{int64(1)})
})

Runnable example · operations guide (online index builds, schema evolution).

Median of 5, same machine, FDB memory engine. Raw results.

SQL

fdb.dev/pkg/relational/sqldriver is an embedded database/sql driver over Record Layer stores. It uses Java Relational's SQL dialect and a port of its Cascades planner. Not PostgreSQL-compatible; differences from Java.

import (
	"database/sql"
	"net/url"

	"fdb.dev/pkg/relational/sqldriver"
)

sqldriver.RegisterDomainIfNotExists("FRL")
db, err := sql.Open("fdbsql", "fdbsql:///FRL/SHOP?schema=APP&cluster_file="+url.QueryEscape(clusterFile))
if err != nil {
	return err
}
defer db.Close()

rows, err := db.QueryContext(ctx,
	"SELECT customer, SUM(amount) FROM orders GROUP BY customer ORDER BY customer")
if err != nil {
	return err
}
defer rows.Close()

Complete example, including creating the database, schema and table.

Median of 5, same machine, FDB memory engine. The selects reuse a cached plan; the insert is planned on every execution. Raw results.

AI Disclaimer

LLMs were used heavily in the development and maintenance of this project. Credit goes to the original authors of FoundationDB and the FoundationDB Record Layer.

Contributing · Test report · Security · Support · Apache-2.0 (NOTICE)