bench

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 9, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package bench is the end-to-end performance suite: the scenarios behind the budgets of ADR-0012, written against database/sql so that the same scenario runs on every storage engine under consideration.

Two scenarios open Phase 2, as ADR-0017 and ADR-0020 ask:

  • The primary-store benchmark (RunStore) writes commits in the shape of ADR-0014 and ADR-0026, the current row, the history row, the child rows and the event rows in one transaction, and reports latency and throughput, unbatched and batched. Its result decides the transactional store.
  • The point-lookup benchmark (RunLookup) measures one keyed lookup against a reference table on a Go map and through a prepared statement, at several table sizes. Its result fixes the size up to which an auto formula's LOOKUP is served from memory.

The SQLite target lives here; the DuckDB target lives in the analytics module, which is where cgo is allowed. The tests of each module run a small smoke scenario always and the full campaign when FADODB_BENCH is set; see the tests for the variables.

Index

Constants

View Source
const DefaultWALLimit = 64 << 20

DefaultWALLimit is the write-ahead log size at which the background checkpointer truncates: sixteen times SQLite's own threshold of 1,000 pages, so that the pause the truncation costs comes sixteen times less often than the automatic checkpoint's.

View Source
const Header = "| Target | Mode | Work | Throughput | p50 | p90 | p99 | max | File |\n|---|---|---|---|---|---|---|---|---|"

Header is the header of the table String rows belong to.

View Source
const LookupHeader = "| Engine | Rows | Go map | Map heap | Prepared statement |\n|---|---|---|---|---|"

LookupHeader is the header of the table String rows belong to.

Variables

This section is empty.

Functions

func Campaign

func Campaign(ctx context.Context, target Target, dir, which string, w io.Writer) (bool, error)

Campaign runs the store scenario, the lookup scenario or both for a target, as which says ("store", "lookup" or "store,lookup"), writing its databases under dir and the Markdown tables to w. It returns false without running when which is empty, or when FADODB_BENCH_TARGET is set and the target's name does not contain it. The tests pass FADODB_BENCH as which; FADODB_BENCH_COMMITS sizes the store campaign (default 100,000) and FADODB_BENCH_ITERS the lookups per table size (default 50,000).

dir must be on the disk the numbers are meant for. The system's temporary directory is the wrong place: on many Linux machines /tmp is a tmpfs in memory, where an fsync costs nothing and every row of the store table is wrong by an order of magnitude.

func CampaignDir

func CampaignDir() string

CampaignDir is the directory the campaign writes to: FADODB_BENCH_DIR, or the working directory, which under go test is the package's, on the disk the checkout is on. Never the temporary directory; see Campaign.

func FileSizeOf

func FileSizeOf(paths ...string) int64

FileSizeOf sums the sizes of the files at the paths that exist.

func HoldAndProbe

func HoldAndProbe(ctx context.Context, target Target, path string, probeTest string) (string, error)

HoldAndProbe writes a small scenario into path, keeps writing on the single connection while the probe command runs, and returns the probe's combined output and its error. The probe command is the test binary re-run on its probe test with FADODB_BENCH_PROBE naming the file, so that the reader is another process, which is the question of ADR-0017.

func Probe

func Probe(target Target, path string) (int64, error)

Probe opens the database at path read-only, as a second process does, and counts the documents.

func Schema

func Schema(t Target) []string

Schema returns the DDL of the scenario for a target: the current table, the history table, the child table, events and event_docs, with the indexes ADR-0014 and ADR-0026 name.

func Verify

func Verify(ctx context.Context, db *sql.DB, o Options) error

Verify checks the invariants the store will rely on after a run: one current row per document, one history row per commit, Lines child rows per version, one event and one event_docs row per commit, and every current row's version equal to its last history row's.

Types

type Column

type Column struct {
	Name string
	Kind formula.Kind
}

Column is one typed column of the scenario class.

type Execer

type Execer interface {
	ExecContext(ctx context.Context, query string, args ...any) (sql.Result, error)
}

Execer is what a target writes through: a *sql.Conn or a *sql.DB.

type LookupReport

type LookupReport struct {
	Target string
	Rows   int
	// Map is the time of one lookup in a map[string]formula.Value built
	// from the table; MapBytes the heap the map and its keys took, as
	// the runtime measured it.
	Map      time.Duration
	MapBytes int64
	// Stmt is the time of one prepared-statement lookup on the engine.
	Stmt time.Duration
}

LookupReport is the result of the point-lookup benchmark at one table size: the cost of one keyed lookup on a Go map holding the table and through a prepared statement on the engine.

func RunLookup

func RunLookup(ctx context.Context, t Target, db *sql.DB, sizes []int, iters int) ([]LookupReport, error)

RunLookup measures one keyed lookup at each table size: the rates table is filled by the target, loaded into a Go map as the in-memory index of ADR-0016 would hold it, and then looked up iters times by pseudo-random key, once on the map and once through a prepared SELECT on a pinned connection. The keys found must carry the rate the fill wrote, which checks that both paths read the same table.

func (LookupReport) String

func (r LookupReport) String() string

String renders the report as one Markdown table row.

type Options

type Options struct {
	// Commits is the number of commits to write.
	Commits int
	// Documents is the number of documents the commits are spread over.
	// The first Documents commits create, the rest update, so that every
	// document receives Commits/Documents versions.
	Documents int
	// Lines is the number of child rows per version.
	Lines int
	// Batch is the number of commits per transaction; 1 is unbatched, one
	// transaction per commit, which is how a user's edit commits.
	Batch int
	// Seed seeds the values.
	Seed uint64
}

Options size a store run.

type Report

type Report struct {
	Target    string
	Commits   int
	Documents int
	Lines     int
	Batch     int
	// Elapsed is the wall time of all commits; PerSecond the throughput.
	Elapsed   time.Duration
	PerSecond float64
	// The latency percentiles of one commit (Batch 1) or of one batch.
	P50, P90, P99, Max time.Duration
	// FileBytes is the size of the database on disk after the run.
	FileBytes int64
}

Report is the result of one store run.

func RunStore

func RunStore(ctx context.Context, t Target, db *sql.DB, path string, o Options) (Report, error)

RunStore creates the scenario schema in db, which Open gave for path, and writes the commits of o, measuring each transaction. The run pins one connection, as the store's single writer will, and prepares every statement once. The statements are closed before the connection goes back to the pool, where another goroutine of the target may take it.

func (Report) String

func (r Report) String() string

String renders the report as one Markdown table row: target, mode, throughput, p50, p99, max and the file size.

type SQLite

type SQLite struct {
	// Synchronous is the PRAGMA synchronous setting: NORMAL, the
	// architecture page's choice, where a commit is durable against a
	// crash of the process but the last commits may be lost on power
	// failure, or FULL, where every commit is fsynced.
	Synchronous string
	// BackgroundCheckpoints turns SQLite's automatic checkpoint off, where
	// the committing connection copies the log into the database file
	// every 1,000 pages and pays that fsync inside a commit, and runs
	// PRAGMA wal_checkpoint(PASSIVE) on a second connection every 100 ms
	// instead, off the commit path. A passive checkpoint never resets the
	// log while a writer keeps appending, so when the log passes
	// WALLimit bytes the checkpointer runs wal_checkpoint(TRUNCATE),
	// which waits for the writer's transaction, holds the next one back
	// while it copies the rest, and empties the log.
	BackgroundCheckpoints bool
	// WALLimit is the log size that triggers a truncating checkpoint; 0
	// is 64 MB.
	WALLimit int64
}

SQLite is the transactional store candidate of ADR-0017 as the architecture page configures it: WAL mode, synchronous NORMAL, one writer connection.

func (SQLite) ColumnType

func (SQLite) ColumnType(k formula.Kind) string

ColumnType implements Target: SQLite's storage classes, a date as INTEGER days and a datetime as INTEGER seconds (ADR-0023), an exact value as its INTEGER mantissa (ADR-0022).

func (SQLite) FileSize

func (SQLite) FileSize(path string) int64

FileSize implements Target: the database file and its write-ahead log.

func (SQLite) FillRates

func (SQLite) FillRates(ctx context.Context, conn Execer, n int) error

FillRates implements Target with a recursive CTE.

func (SQLite) Name

func (s SQLite) Name() string

Name implements Target.

func (SQLite) Open

func (s SQLite) Open(path string) (*sql.DB, error)

Open implements Target: one connection for the writer, and a second one for the checkpointer when checkpoints run in the background.

func (SQLite) OpenReadOnly

func (s SQLite) OpenReadOnly(path string) (*sql.DB, error)

OpenReadOnly implements Target.

type Target

type Target interface {
	// Name names the engine in reports.
	Name() string
	// Open opens or creates the database at path for the single writer,
	// configured as the store would configure it.
	Open(path string) (*sql.DB, error)
	// OpenReadOnly opens the database at path for reading, as a second
	// process would while the writer holds it.
	OpenReadOnly(path string) (*sql.DB, error)
	// ColumnType spells the SQL type of a column holding values of the
	// kind, as the generated tables of ADR-0014 would declare it.
	ColumnType(k formula.Kind) string
	// FillRates fills the rates table of the lookup benchmark with n rows
	// whose code is "K" and seven digits and whose rate is the row number
	// times 0.001, in the engine's own bulk way, on the given connection.
	FillRates(ctx context.Context, conn Execer, n int) error
	// FileSize reports the bytes the database at path occupies on disk,
	// its log included.
	FileSize(path string) int64
}

Target is one storage engine under measurement. database/sql carries the statements; the target spells what differs between engines.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL