Documentation
¶
Overview ¶
Package bench is the end-to-end performance suite: the scenarios behind the budgets of ADR-0012, written against database/sql so that the same scenario runs on every storage engine under consideration.
Two scenarios open Phase 2, as ADR-0017 and ADR-0020 ask:
- The primary-store benchmark (RunStore) writes commits in the shape of ADR-0014 and ADR-0026, the current row, the history row, the child rows and the event rows in one transaction, and reports latency and throughput, unbatched and batched. Its result decides the transactional store.
- The point-lookup benchmark (RunLookup) measures one keyed lookup against a reference table on a Go map and through a prepared statement, at several table sizes. Its result fixes the size up to which an auto formula's LOOKUP is served from memory.
The SQLite target lives here; the DuckDB target lives in the analytics module, which is where cgo is allowed. The tests of each module run a small smoke scenario always and the full campaign when FADODB_BENCH is set; see the tests for the variables.
Index ¶
- Constants
- func Campaign(ctx context.Context, target Target, dir, which string, w io.Writer) (bool, error)
- func CampaignDir() string
- func FileSizeOf(paths ...string) int64
- func HoldAndProbe(ctx context.Context, target Target, path string, probeTest string) (string, error)
- func Probe(target Target, path string) (int64, error)
- func Schema(t Target) []string
- func Verify(ctx context.Context, db *sql.DB, o Options) error
- type Column
- type Execer
- type LookupReport
- type Options
- type Report
- type SQLite
- func (SQLite) ColumnType(k formula.Kind) string
- func (SQLite) FileSize(path string) int64
- func (SQLite) FillRates(ctx context.Context, conn Execer, n int) error
- func (s SQLite) Name() string
- func (s SQLite) Open(path string) (*sql.DB, error)
- func (s SQLite) OpenReadOnly(path string) (*sql.DB, error)
- type Target
Constants ¶
const DefaultWALLimit = 64 << 20
DefaultWALLimit is the write-ahead log size at which the background checkpointer truncates: sixteen times SQLite's own threshold of 1,000 pages, so that the pause the truncation costs comes sixteen times less often than the automatic checkpoint's.
const Header = "| Target | Mode | Work | Throughput | p50 | p90 | p99 | max | File |\n|---|---|---|---|---|---|---|---|---|"
Header is the header of the table String rows belong to.
const LookupHeader = "| Engine | Rows | Go map | Map heap | Prepared statement |\n|---|---|---|---|---|"
LookupHeader is the header of the table String rows belong to.
Variables ¶
This section is empty.
Functions ¶
func Campaign ¶
Campaign runs the store scenario, the lookup scenario or both for a target, as which says ("store", "lookup" or "store,lookup"), writing its databases under dir and the Markdown tables to w. It returns false without running when which is empty, or when FADODB_BENCH_TARGET is set and the target's name does not contain it. The tests pass FADODB_BENCH as which; FADODB_BENCH_COMMITS sizes the store campaign (default 100,000) and FADODB_BENCH_ITERS the lookups per table size (default 50,000).
dir must be on the disk the numbers are meant for. The system's temporary directory is the wrong place: on many Linux machines /tmp is a tmpfs in memory, where an fsync costs nothing and every row of the store table is wrong by an order of magnitude.
func CampaignDir ¶
func CampaignDir() string
CampaignDir is the directory the campaign writes to: FADODB_BENCH_DIR, or the working directory, which under go test is the package's, on the disk the checkout is on. Never the temporary directory; see Campaign.
func FileSizeOf ¶
FileSizeOf sums the sizes of the files at the paths that exist.
func HoldAndProbe ¶
func HoldAndProbe(ctx context.Context, target Target, path string, probeTest string) (string, error)
HoldAndProbe writes a small scenario into path, keeps writing on the single connection while the probe command runs, and returns the probe's combined output and its error. The probe command is the test binary re-run on its probe test with FADODB_BENCH_PROBE naming the file, so that the reader is another process, which is the question of ADR-0017.
func Probe ¶
Probe opens the database at path read-only, as a second process does, and counts the documents.
func Schema ¶
Schema returns the DDL of the scenario for a target: the current table, the history table, the child table, events and event_docs, with the indexes ADR-0014 and ADR-0026 name.
Types ¶
type Execer ¶
type Execer interface {
ExecContext(ctx context.Context, query string, args ...any) (sql.Result, error)
}
Execer is what a target writes through: a *sql.Conn or a *sql.DB.
type LookupReport ¶
type LookupReport struct {
Target string
Rows int
// Map is the time of one lookup in a map[string]formula.Value built
// from the table; MapBytes the heap the map and its keys took, as
// the runtime measured it.
Map time.Duration
MapBytes int64
// Stmt is the time of one prepared-statement lookup on the engine.
Stmt time.Duration
}
LookupReport is the result of the point-lookup benchmark at one table size: the cost of one keyed lookup on a Go map holding the table and through a prepared statement on the engine.
func RunLookup ¶
func RunLookup(ctx context.Context, t Target, db *sql.DB, sizes []int, iters int) ([]LookupReport, error)
RunLookup measures one keyed lookup at each table size: the rates table is filled by the target, loaded into a Go map as the in-memory index of ADR-0016 would hold it, and then looked up iters times by pseudo-random key, once on the map and once through a prepared SELECT on a pinned connection. The keys found must carry the rate the fill wrote, which checks that both paths read the same table.
func (LookupReport) String ¶
func (r LookupReport) String() string
String renders the report as one Markdown table row.
type Options ¶
type Options struct {
// Commits is the number of commits to write.
Commits int
// Documents is the number of documents the commits are spread over.
// The first Documents commits create, the rest update, so that every
// document receives Commits/Documents versions.
Documents int
// Lines is the number of child rows per version.
Lines int
// Batch is the number of commits per transaction; 1 is unbatched, one
// transaction per commit, which is how a user's edit commits.
Batch int
// Seed seeds the values.
Seed uint64
}
Options size a store run.
type Report ¶
type Report struct {
Target string
Commits int
Documents int
Lines int
Batch int
// Elapsed is the wall time of all commits; PerSecond the throughput.
Elapsed time.Duration
PerSecond float64
// The latency percentiles of one commit (Batch 1) or of one batch.
P50, P90, P99, Max time.Duration
// FileBytes is the size of the database on disk after the run.
FileBytes int64
}
Report is the result of one store run.
func RunStore ¶
RunStore creates the scenario schema in db, which Open gave for path, and writes the commits of o, measuring each transaction. The run pins one connection, as the store's single writer will, and prepares every statement once. The statements are closed before the connection goes back to the pool, where another goroutine of the target may take it.
type SQLite ¶
type SQLite struct {
// Synchronous is the PRAGMA synchronous setting: NORMAL, the
// architecture page's choice, where a commit is durable against a
// crash of the process but the last commits may be lost on power
// failure, or FULL, where every commit is fsynced.
Synchronous string
// BackgroundCheckpoints turns SQLite's automatic checkpoint off, where
// the committing connection copies the log into the database file
// every 1,000 pages and pays that fsync inside a commit, and runs
// PRAGMA wal_checkpoint(PASSIVE) on a second connection every 100 ms
// instead, off the commit path. A passive checkpoint never resets the
// log while a writer keeps appending, so when the log passes
// WALLimit bytes the checkpointer runs wal_checkpoint(TRUNCATE),
// which waits for the writer's transaction, holds the next one back
// while it copies the rest, and empties the log.
BackgroundCheckpoints bool
// WALLimit is the log size that triggers a truncating checkpoint; 0
// is 64 MB.
WALLimit int64
}
SQLite is the transactional store candidate of ADR-0017 as the architecture page configures it: WAL mode, synchronous NORMAL, one writer connection.
func (SQLite) ColumnType ¶
ColumnType implements Target: SQLite's storage classes, a date as INTEGER days and a datetime as INTEGER seconds (ADR-0023), an exact value as its INTEGER mantissa (ADR-0022).
type Target ¶
type Target interface {
// Name names the engine in reports.
Name() string
// Open opens or creates the database at path for the single writer,
// configured as the store would configure it.
Open(path string) (*sql.DB, error)
// OpenReadOnly opens the database at path for reading, as a second
// process would while the writer holds it.
OpenReadOnly(path string) (*sql.DB, error)
// ColumnType spells the SQL type of a column holding values of the
// kind, as the generated tables of ADR-0014 would declare it.
ColumnType(k formula.Kind) string
// FillRates fills the rates table of the lookup benchmark with n rows
// whose code is "K" and seven digits and whose rate is the row number
// times 0.001, in the engine's own bulk way, on the given connection.
FillRates(ctx context.Context, conn Execer, n int) error
// FileSize reports the bytes the database at path occupies on disk,
// its log included.
FileSize(path string) int64
}
Target is one storage engine under measurement. database/sql carries the statements; the target spells what differs between engines.