source

package
v6.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 12, 2026 License: Apache-2.0, Apache-2.0 Imports: 0 Imported by: 0

Documentation

Overview

Package source defines the row-production contract that every load-time data generator implements and every driver consumes. It is the seam that lets a driver stream rows without knowing whether they come from a ported canonical generator such as TPC-H dbgen (pkg/datagen/tpchgen) or TPC-DS dsdgen (pkg/datagen/tpcdsgen), or from a typed gen.BatchSource adapted via the common.BatchRowSource shim.

The contract is deliberately tiny: a RowSource is a forward iterator with a known column order, and a Partitionable hands out independent RowSources for disjoint row ranges so the load can fan out across workers.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Partitionable

type Partitionable interface {
	Units() int64
	TotalRows() int64
	Partition(start, count int64) (RowSource, error)
}

Partitionable is a generator that can be split into independent RowSources.

Units is the number of partitionable units — the range the loader carves into chunks and the unit of Partition's (start, count). For row-at-a-time generators (TPC-H itself, TPC-DS facts) a unit is one output row, so Units == TotalRows. For generators whose unit fans out into many rows (TPC-H: one order -> many lineitems, one part -> many partsupps) a unit is one entity and Units < TotalRows.

TotalRows is the number of output rows the generator emits in full. It is used for progress and stats, not for chunking. For fan-out tables whose exact row count is only known after generation it may be the spec-nominal estimate.

Partition returns a RowSource for units [start, start+count) — already positioned at start, so workers need no warm-up. A negative count means "from start to the end" (the single-worker path).

Partition implementations must be safe to call and drain concurrently across chunks (the instanced TPC-H generator and TPC-DS streams own their per-partition state).

type RowSource

type RowSource interface {
	Columns() []string
	Next() ([]any, error)
}

RowSource is a forward, single-pass iterator over generated rows. Columns reports the emission order; Next yields one row at a time and returns io.EOF when the (possibly partition-bounded) range is exhausted.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL