sample

package
v0.12.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 10, 2026 License: Apache-2.0 Imports: 10 Imported by: 0

Documentation

Overview

Package sample reads small, bounded samples of rows. Every query is a plain SELECT with a LIMIT, ordered by the primary key where there is one, so the server reads only a few index pages per query.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IntegerPK added in v0.12.0 

func IntegerPK(t *model.Table) *model.Column

IntegerPK returns the single integer primary key, or nil.

func IsInteger 

func IsInteger(c *model.Column) bool

IsInteger reports whether a column holds whole numbers.

func IsText added in v0.3.0 

func IsText(c *model.Column) bool

IsText reports whether a column holds text, which the server can cut.

func QuoteIdent 

func QuoteIdent(s string) string

QuoteIdent quotes a table or column name for MySQL.

func Sampleable 

func Sampleable(c *model.Column) bool

Sampleable reports whether the scanner reads a column's values. Binary, spatial and vector columns are skipped.

func ShortBinary added in v0.4.0 

func ShortBinary(c *model.Column) bool

ShortBinary reports whether a column is binary or varbinary of at most 16 bytes, the size of a stored IPv4 or IPv6 address.

func ValueExpr 

func ValueExpr(c *model.Column, n int) string

ValueExpr is the select expression for a column: text is cut to n characters on the server so huge values never cross the network.

Types

type Batch added in v0.3.0 

type Batch struct {
	Rows map[int][]KV // the rows of each lookup, by its index
	// Complete marks the lookups that ran to their end. The others were
	// cut short by the byte budget or an error, or never ran, and may have
	// some rows or none.
	Complete []bool
	Bytes    int64 // keys and values read
}

Batch is what KeyValuesBatch or ProbeScan read for a list of lookups.

type KV added in v0.3.0 

type KV struct {
	Key   string
	PK    string
	Value string
	Null  bool
}

KV is one row of a key/value table: its key, its primary key (empty without a single-column one) and its value, cut on the server.

type Lookup added in v0.3.0 

type Lookup struct {
	Key    string
	Prefix bool // Key is a prefix: rows whose key starts with it
	Limit  int
}

Lookup asks for the rows of one key, or of every key with a prefix.

type PKValue added in v0.12.0 

type PKValue struct {
	PK    string
	Value string
	Valid bool
}

PKValue is one value read by primary key.

type Querier 

type Querier interface {
	QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
	QueryRowContext(ctx context.Context, query string, args ...any) *sql.Row
}

Querier is the part of *sql.DB the package needs.

type Sample 

type Sample struct {
	Columns []*model.Column
	Rows    [][]sql.NullString
	Method  string
	// Hex marks the columns read as HEX(column), in the order of Columns.
	Hex []bool
}

Sample is the set of rows read from one table.

func (*Sample) Index 

func (s *Sample) Index(column string) int

Index returns the position of a column in the sample, or -1.

func (*Sample) Values 

func (s *Sample) Values(i int) []string

Values returns the non-NULL values of one sampled column.

type Sampler 

type Sampler struct {
	DB          Querier
	MaxValueLen int           // longer text values are cut on the server
	Timeout     time.Duration // client-side limit per query
	// Hex, when set, picks short binary columns (see ShortBinary) that Table
	// reads as HEX(column), such as IP addresses stored as bytes. No other
	// read selects a binary column.
	Hex func(t *model.Table, c *model.Column) bool
}

Sampler reads samples from one database.

func (*Sampler) KeyValues 

func (sm *Sampler) KeyValues(ctx context.Context, t *model.Table, keyCol, valueCol *model.Column, key string, limit int) ([]string, error)

KeyValues reads up to limit values of valueCol from rows where keyCol equals key. It is used to look at a known meta key directly when the window sample happened to miss it.

func (*Sampler) KeyValuesBatch added in v0.3.0 

func (sm *Sampler) KeyValuesBatch(ctx context.Context, t *model.Table, keyCol, valueCol *model.Column, lookups []Lookup, valueLen int, maxBytes int64) (Batch, error)

KeyValuesBatch runs lookups in UNION ALL batches, each part an indexed read with its own LIMIT. It stops at maxBytes of keys and values, and at the first failed batch.

func (*Sampler) Keys added in v0.3.0 

func (sm *Sampler) Keys(ctx context.Context, t *model.Table, keyCol *model.Column, max int) (keys []string, more bool, err error)

Keys lists the distinct keys of a key/value table in key order, up to max. more reports that there were more.

func (*Sampler) ProbeScan added in v0.3.0 

func (sm *Sampler) ProbeScan(ctx context.Context, t *model.Table, keyCol, valueCol *model.Column, lookups []Lookup, valueLen int, maxBytes int64) (Batch, error)

ProbeScan reads, in one pass over a table whose key column has no index, up to Limit rows for each lookup. The pass reads keys and primary keys only and is cancelled once every lookup has its rows; the values are then read by primary key, in lookup order. Without a single-column primary key the pass reads the values itself. Only the rows kept count against maxBytes, so a busy family can't crowd out a rare key, and rows whose values weren't read are dropped.

func (*Sampler) Spread added in v0.12.0 

func (sm *Sampler) Spread(ctx context.Context, t *model.Table, exprs []string, args []any, points, perPoint int, pick func(n uint64) uint64) (*SpreadResult, error)

Spread reads a sample of a table's rows, with one read per point over the integer primary key's range. The range is cut into points equal slices, and each gets a random spot in it and the perPoint rows from there, so a row is about as likely as any other to be read, and any unbroken run of rows that covers two slices is certain to be. Each point is one indexed seek, and no statement reads more than its LIMITs say.

A table that fits in points*perPoint rows is read whole instead, in key order; one with no single integer primary key gets its first rows. Gaps in the key make points land on the same rows, which are dropped, and on the rows right after each gap. So under half the points with a row of their own makes the method model.SampleThin, and so do rows far from their points, as with keys in tight clusters, where every point can have a row and all of them be first in their cluster.

args are the arguments of exprs, and are only allowed without an integer key or with a table read whole: a statement of many points would repeat them for each. On an error, the rows read so far come back with it. pick picks a number below n; nil picks at random. Reports must never print what comes back.

func (*Sampler) Table 

func (sm *Sampler) Table(ctx context.Context, t *model.Table, limit int) (*Sample, error)

Table samples up to limit rows. With a single integer primary key the rows come from three windows: the oldest, the middle and the newest rows, since old and new data often differ in shape. Otherwise the first rows the server returns are used.

func (*Sampler) ValuesByPK added in v0.12.0 

func (sm *Sampler) ValuesByPK(ctx context.Context, t *model.Table, pk, c *model.Column, chars int, ids []string) ([]PKValue, error)

ValuesByPK reads one column of the rows with these integer primary keys. An id that isn't a whole number is refused, as the ids go into the statement's text.

func (*Sampler) Where added in v0.3.0 

func (sm *Sampler) Where(ctx context.Context, t *model.Table, c *model.Column, where string, limit int) ([]string, error)

Where reads up to limit values of one column from the rows that match a condition. The condition comes from a preset, never from the database.

func (*Sampler) Whole added in v0.12.0 

func (sm *Sampler) Whole(ctx context.Context, t *model.Table, exprs []string, args []any, chunk int, fn func([]sql.RawBytes) error) (int64, error)

Whole hands every row of a table to fn, without its primary key. A table with a single integer primary key is read in chunks, each a statement of its own, so no statement runs long and no snapshot or lock outlives one; the first has no lower bound, so negative keys aren't skipped. Any other table is one statement, which a time limit may stop. Whole returns the rows it handed over, with the error that stopped it, if any.

func (*Sampler) WholeTable added in v0.3.0 

func (sm *Sampler) WholeTable(ctx context.Context, t *model.Table, keyCol, valueCol *model.Column, valueLen, maxRows int, maxBytes int64) (rows []KV, complete bool, read int64, err error)

WholeTable reads the key and value of up to maxRows rows, in primary key order. It reports whether it read the whole table: false when there are more rows than maxRows, or when maxBytes of values were read first. At the byte cap it cancels the statement, since closing the rows would make the server send the rest anyway.

type SpreadResult added in v0.12.0 

type SpreadResult struct {
	Rows   []SpreadRow
	Method string // model.SampleSpread, SampleThin, SampleAllRows or SampleFirstRows
}

SpreadResult is what a Spread read.

type SpreadRow added in v0.12.0 

type SpreadRow struct {
	Point  int
	PK     string
	Values []sql.NullString
}

SpreadRow is one row a Spread read: the point it came from, its primary key (empty without a single integer one) and the values of the expressions asked for.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.