Documentation
¶
Overview ¶
Package sample reads small, bounded samples of rows. Every query is a plain SELECT with a LIMIT, ordered by the primary key where there is one, so the server reads only a few index pages per query.
Index ¶
- func IntegerPK(t *model.Table) *model.Column
- func IsInteger(c *model.Column) bool
- func IsText(c *model.Column) bool
- func QuoteIdent(s string) string
- func Sampleable(c *model.Column) bool
- func ShortBinary(c *model.Column) bool
- func ValueExpr(c *model.Column, n int) string
- type Batch
- type KV
- type Lookup
- type PKValue
- type Querier
- type Sample
- type Sampler
- func (sm *Sampler) KeyValues(ctx context.Context, t *model.Table, keyCol, valueCol *model.Column, ...) ([]string, error)
- func (sm *Sampler) KeyValuesBatch(ctx context.Context, t *model.Table, keyCol, valueCol *model.Column, ...) (Batch, error)
- func (sm *Sampler) Keys(ctx context.Context, t *model.Table, keyCol *model.Column, max int) (keys []string, more bool, err error)
- func (sm *Sampler) ProbeScan(ctx context.Context, t *model.Table, keyCol, valueCol *model.Column, ...) (Batch, error)
- func (sm *Sampler) Spread(ctx context.Context, t *model.Table, exprs []string, args []any, ...) (*SpreadResult, error)
- func (sm *Sampler) Table(ctx context.Context, t *model.Table, limit int) (*Sample, error)
- func (sm *Sampler) ValuesByPK(ctx context.Context, t *model.Table, pk, c *model.Column, chars int, ...) ([]PKValue, error)
- func (sm *Sampler) Where(ctx context.Context, t *model.Table, c *model.Column, where string, limit int) ([]string, error)
- func (sm *Sampler) Whole(ctx context.Context, t *model.Table, exprs []string, args []any, chunk int, ...) (int64, error)
- func (sm *Sampler) WholeTable(ctx context.Context, t *model.Table, keyCol, valueCol *model.Column, ...) (rows []KV, complete bool, read int64, err error)
- type SpreadResult
- type SpreadRow
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func QuoteIdent ¶
QuoteIdent quotes a table or column name for MySQL.
func Sampleable ¶
Sampleable reports whether the scanner reads a column's values. Binary, spatial and vector columns are skipped.
func ShortBinary ¶ added in v0.4.0
ShortBinary reports whether a column is binary or varbinary of at most 16 bytes, the size of a stored IPv4 or IPv6 address.
Types ¶
type Batch ¶ added in v0.3.0
type Batch struct {
Rows map[int][]KV // the rows of each lookup, by its index
// Complete marks the lookups that ran to their end. The others were
// cut short by the byte budget or an error, or never ran, and may have
// some rows or none.
Complete []bool
Bytes int64 // keys and values read
}
Batch is what KeyValuesBatch or ProbeScan read for a list of lookups.
type KV ¶ added in v0.3.0
KV is one row of a key/value table: its key, its primary key (empty without a single-column one) and its value, cut on the server.
type Lookup ¶ added in v0.3.0
type Lookup struct {
Key string
Prefix bool // Key is a prefix: rows whose key starts with it
Limit int
}
Lookup asks for the rows of one key, or of every key with a prefix.
type Querier ¶
type Querier interface {
QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
QueryRowContext(ctx context.Context, query string, args ...any) *sql.Row
}
Querier is the part of *sql.DB the package needs.
type Sample ¶
type Sample struct {
Columns []*model.Column
Rows [][]sql.NullString
Method string
// Hex marks the columns read as HEX(column), in the order of Columns.
Hex []bool
}
Sample is the set of rows read from one table.
type Sampler ¶
type Sampler struct {
DB Querier
MaxValueLen int // longer text values are cut on the server
Timeout time.Duration // client-side limit per query
// Hex, when set, picks short binary columns (see ShortBinary) that Table
// reads as HEX(column), such as IP addresses stored as bytes. No other
// read selects a binary column.
Hex func(t *model.Table, c *model.Column) bool
}
Sampler reads samples from one database.
func (*Sampler) KeyValues ¶
func (sm *Sampler) KeyValues(ctx context.Context, t *model.Table, keyCol, valueCol *model.Column, key string, limit int) ([]string, error)
KeyValues reads up to limit values of valueCol from rows where keyCol equals key. It is used to look at a known meta key directly when the window sample happened to miss it.
func (*Sampler) KeyValuesBatch ¶ added in v0.3.0
func (sm *Sampler) KeyValuesBatch(ctx context.Context, t *model.Table, keyCol, valueCol *model.Column, lookups []Lookup, valueLen int, maxBytes int64) (Batch, error)
KeyValuesBatch runs lookups in UNION ALL batches, each part an indexed read with its own LIMIT. It stops at maxBytes of keys and values, and at the first failed batch.
func (*Sampler) Keys ¶ added in v0.3.0
func (sm *Sampler) Keys(ctx context.Context, t *model.Table, keyCol *model.Column, max int) (keys []string, more bool, err error)
Keys lists the distinct keys of a key/value table in key order, up to max. more reports that there were more.
func (*Sampler) ProbeScan ¶ added in v0.3.0
func (sm *Sampler) ProbeScan(ctx context.Context, t *model.Table, keyCol, valueCol *model.Column, lookups []Lookup, valueLen int, maxBytes int64) (Batch, error)
ProbeScan reads, in one pass over a table whose key column has no index, up to Limit rows for each lookup. The pass reads keys and primary keys only and is cancelled once every lookup has its rows; the values are then read by primary key, in lookup order. Without a single-column primary key the pass reads the values itself. Only the rows kept count against maxBytes, so a busy family can't crowd out a rare key, and rows whose values weren't read are dropped.
func (*Sampler) Spread ¶ added in v0.12.0
func (sm *Sampler) Spread(ctx context.Context, t *model.Table, exprs []string, args []any, points, perPoint int, pick func(n uint64) uint64) (*SpreadResult, error)
Spread reads a sample of a table's rows, with one read per point over the integer primary key's range. The range is cut into points equal slices, and each gets a random spot in it and the perPoint rows from there, so a row is about as likely as any other to be read, and any unbroken run of rows that covers two slices is certain to be. Each point is one indexed seek, and no statement reads more than its LIMITs say.
A table that fits in points*perPoint rows is read whole instead, in key order; one with no single integer primary key gets its first rows. Gaps in the key make points land on the same rows, which are dropped, and on the rows right after each gap. So under half the points with a row of their own makes the method model.SampleThin, and so do rows far from their points, as with keys in tight clusters, where every point can have a row and all of them be first in their cluster.
args are the arguments of exprs, and are only allowed without an integer key or with a table read whole: a statement of many points would repeat them for each. On an error, the rows read so far come back with it. pick picks a number below n; nil picks at random. Reports must never print what comes back.
func (*Sampler) Table ¶
Table samples up to limit rows. With a single integer primary key the rows come from three windows: the oldest, the middle and the newest rows, since old and new data often differ in shape. Otherwise the first rows the server returns are used.
func (*Sampler) ValuesByPK ¶ added in v0.12.0
func (sm *Sampler) ValuesByPK(ctx context.Context, t *model.Table, pk, c *model.Column, chars int, ids []string) ([]PKValue, error)
ValuesByPK reads one column of the rows with these integer primary keys. An id that isn't a whole number is refused, as the ids go into the statement's text.
func (*Sampler) Where ¶ added in v0.3.0
func (sm *Sampler) Where(ctx context.Context, t *model.Table, c *model.Column, where string, limit int) ([]string, error)
Where reads up to limit values of one column from the rows that match a condition. The condition comes from a preset, never from the database.
func (*Sampler) Whole ¶ added in v0.12.0
func (sm *Sampler) Whole(ctx context.Context, t *model.Table, exprs []string, args []any, chunk int, fn func([]sql.RawBytes) error) (int64, error)
Whole hands every row of a table to fn, without its primary key. A table with a single integer primary key is read in chunks, each a statement of its own, so no statement runs long and no snapshot or lock outlives one; the first has no lower bound, so negative keys aren't skipped. Any other table is one statement, which a time limit may stop. Whole returns the rows it handed over, with the error that stopped it, if any.
func (*Sampler) WholeTable ¶ added in v0.3.0
func (sm *Sampler) WholeTable(ctx context.Context, t *model.Table, keyCol, valueCol *model.Column, valueLen, maxRows int, maxBytes int64) (rows []KV, complete bool, read int64, err error)
WholeTable reads the key and value of up to maxRows rows, in primary key order. It reports whether it read the whole table: false when there are more rows than maxRows, or when maxBytes of values were read first. At the byte cap it cancels the statement, since closing the rows would make the server send the rest anyway.
type SpreadResult ¶ added in v0.12.0
type SpreadResult struct {
Rows []SpreadRow
Method string // model.SampleSpread, SampleThin, SampleAllRows or SampleFirstRows
}
SpreadResult is what a Spread read.
Source Files