Documentation
¶
Overview ¶
Package data is refract's data layer: columnar, batch-oriented access to a table of values.
The interface returns whole typed columns, never one value at a time. Scalar access is the single easiest way to make a plotting library slow, and a columnar shape is also what lets a []float64-backed source be borrowed instead of copied.
Index ¶
- Variables
- func AnyNull(mask []bool) bool
- func FormatNumber(v float64) string
- func GroupBy(src Source, col string) (keys []string, rows [][]int, ok bool)
- func IsNull(mask []bool, i int) bool
- func Labels(src Source, col string) ([]string, bool)
- func NullMask(src Source, name string) (null []bool, ok bool)
- func Origins(src Source) []int
- type Nulls
- type Source
- type Stream
- func (s *Stream) Append(vals ...float64) error
- func (s *Stream) AppendTime(t time.Time, vals ...float64) error
- func (s *Stream) Columns() []string
- func (s *Stream) Len() int
- func (s *Stream) Reset()
- func (s *Stream) Snapshot() Source
- func (s *Stream) Source() Source
- func (s *Stream) Window(n int) *Stream
- type Subset
- type Table
- func (t *Table) Columns() []string
- func (t *Table) Float64(name string, v []float64) *Table
- func (t *Table) Float64Column(name string) ([]float64, bool)
- func (t *Table) Len() int
- func (t *Table) Nulls(name string) ([]bool, bool)
- func (t *Table) String(name string, v []string) *Table
- func (t *Table) StringColumn(name string) ([]string, bool)
- func (t *Table) Time(name string, v []time.Time) *Table
- func (t *Table) TimeColumn(name string) ([]time.Time, bool)
- func (t *Table) WithNulls(name string, null []bool) *Table
Constants ¶
This section is empty.
Variables ¶
var ErrColumnCount = fmt.Errorf("refract/data: wrong number of values")
ErrColumnCount reports an Append whose value count does not match the stream's columns.
Functions ¶
func AnyNull ¶ added in v1.3.0
AnyNull reports whether mask marks any row at all.
It is what lets a reader decide between the borrowed column and a copy: a mask that marks nothing changes no value, so there is nothing to write and the caller's slice is handed on untouched.
func FormatNumber ¶ added in v0.3.0
FormatNumber is how a numeric value is spelled when it is used as a category name. Both faceting and a categorical axis go through it, so a panel key and an axis tick for the same number are the same string.
func GroupBy ¶ added in v0.3.0
GroupBy splits src into groups by the values of a column, returning the distinct values in first-appearance order and the row numbers of each.
The column may be textual, numeric or temporal; whichever it is, the group key is its formatted label, so a facet over a numeric column gets one panel per distinct number rather than a continuous axis. ok is false if src has no such column.
First-appearance order rather than sorted order is deliberate: it is the one ordering that is stable under every column type and lets a caller control panel order by ordering its rows.
func IsNull ¶ added in v1.3.0
IsNull reports whether row i of mask is absent, tolerating a nil or short mask.
A mask is as long as the column it describes, so the length test is not defensive padding: Rows gathers a mask along with the column it belongs to, and a caller composing sources by hand may hand over one that stops early. A row past the end has a value, because that is what the column says.
func Labels ¶ added in v0.3.0
Labels reads a column as one text label per row, whatever its type.
It is the shared spelling of "what does this row say in that column" — faceting groups by it, and a categorical axis encodes by it, so the two agree about what counts as one category.
func NullMask ¶ added in v1.3.0
NullMask reports which of a column's rows are absent, or ok == false when src cannot say or has nothing to say.
It is the shared spelling of the question, the way Labels is for "what does this row say in that column": every reader asks through this so that one Source implementing Nulls answers every channel at once.
func Origins ¶ added in v0.5.0
Origins returns the mapping from src's rows to the rows of the table it was cut from, or nil if src is not a cut of anything.
One level, not the whole chain: faceting cuts once, and a caller that has composed cuts of cuts knows it has and can compose the mappings the same way.
Types ¶
type Nulls ¶ added in v1.3.0
type Nulls interface {
// Nulls returns one flag per row, true where the column has no value.
//
// ok is false when the column does not exist *or has no nulls at all*.
// That second half is what keeps the zero-copy path in
// [Float64Columns] intact: a reader asks first and copies only when the
// answer is yes, so a table without nulls costs exactly what it did.
// The result is read-only.
Nulls(name string) (null []bool, ok bool)
}
Nulls is implemented by a Source that can say which of a column's values are absent, rather than merely which are unrepresentable.
A numeric column needs no such thing: a missing number is NaN, every scale refuses to place it, and github.com/timzifer/refract/geom.OnMissing is written against exactly that. A *text* column has no NaN — a null read back as "" is indistinguishable from a genuine empty string, and on an ordinal axis it becomes a band of its own — and a *time* column has none either: a null read back as the zero time is the year 1, which is a real instant that stretches a domain over two millennia. Both were silent: the chart drew, and it drew something nobody measured.
This is the optional interface Source's own documentation promises for a capability arriving after the freeze, and it is asked for with a type assertion. A Source that does not implement it has no nulls beyond the NaNs already in its numbers, which is what every Source in this package that predates it means.
What a null means ¶
One rule, everywhere a column is read: **a null is a missing value.** On a position axis the row has no place, so [OnMissing] decides whether the chart gaps, interpolates or errors — the same three answers it already gives a NaN. On a colour channel the row takes the scale's undefined colour. In a group or facet column the row belongs to no series and no panel, so it is not drawn: a category named "" is not a reading, and inventing one is how the zero time got onto an axis in the first place. See [ADR 0034](../docs/adr/0034-null-values.md).
type Source ¶
type Source interface {
// Len reports the number of rows. Every column has this length.
Len() int
// Columns lists the available column names. The order is stable across
// calls on the same Source.
Columns() []string
// Float64Column returns a numeric column by name. ok is false if the
// column does not exist or is not numeric.
Float64Column(name string) (data []float64, ok bool)
// TimeColumn returns a time column by name. ok is false if the column does
// not exist or is not temporal.
TimeColumn(name string) (data []time.Time, ok bool)
// StringColumn returns a categorical column by name. ok is false if the
// column does not exist or is not textual.
StringColumn(name string) (data []string, ok bool)
}
Source exposes columnar, batch access to a table.
Implementations return read-only views: the caller must not mutate a returned slice, and refract never does. An implementation that already holds its data as a Go slice should return that slice directly rather than copying.
Stability ¶
Source is implemented outside this module, so it never gains a method. A fourth column kind — exact integers, booleans, durations — arrives as an optional interface beside it, the way Subset did, and a caller that wants one asks for it with a type assertion and falls back when it is absent.
func Float64Columns ¶
Float64Columns builds a Source over the given numeric columns.
The slices are borrowed, not copied: the returned Source aliases the caller's memory, and mutating it afterwards mutates what refract will plot. All columns must have the same length; Float64Columns panics otherwise, because a ragged table is a programming error rather than a runtime condition.
func Rows ¶ added in v0.3.0
Rows returns a Source over the rows of src named by idx, in the order given.
It is how faceting cuts one table into panels: a facet reads the column it splits on, groups the row numbers, and hands each group to Rows. Out-of-range indices are dropped rather than panicking, because they come from a grouping pass rather than from the caller.
The result materialises the rows it is asked for. That is a copy — the zero-copy promise in Float64Columns is about the whole-column path, and a gathered subset has no contiguous slice to borrow. Columns are gathered lazily, so a table with forty columns and a chart that reads three copies three.
type Stream ¶ added in v0.5.0
type Stream struct {
// contains filtered or unexported fields
}
Stream is a table a producer appends to while a renderer draws.
It is deliberately not a Source. A Source is read column by column, over several calls, and a table being appended to between two of them is a table that disagrees with itself — so the only way to draw a Stream is to freeze it:
st := data.NewStream("t", "y").Window(2000)
p.Add(geom.Line(st.Source(), geom.X("t"), geom.Y("y")))
go func() {
for sample := range samples {
st.Append(scale.Nanos(sample.At), sample.Value)
}
}()
for range ticker.C {
st.Snapshot() // freeze what has arrived
live.Draw() // draw the frozen view
}
Stream.Source hands back a Source that reads the most recent snapshot, so a layer is built once rather than per frame. Stream.Snapshot is what moves it forward.
Two buffers, one copy ¶
Snapshot copies the live rows into a buffer the renderer is not reading and swaps the two. That is one copy per frame — not per row, and not per column read — and the buffers are reused, so a steady stream costs no allocations after the first frame at each size.
The one rule ¶
Append may be called from any goroutine, at any time. Snapshot may not be called while a render is in flight: it is the swap, and swapping the table out from under a half-drawn chart is the race this type exists to remove. Producer appends, renderer snapshots and draws, in that order.
Columns ¶
A Stream carries numeric columns only. A timestamp is its Unix nanoseconds — github.com/timzifer/refract/scale.Nanos converts one, and github.com/timzifer/refract/scale.Time maps that domain — so a time series needs no second column type and no per-row allocation to carry one.
func NewStream ¶ added in v0.5.0
NewStream returns an empty stream over the named numeric columns.
The order of the names is the order Stream.Append takes values in. It panics on no columns or a duplicate name, which are programming errors rather than runtime conditions.
func (*Stream) Append ¶ added in v0.5.0
Append adds one row. The values are positional, in the order NewStream was given.
It is safe to call from any goroutine, and it allocates nothing in the steady state: the row is copied into buffers that are already the right size.
func (*Stream) AppendTime ¶ added in v0.5.0
AppendTime is Stream.Append with the first column given as a timestamp, which is what the first column of a live chart almost always is. The timestamp becomes its Unix nanoseconds, the domain a github.com/timzifer/refract/scale.Time axis maps.
func (*Stream) Len ¶ added in v0.5.0
Len reports how many rows are live. It is the producer's view, which may be ahead of what the last snapshot froze.
func (*Stream) Reset ¶ added in v0.5.0
func (s *Stream) Reset()
Reset empties the stream, keeping its buffers. The snapshot a renderer is holding is untouched until the next Stream.Snapshot.
func (*Stream) Snapshot ¶ added in v0.5.0
Snapshot freezes the live rows and returns a Source over them.
The Source Stream.Source returned reads the same frozen rows, so a layer built once draws whatever the last Snapshot froze.
The returned Source is valid until the next call to Snapshot, which reuses its memory. Call it between frames, never during one.
func (*Stream) Source ¶ added in v0.5.0
Source returns a Source over the stream's most recent snapshot.
It is stable: the same value reads every frame, and what it reads changes only when Stream.Snapshot is called. That is what lets a layer be built once, before the first row has even arrived.
func (*Stream) Window ¶ added in v0.5.0
Window caps the stream at the last n rows, dropping the oldest as new ones arrive. It returns s so the call can be chained onto NewStream.
A window is what makes a stream a *stream* rather than a log: a live chart shows the last few thousand samples, and keeping every sample since the process started is a memory leak with a plot attached. Zero means unbounded, which is the default.
Setting a window smaller than the rows already held drops the oldest of them, because that is what the window means.
type Subset ¶ added in v0.5.0
type Subset interface {
// SourceRows returns, for each of this source's rows, the row it came from
// in the source it selects from. The result is read-only.
SourceRows() []int
}
Subset is implemented by a Source that is a selection of another source's rows, so that a row number can be traced back to the table it came from.
It exists because faceting makes such a source without the caller ever seeing it: github.com/timzifer/refract.Plot.Facet cuts each layer down to its panel's rows with Rows, and a row number relative to that cut is a row number in a table nobody holds. A geom reporting where its rows landed resolves them through this first, so what comes out is a row of the table that was handed in.
It is an optional interface. A Source that is not a selection of another does not implement it, and its rows are already its own.
type Table ¶
type Table struct {
// contains filtered or unexported fields
}
Table is a Source that mixes numeric, temporal and categorical columns.
It is the general-purpose implementation: use it when a chart plots time or a category against values, which is the common case for the Time and Ordinal scales.
func (*Table) Float64 ¶
Float64 adds a numeric column, borrowing the slice. It returns t so calls can be chained. It panics if the column length disagrees with columns already added, or if the name is already taken.
func (*Table) Float64Column ¶
Float64Column returns a numeric column by name.
func (*Table) Nulls ¶ added in v1.3.0
Nulls implements Nulls. ok is false for a column with no nulls, which is every column of a table that was never told about one.
func (*Table) String ¶ added in v0.2.0
String adds a categorical column, borrowing the slice. It returns t so calls can be chained. It panics if the column length disagrees with columns already added, or if the name is already taken.
Plot such a column against a [scale.Ordinal] axis; a continuous scale has no position for a category name and a geom says so rather than guessing one.
func (*Table) StringColumn ¶ added in v0.2.0
StringColumn returns a categorical column by name.
func (*Table) Time ¶
Time adds a temporal column, borrowing the slice. It returns t so calls can be chained. It panics if the column length disagrees with columns already added, or if the name is already taken.
func (*Table) TimeColumn ¶
TimeColumn returns a temporal column by name.
func (*Table) WithNulls ¶ added in v1.3.0
WithNulls marks rows of an existing column as absent, borrowing the mask. It returns t so calls can be chained. It panics if the column does not exist or the mask is not one flag per row.
A text or temporal column needs this because it has no NaN to be missing with: "" is a string somebody may have measured and the zero time is an instant, so absence has to be said beside the values rather than inside them. A numeric column may use it too, and the two spellings agree — a NaN and a marked row are both missing, and neither outranks the other.
A mask that marks nothing is not stored: Nulls answers "no nulls" either way, and a reader that asked would otherwise copy a column to change none of it.
Source Files