Documentation
¶
Overview ¶
Package data is refract's data layer: columnar, batch-oriented access to a table of values.
The interface returns whole typed columns, never one value at a time. Scalar access is the single easiest way to make a plotting library slow, and a columnar shape is also what lets a []float64-backed source be borrowed instead of copied.
Index ¶
- Variables
- func AnyNull(mask []bool) bool
- func FormatNumber(v float64) string
- func GroupBy(src Source, col string) (keys []string, rows [][]int, ok bool)
- func IsNull(mask []bool, i int) bool
- func Label(src Source, col string, row int) (string, bool)
- func Labels(src Source, col string) ([]string, bool)
- func NullMask(src Source, name string) (null []bool, ok bool)
- func Origins(src Source) []int
- type Alignment
- type Nulls
- type Source
- type Stream
- func (s *Stream) Append(vals ...float64) error
- func (s *Stream) AppendTime(t time.Time, vals ...float64) error
- func (s *Stream) Columns() []string
- func (s *Stream) Len() int
- func (s *Stream) Reset()
- func (s *Stream) Snapshot() Source
- func (s *Stream) Source() Source
- func (s *Stream) Window(n int) *Stream
- type Subset
- type Table
- func (t *Table) Columns() []string
- func (t *Table) Float64(name string, v []float64) *Table
- func (t *Table) Float64Column(name string) ([]float64, bool)
- func (t *Table) Len() int
- func (t *Table) Nulls(name string) ([]bool, bool)
- func (t *Table) String(name string, v []string) *Table
- func (t *Table) StringColumn(name string) ([]string, bool)
- func (t *Table) Time(name string, v []time.Time) *Table
- func (t *Table) TimeColumn(name string) ([]time.Time, bool)
- func (t *Table) WithNulls(name string, null []bool) *Table
- type Tween
- type TweenOption
Constants ¶
This section is empty.
Variables ¶
var ErrColumnCount = fmt.Errorf("refract/data: wrong number of values")
ErrColumnCount reports an Append whose value count does not match the stream's columns.
var ErrNoKeyColumn = errors.New("refract/data: no such key column")
ErrNoKeyColumn reports a key column neither table has, or one of a type that cannot name a row.
Functions ¶
func AnyNull ¶ added in v1.3.0
AnyNull reports whether mask marks any row at all.
It is what lets a reader decide between the borrowed column and a copy: a mask that marks nothing changes no value, so there is nothing to write and the caller's slice is handed on untouched.
func FormatNumber ¶ added in v0.3.0
FormatNumber is how a numeric value is spelled when it is used as a category name. Both faceting and a categorical axis go through it, so a panel key and an axis tick for the same number are the same string.
func GroupBy ¶ added in v0.3.0
GroupBy splits src into groups by the values of a column, returning the distinct values in first-appearance order and the row numbers of each.
The column may be textual, numeric or temporal; whichever it is, the group key is its formatted label, so a facet over a numeric column gets one panel per distinct number rather than a continuous axis. ok is false if src has no such column.
First-appearance order rather than sorted order is deliberate: it is the one ordering that is stable under every column type and lets a caller control panel order by ordering its rows.
func IsNull ¶ added in v1.3.0
IsNull reports whether row i of mask is absent, tolerating a nil or short mask.
A mask is as long as the column it describes, so the length test is not defensive padding: Rows gathers a mask along with the column it belongs to, and a caller composing sources by hand may hand over one that stops early. A row past the end has a value, because that is what the column says.
func Label ¶ added in v1.7.0
Label reads one cell of a column as the text that row says there.
It is Labels for a single row, and it does not build a column to answer. That is the whole reason it exists: Labels spells a numeric or temporal column by allocating a []string as long as the table, which is the right shape for faceting — done once, for every row — and the wrong shape for a pointer, which asks about one row on every move. A hover over a chart of a million rows should not allocate a million strings to name one of them.
The spellings are Labels', so a key, a facet panel key and a categorical tick for the same value are the same string.
ok is false when the source is nil, the column is not there, the column is of no type this understands, or row is outside the table.
func Labels ¶ added in v0.3.0
Labels reads a column as one text label per row, whatever its type.
It is the shared spelling of "what does this row say in that column" — faceting groups by it, and a categorical axis encodes by it, so the two agree about what counts as one category.
func NullMask ¶ added in v1.3.0
NullMask reports which of a column's rows are absent, or ok == false when src cannot say or has nothing to say.
It is the shared spelling of the question, the way Labels is for "what does this row say in that column": every reader asks through this so that one Source implementing Nulls answers every channel at once.
func Origins ¶ added in v0.5.0
Origins returns the mapping from src's rows to the rows of the table it was cut from, or nil if src is not a cut of anything.
One level, not the whole chain: faceting cuts once, and a caller that has composed cuts of cuts knows it has and can compose the mappings the same way.
Types ¶
type Alignment ¶ added in v1.7.0
type Alignment struct {
// Keys are the distinct keys of both tables, in the order above.
Keys []string
// A and B are, for each key, the row carrying it in each table, or -1 for
// a key that table does not have.
A, B []int
}
Alignment is two tables lined up by a key column: one entry per distinct key, saying which row of each table carries it.
It is the join a transition is built on, and it is separate from Tween so that a caller can ask what changed without interpolating anything.
Enter, update and exit ¶
This is D3's data join, and deliberately the same three words, because the vocabulary is the useful part and there is nothing to be gained by inventing a fourth name for it. A key in both tables is an **update** — the same thing, somewhere else. A key only the end state has is an **enter**. A key only the start state has is an **exit**. Alignment.Updated, Alignment.Entered and Alignment.Exited are those three lists.
Where refract differs from D3 is what happens next, and it is worth being clear about because the vocabulary invites the assumption. In D3 the three selections are things you *attach behaviour to*: enter gets its own append and its own transition, exit gets a transition that ends in remove. Here they are three readings of one table. Every key is a row of the blend for the whole transition — see the section below — so an entering row is not something that arrives partway through, it is a row that spends the transition travelling from wherever EnterFrom put it. There is no enter selection to hang a different animation on, and no remove: an exiting row is still drawn at f == 1, sitting at its ExitTo.
That is a smaller vocabulary than D3's on purpose. What it buys is that the frame's structure never changes, which is what keeps an animation off the full-repaint path.
Key order ¶
The key order is first appearance in a, then the keys only b has, in first appearance in b. It is never map iteration order: a chart whose panels are built on separate goroutines has to draw what a serial one drew, so nothing here may depend on how a map felt like enumerating itself. See docs/adr/0012-parallel-panels.md.
func Align ¶ added in v1.7.0
Align lines up two tables by a key column.
A key that appears more than once in a table takes its first row and the rest are ignored. That is a choice rather than an error: a duplicate key is a caller saying two rows are the same thing, and refusing the whole transition over it would be refusing to draw a chart that draws perfectly well. What it costs is that the second row does not move, which is visible and diagnosable, where a refusal at the top of an animation is neither.
func (Alignment) Entered ¶ added in v1.7.0
Entered lists the keys only b has: the rows a transition brings in.
func (Alignment) Exited ¶ added in v1.7.0
Exited lists the keys only a has: the rows a transition takes away.
type Nulls ¶ added in v1.3.0
type Nulls interface {
// Nulls returns one flag per row, true where the column has no value.
//
// ok is false when the column does not exist *or has no nulls at all*.
// That second half is what keeps the zero-copy path in
// [Float64Columns] intact: a reader asks first and copies only when the
// answer is yes, so a table without nulls costs exactly what it did.
// The result is read-only.
Nulls(name string) (null []bool, ok bool)
}
Nulls is implemented by a Source that can say which of a column's values are absent, rather than merely which are unrepresentable.
A numeric column needs no such thing: a missing number is NaN, every scale refuses to place it, and github.com/timzifer/refract/geom.OnMissing is written against exactly that. A *text* column has no NaN — a null read back as "" is indistinguishable from a genuine empty string, and on an ordinal axis it becomes a band of its own — and a *time* column has none either: a null read back as the zero time is the year 1, which is a real instant that stretches a domain over two millennia. Both were silent: the chart drew, and it drew something nobody measured.
This is the optional interface Source's own documentation promises for a capability arriving after the freeze, and it is asked for with a type assertion. A Source that does not implement it has no nulls beyond the NaNs already in its numbers, which is what every Source in this package that predates it means.
What a null means ¶
One rule, everywhere a column is read: **a null is a missing value.** On a position axis the row has no place, so [OnMissing] decides whether the chart gaps, interpolates or errors — the same three answers it already gives a NaN. On a colour channel the row takes the scale's undefined colour. In a group or facet column the row belongs to no series and no panel, so it is not drawn: a category named "" is not a reading, and inventing one is how the zero time got onto an axis in the first place. See [ADR 0034](../docs/adr/0034-null-values.md).
type Source ¶
type Source interface {
// Len reports the number of rows. Every column has this length.
Len() int
// Columns lists the available column names. The order is stable across
// calls on the same Source.
Columns() []string
// Float64Column returns a numeric column by name. ok is false if the
// column does not exist or is not numeric.
Float64Column(name string) (data []float64, ok bool)
// TimeColumn returns a time column by name. ok is false if the column does
// not exist or is not temporal.
TimeColumn(name string) (data []time.Time, ok bool)
// StringColumn returns a categorical column by name. ok is false if the
// column does not exist or is not textual.
StringColumn(name string) (data []string, ok bool)
}
Source exposes columnar, batch access to a table.
Implementations return read-only views: the caller must not mutate a returned slice, and refract never does. An implementation that already holds its data as a Go slice should return that slice directly rather than copying.
Stability ¶
Source is implemented outside this module, so it never gains a method. A fourth column kind — exact integers, booleans, durations — arrives as an optional interface beside it, the way Subset did, and a caller that wants one asks for it with a type assertion and falls back when it is absent.
func Float64Columns ¶
Float64Columns builds a Source over the given numeric columns.
The slices are borrowed, not copied: the returned Source aliases the caller's memory, and mutating it afterwards mutates what refract will plot. All columns must have the same length; Float64Columns panics otherwise, because a ragged table is a programming error rather than a runtime condition.
func Rows ¶ added in v0.3.0
Rows returns a Source over the rows of src named by idx, in the order given.
It is how faceting cuts one table into panels: a facet reads the column it splits on, groups the row numbers, and hands each group to Rows. Out-of-range indices are dropped rather than panicking, because they come from a grouping pass rather than from the caller.
The result materialises the rows it is asked for. That is a copy — the zero-copy promise in Float64Columns is about the whole-column path, and a gathered subset has no contiguous slice to borrow. Columns are gathered lazily, so a table with forty columns and a chart that reads three copies three.
type Stream ¶ added in v0.5.0
type Stream struct {
// contains filtered or unexported fields
}
Stream is a table a producer appends to while a renderer draws.
It is deliberately not a Source. A Source is read column by column, over several calls, and a table being appended to between two of them is a table that disagrees with itself — so the only way to draw a Stream is to freeze it:
st := data.NewStream("t", "y").Window(2000)
p.Add(geom.Line(st.Source(), geom.X("t"), geom.Y("y")))
go func() {
for sample := range samples {
st.Append(scale.Nanos(sample.At), sample.Value)
}
}()
for range ticker.C {
st.Snapshot() // freeze what has arrived
live.Draw() // draw the frozen view
}
Stream.Source hands back a Source that reads the most recent snapshot, so a layer is built once rather than per frame. Stream.Snapshot is what moves it forward.
Two buffers, one copy ¶
Snapshot copies the live rows into a buffer the renderer is not reading and swaps the two. That is one copy per frame — not per row, and not per column read — and the buffers are reused, so a steady stream costs no allocations after the first frame at each size.
The one rule ¶
Append may be called from any goroutine, at any time. Snapshot may not be called while a render is in flight: it is the swap, and swapping the table out from under a half-drawn chart is the race this type exists to remove. Producer appends, renderer snapshots and draws, in that order.
Columns ¶
A Stream carries numeric columns only. A timestamp is its Unix nanoseconds — github.com/timzifer/refract/scale.Nanos converts one, and github.com/timzifer/refract/scale.Time maps that domain — so a time series needs no second column type and no per-row allocation to carry one.
func NewStream ¶ added in v0.5.0
NewStream returns an empty stream over the named numeric columns.
The order of the names is the order Stream.Append takes values in. It panics on no columns or a duplicate name, which are programming errors rather than runtime conditions.
func (*Stream) Append ¶ added in v0.5.0
Append adds one row. The values are positional, in the order NewStream was given.
It is safe to call from any goroutine, and it allocates nothing in the steady state: the row is copied into buffers that are already the right size.
func (*Stream) AppendTime ¶ added in v0.5.0
AppendTime is Stream.Append with the first column given as a timestamp, which is what the first column of a live chart almost always is. The timestamp becomes its Unix nanoseconds, the domain a github.com/timzifer/refract/scale.Time axis maps.
func (*Stream) Len ¶ added in v0.5.0
Len reports how many rows are live. It is the producer's view, which may be ahead of what the last snapshot froze.
func (*Stream) Reset ¶ added in v0.5.0
func (s *Stream) Reset()
Reset empties the stream, keeping its buffers. The snapshot a renderer is holding is untouched until the next Stream.Snapshot.
func (*Stream) Snapshot ¶ added in v0.5.0
Snapshot freezes the live rows and returns a Source over them.
The Source Stream.Source returned reads the same frozen rows, so a layer built once draws whatever the last Snapshot froze.
The returned Source is valid until the next call to Snapshot, which reuses its memory. Call it between frames, never during one.
func (*Stream) Source ¶ added in v0.5.0
Source returns a Source over the stream's most recent snapshot.
It is stable: the same value reads every frame, and what it reads changes only when Stream.Snapshot is called. That is what lets a layer be built once, before the first row has even arrived.
func (*Stream) Window ¶ added in v0.5.0
Window caps the stream at the last n rows, dropping the oldest as new ones arrive. It returns s so the call can be chained onto NewStream.
A window is what makes a stream a *stream* rather than a log: a live chart shows the last few thousand samples, and keeping every sample since the process started is a memory leak with a plot attached. Zero means unbounded, which is the default.
Setting a window smaller than the rows already held drops the oldest of them, because that is what the window means.
type Subset ¶ added in v0.5.0
type Subset interface {
// SourceRows returns, for each of this source's rows, the row it came from
// in the source it selects from. The result is read-only.
SourceRows() []int
}
Subset is implemented by a Source that is a selection of another source's rows, so that a row number can be traced back to the table it came from.
It exists because faceting makes such a source without the caller ever seeing it: github.com/timzifer/refract.Plot.Facet cuts each layer down to its panel's rows with Rows, and a row number relative to that cut is a row number in a table nobody holds. A geom reporting where its rows landed resolves them through this first, so what comes out is a row of the table that was handed in.
It is an optional interface. A Source that is not a selection of another does not implement it, and its rows are already its own.
type Table ¶
type Table struct {
// contains filtered or unexported fields
}
Table is a Source that mixes numeric, temporal and categorical columns.
It is the general-purpose implementation: use it when a chart plots time or a category against values, which is the common case for the Time and Ordinal scales.
func (*Table) Float64 ¶
Float64 adds a numeric column, borrowing the slice. It returns t so calls can be chained. It panics if the column length disagrees with columns already added, or if the name is already taken.
func (*Table) Float64Column ¶
Float64Column returns a numeric column by name.
func (*Table) Nulls ¶ added in v1.3.0
Nulls implements Nulls. ok is false for a column with no nulls, which is every column of a table that was never told about one.
func (*Table) String ¶ added in v0.2.0
String adds a categorical column, borrowing the slice. It returns t so calls can be chained. It panics if the column length disagrees with columns already added, or if the name is already taken.
Plot such a column against a [scale.Ordinal] axis; a continuous scale has no position for a category name and a geom says so rather than guessing one.
func (*Table) StringColumn ¶ added in v0.2.0
StringColumn returns a categorical column by name.
func (*Table) Time ¶
Time adds a temporal column, borrowing the slice. It returns t so calls can be chained. It panics if the column length disagrees with columns already added, or if the name is already taken.
func (*Table) TimeColumn ¶
TimeColumn returns a temporal column by name.
func (*Table) WithNulls ¶ added in v1.3.0
WithNulls marks rows of an existing column as absent, borrowing the mask. It returns t so calls can be chained. It panics if the column does not exist or the mask is not one flag per row.
A text or temporal column needs this because it has no NaN to be missing with: "" is a string somebody may have measured and the zero time is an instant, so absence has to be said beside the values rather than inside them. A numeric column may use it too, and the two spellings agree — a NaN and a marked row are both missing, and neither outranks the other.
A mask that marks nothing is not stored: Nulls answers "no nulls" either way, and a reader that asked would otherwise copy a column to change none of it.
type Tween ¶ added in v1.7.0
type Tween struct {
// contains filtered or unexported fields
}
Tween is a table that reads as a blend of two others.
It is a Source whose *contents* change rather than a Source per frame: the columns are allocated once, at the joined row count, and rewritten in place by Tween.At. Build the layer over Tween.Source once, and a transition then costs no allocations per frame and none per row — the same promise Stream makes, for the same reason and by the same means.
What interpolates and what does not ¶
Numeric and temporal columns interpolate. A time is its Unix nanoseconds, which is the domain scale.Time already maps, so a time axis needs nothing special.
String columns do not. They take the end state's value where the key is in it and the start's otherwise, because a string is a name rather than a quantity: a sankey's from and to are which nodes an edge joins, and half of "ingest" is not a node.
Text ¶
The rule above decides two different-looking things, and only one of them is a limitation.
**A label that is a string snaps.** A [geom.Text] layer over a string column changes from one word to the other at the moment the column does, with nothing in between. There is no cross-fade and no character-level morph: both would need a per-row opacity or a second draw of the same label, and opacity is a property of a layer rather than of a row — the IR change docs/adr/0007-per-mark-colour.md exists to refuse. What *can* move is where the label is, because that comes from its position columns, so a label travelling to a new place while its text changes once is available and is usually what was wanted.
**A label that is a number counts.** A text layer reads its column through Labels, in Train, on every frame — so a numeric column bound to [geom.TextBy] is re-spelled from whatever the blend currently holds, and the label counts from one value to the other by itself. That is the animation people mean by "animated text" nine times in ten, and it needs nothing but a numeric column.
It does need Round. FormatNumber spells a float at full precision, so a third of the way from 0 to 100 reads "33.300000000000004" — arithmetic rather than a number. Round("n", 0) makes it 33.
The same rule reaches further than labels, and this is the part worth knowing before reaching for a transition at all: **anything a geom decides from a string column decides it abruptly.** A bar's slot on a categorical axis, a discrete colour class, a GroupBy membership — those snap at the moment the column changes rather than sliding, because the thing they are deciding from has no halfway. A chart that needs a bar to *slide* between categories has to say where it is going in a numeric column, which is what Hold's doc is about from the other direction.
The row set is the union, and it is fixed ¶
A key in either table is a row of the blend for the whole of it. An entering row exists at f == 0 — sitting wherever EnterFrom put it — and an exiting one still exists at f == 1.
That is not a detail. It is what keeps the *structure* of the frame identical from one frame to the next, which is the condition ir.Damage needs to report that two frames are comparable. A blend whose row count changed mid-flight would make every frame a full repaint, and nothing about the picture would look wrong.
It refuses at construction ¶
A column one table has and the other does not, or one numeric on one side and textual on the other, is an error from NewTween rather than a surprise three frames into an animation. A transition is built before any frame runs, so there is somewhere honest to put the failure.
func NewTween ¶ added in v1.7.0
func NewTween(a, b Source, key string, opts ...TweenOption) (*Tween, error)
NewTween lines up two tables by a key column and returns the blend between them, positioned at f == 0.
func (*Tween) At ¶ added in v1.7.0
At sets the blend fraction, clamped to [0, 1], and rewrites the columns in place. It allocates nothing.
func (*Tween) Rows ¶ added in v1.7.0
Rows reports the blend's row count, which is the number of distinct keys and does not change.
func (*Tween) Source ¶ added in v1.7.0
Source returns the blended table.
It is stable for the life of the Tween: hand it to a layer once, and every Tween.At afterwards changes what that layer draws without the chart being rebuilt. That is the same arrangement Stream.Source has, and it is what makes a transition cost a frame rather than a frame and a rebuild.
type TweenOption ¶ added in v1.7.0
type TweenOption func(*tweenConfig)
TweenOption configures a Tween.
func EnterFrom ¶ added in v1.7.0
func EnterFrom(col string, v float64) TweenOption
EnterFrom is the value a numeric column takes, at f == 0, for a row only the end state has — so a bar grows out of its baseline instead of appearing at full height.
A column with no EnterFrom holds its end value, which is a row that arrives by not moving. That is the default because it needs no configuration and is never wrong; growing from somewhere is the choice, and only the caller knows where "nothing yet" is for their data.
There is no fade, and that is a limit rather than an oversight: opacity is a property of a layer and not of a row, and adding a per-row opacity channel would be a change to the IR that ADR 0007 exists to refuse. A value in data space is the honest substitute.
func ExitTo ¶ added in v1.7.0
func ExitTo(col string, v float64) TweenOption
ExitTo is EnterFrom for a row only the start state has: where it goes on its way out. A column with no ExitTo holds its start value.
func Hold ¶ added in v1.7.0
func Hold(cols ...string) TweenOption
Hold names numeric columns that snap at the halfway point rather than interpolating.
It is for a number that is a name: an identifier, a category encoded as an integer, an axis slot. Halfway between category 2 and category 5 is category 3.5, which is not a category — so a column like that wants the abruptness a string column gets for free.
func Round ¶ added in v1.7.0
func Round(col string, digits int) TweenOption
Round quantises a column to a number of decimal places as it blends, so that what comes out is a number somebody would write down.
It exists for the label that counts up. A text layer reads its column through Labels, which spells a float at full precision — so a value interpolated a third of the way from 0 to 100 is drawn as "33.300000000000004", which is arithmetic rather than a number. Rounding it to zero places makes the label read 33, and the count is the animation anybody wanted from it.
It is not formatting. FormatNumber is deliberately shared by a facet panel key, a categorical tick and a text label, so that one number is spelled one way everywhere; what this changes is the *value*, before anything spells it. That means a column bound to a position as well as to a label will move in steps, which is usually a reason to blend the position from a column of its own.
Negative digits round to tens, hundreds and so on, the way math.Round scaled would: Round("n", -3) counts in thousands.
Source Files