Documentation
¶
Overview ¶
Package stats holds the numerical aggregation behind results: compensated sums, Welford moments with Chan's merge, and the log-bucketed histogram behind quantiles and the exceedance curve.
Index ¶
- func BinsFor(res float64) int
- func LogRange(mean float64) (lo, hi float64)
- func QuantileSorted(sorted []float64, p float64) float64
- type LogHist
- func (h *LogHist) Add(x float64)
- func (h *LogHist) AtOrAbove(x float64) int64
- func (h *LogHist) Bands(edges, dst []float64)
- func (h *LogHist) Bins() int
- func (h *LogHist) Count(k int) int64
- func (h *LogHist) Edge(k int) float64
- func (h *LogHist) Exceedance(x float64) float64
- func (h *LogHist) Merge(o *LogHist)
- func (h *LogHist) Quantile(p float64) (x float64, clipped bool)
- func (h *LogHist) Range() (lo, hi float64)
- func (h *LogHist) Reset()
- type Neumaier
- type Welford
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func BinsFor ¶
BinsFor returns the number of log bins a histogram with these parameters would allocate, for memory planning.
func QuantileSorted ¶
QuantileSorted returns the p-quantile of a sorted sample by the inverse of the empirical CDF (Hyndman–Fan type 1): the smallest value with at least a fraction p of the sample at or below it.
Types ¶
type LogHist ¶
type LogHist struct {
Zero int64
Under int64
Over int64
N int64 // every Add
// contains filtered or unexported fields
}
LogHist is a histogram with bins of fixed relative width over [lo, hi): bin k covers [lo·(1+res)^k, lo·(1+res)^(k+1)). Exact zeros, non-zero values below lo and values at or above hi are counted apart. An integer histogram adds exact buckets for 0 … exact−1 before the log bins and has no lower edge. Counts are integers, so merges are exact.
Example ¶
A log histogram streams values into bins of fixed relative width; its quantiles are exact to that width, and bands read off the curve.
package main
import (
"fmt"
"code.eitanity.com/variate/stats"
)
func main() {
h := stats.NewLogHist(100, 1e-3) // range sized from a mean of 100
for i := 1; i <= 1000; i++ {
h.Add(float64(i))
}
p50, clipped := h.Quantile(0.5)
p99, _ := h.Quantile(0.99)
fmt.Printf("p50 %.0f p99 %.0f clipped %v\n", p50, p99, clipped)
bands := make([]float64, 3)
h.Bands([]float64{0, 250, 500, 1000}, bands)
fmt.Printf("bands %.3f %.3f %.3f\n", bands[0], bands[1], bands[2])
fmt.Printf("P(X >= 900) = %.3f\n", h.Exceedance(900))
}
Output: p50 500 p99 990 clipped false bands 0.249 0.250 0.500 P(X >= 900) = 0.101
func NewIntHist ¶
NewIntHist allocates an integer histogram: exact buckets for 0 … 4095, then log bins up to 1e9·mean.
func NewLogHist ¶
NewLogHist allocates a histogram over [1e-6·mean, 1e9·mean) with relative bin width res. A zero mean gives a histogram with no bins.
func (*LogHist) AtOrAbove ¶
AtOrAbove returns the count of values ≥ x. It is exact when x is 0, an exact bucket, a bin edge, or beyond the range; otherwise the bin holding x is counted whole, so the answer is off by at most that bin.
func (*LogHist) Bands ¶
Bands fills dst[i] with P(edges[i] ≤ X < edges[i+1]) for each consecutive pair; dst must have len(edges)−1 entries.
func (*LogHist) Edge ¶
Edge returns the lower edge of bin k, adjusted so that a value equal to it is counted in bin k and not the one below.
func (*LogHist) Exceedance ¶
Exceedance returns P(X ≥ x).
func (*LogHist) Quantile ¶
Quantile returns a value within one bin of the p-quantile, as the geometric midpoint of the bin the p-quantile falls in. clipped is set when it falls in Under or Over, and the range edge is returned.
type Neumaier ¶
type Neumaier struct {
Sum, C float64
}
Neumaier is a compensated running sum: the low-order part lost by each addition is kept and added back at the end.
type Welford ¶
Welford holds a running count, mean and sum of squared deviations.
Example ¶
Welford keeps a running mean and variance; Neumaier a compensated sum that keeps the small terms a plain sum loses.
package main
import (
"fmt"
"code.eitanity.com/variate/stats"
)
func main() {
var w stats.Welford
var n stats.Neumaier
plain := 0.0
for _, x := range []float64{2, 4, 4, 4, 5, 5, 7, 9} {
w.Add(x)
}
for i := 0; i < 1000; i++ {
n.Add(1e16)
n.Add(1)
n.Add(-1e16)
plain += 1e16
plain += 1
plain += -1e16
}
fmt.Printf("mean %.1f sd %.1f\n", w.Mean, w.StdDev())
fmt.Printf("compensated sum %.0f plain sum %.0f\n", n.Value(), plain)
}
Output: mean 5.0 sd 2.1 compensated sum 1000 plain sum 0
Source Files