README
¶
type: reference audience: analytics consumer status: draft
Status: draft — pre-human-review. Not yet verified against the current documentation standard. Do not cite as authoritative.
analytics — Streaming Analytics and Similarity Measurement
Package tree
analytics/
stats/ Streaming statistics (Welford/Kahan) + convergence detection
similarity/
compression/ Compression-based similarity (NCD, CCC) — compressor-agnostic
stylometry/ Authorship analysis with convergence-aware streaming on top of compression/
stats
Online (single-pass) streaming statistics using Compensated Welford's Algorithm with Kahan summation. Mean, Variance, StdDev, Skewness, Kurtosis, Min, Max. Merge via Pébay's parallel formulas. Convergence detection via sliding-window variance stability.
similarity/compression
Compression-based similarity measurement: NCD (Normalized Compression Distance) and CCC (Conditional Complexity of Compression). Compressor-agnostic (gzip, zstd, bzip2, ...) with a zstd raw-dictionary optimization.
similarity/compression/stylometry
Authorship analysis built on compression. Streaming NCD/CCC with automatic convergence detection (Instance mode) and equal-length profile comparison (Profile mode).
References
- Li, Chen, Li, Ma, Vitanyi. The Similarity Metric. IEEE Trans. Inf. Theory, 2004.
- Cilibrasi, Vitanyi. Clustering by Compression. IEEE Trans. Inf. Theory, 2005.
- Pébay. Formulas for Robust, One-Pass Parallel Computation of Covariances and Arbitrary-Order Statistical Moments. Sandia National Laboratories, 2008.
- Halford. Text classification using compression, 2023.
Directories
¶
| Path | Synopsis |
|---|---|
|
graph
|
|
|
algo
Package algo holds the whole-graph metrics of the graph analytics engine (ADR-0229 §SD3): degrees, breadth-first distances, connected and strongly connected components, PageRank, k-core decomposition, triangle counting, betweenness centrality and maximal cliques — and, over a distance-weighted graph, HDBSCAN (ADR-0230 §SD3).
|
Package algo holds the whole-graph metrics of the graph analytics engine (ADR-0229 §SD3): degrees, breadth-first distances, connected and strongly connected components, PageRank, k-core decomposition, triangle counting, betweenness centrality and maximal cliques — and, over a distance-weighted graph, HDBSCAN (ADR-0230 §SD3). |
|
csr
Package csr is the one adjacency container of the graph analytics engine (ADR-0229 §SD1): vertex ids mapped to dense slots, forward and reverse adjacency as offset and target arrays, neighbours sorted by id within each row, built once per topology by counting sort.
|
Package csr is the one adjacency container of the graph analytics engine (ADR-0229 §SD1): vertex ids mapped to dense slots, forward and reverse adjacency as offset and target arrays, neighbours sorted by id within each row, built once per topology by counting sort. |
|
engine
Package engine is the iteration layer of the graph analytics engine (ADR-0229 §SD2): a frontier (Subset) kept sparse or dense, an edge map whose direction is chosen per sweep, and chunked parallelism whose result is bit-identical to the serial one.
|
Package engine is the iteration layer of the graph analytics engine (ADR-0229 §SD2): a frontier (Subset) kept sparse or dense, an edge map whose direction is chosen per sweep, and chunked parallelism whose result is bit-identical to the serial one. |
|
knn
Package knn is the neighbour-graph producer of the graph analytics engine (ADR-0230 §SD1): a feature matrix in, a weighted undirected github.com/stergiotis/boxer/public/analytics/graph/csr.Graph out.
|
Package knn is the neighbour-graph producer of the graph analytics engine (ADR-0230 §SD1): a feature matrix in, a weighted undirected github.com/stergiotis/boxer/public/analytics/graph/csr.Graph out. |
|
similarity
|
|
|
distsql
Package distsql implements the ADR-0161 distribution result contract: the fixed probability grid the descriptiveStatistics macro emits, the claim validation rules the play Distribution panel applies, and the letterval.QuantileOracle adapter that feeds a boxenplot ladder from a server-side quantile grid.
|
Package distsql implements the ADR-0161 distribution result contract: the fixed probability grid the descriptiveStatistics macro emits, the claim validation rules the play Distribution panel applies, and the letterval.QuantileOracle adapter that feeds a boxenplot ladder from a server-side quantile grid. |
|
ecdfbands
Package ecdfbands implements finite-sample exact simultaneous confidence bands on the empirical cumulative distribution function of an iid univariate sample.
|
Package ecdfbands implements finite-sample exact simultaneous confidence bands on the empirical cumulative distribution function of an iid univariate sample. |
|
ecdfbands/digest
Package digest bridges a tdigest.TDigest to the ecdfbands library.
|
Package digest bridges a tdigest.TDigest to the ecdfbands library. |
|
letterval
Package letterval computes letter-value summaries (Hofmann, Wickham & Kafadar 2017) from any source that can answer Quantile/CDF queries.
|
Package letterval computes letter-value summaries (Hofmann, Wickham & Kafadar 2017) from any source that can answer Quantile/CDF queries. |
|
tdigest
Package tdigest provides a streaming quantile sketch with tail-biased accuracy, based on Dunning 2019 (arXiv:1902.04023).
|
Package tdigest provides a streaming quantile sketch with tail-biased accuracy, based on Dunning 2019 (arXiv:1902.04023). |
|
timeseries
|
|
|
adscore
Package adscore scores time-series anomaly detectors against labelled data, and generates labelled fixtures that are worth scoring against.
|
Package adscore scores time-series anomaly detectors against labelled data, and generates labelled fixtures that are worth scoring against. |
|
damp
Package damp finds left discords over an arriving stream: subsequences whose nearest neighbour among everything that came *before* them is far away.
|
Package damp finds left discords over an arriving stream: subsequences whose nearest neighbour among everything that came *before* them is far away. |
|
loadstudy
Package loadstudy pulls real recorded load metrics and application events out of ClickHouse so the ADR-0150 detectors can be measured against something nobody synthesised.
|
Package loadstudy pulls real recorded load metrics and application events out of ClickHouse so the ADR-0150 detectors can be measured against something nobody synthesised. |
|
matrixprofile
Package matrixprofile computes the matrix profile of a real-valued time series: for every subsequence of a fixed length, the z-normalized Euclidean distance to its nearest non-trivial neighbour, and that neighbour's index.
|
Package matrixprofile computes the matrix profile of a real-valued time series: for every subsequence of a fixed length, the z-normalized Euclidean distance to its nearest non-trivial neighbour, and that neighbour's index. |
|
mssmooth
Package mssmooth smooths equidistant series by convolution with a modified-sinc (MS) kernel — the Savitzky–Golay replacement of Schmid, Rath and Diebold, ACS Meas.
|
Package mssmooth smooths equidistant series by convolution with a modified-sinc (MS) kernel — the Savitzky–Golay replacement of Schmid, Rath and Diebold, ACS Meas. |
Click to show internal directories.
Click to hide internal directories.