analytics

package
v1.6.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 23, 2026 License: AGPL-3.0 Imports: 7 Imported by: 0

Documentation

Overview

Package analytics turns Goma Gateway's per-request event stream into minute-bucketed rollups (models.AnalyticsRollup) and answers the Traffic, Performance and Web Analytics queries over them. It holds no per-request rows and no PII: latency is a histogram, uniques are a HyperLogLog sketch, and the visitor id is a daily-salted hash produced at the edge (never an IP).

Index

Constants

This section is empty.

Variables

View Source 
var LatencyBoundsMs = []int64{5, 10, 25, 50, 100, 250, 500, 1000, 2500, 5000}

LatencyBoundsMs are the histogram bucket upper bounds (milliseconds). A value is counted in the first bucket whose bound it does not exceed; anything above the last bound lands in a trailing overflow bucket (index len(bounds)).

Functions

func Merge 

func Merge(dst, src *models.AnalyticsRollup)

Merge adds src into dst: counters, element-wise histograms, sums, top-K maps, and the HLL sketch. Used by the repository to combine a flushed bucket with an existing row (multi-consumer / re-flush safe).

func MergeUniques 

func MergeUniques(sketches [][]byte) int64

MergeUniques merges several serialized sketches and returns the estimate — the unique-visitor count over a range is the merge of its buckets' sketches.

func Percentile 

func Percentile(hist []int64, p float64) float64

Percentile estimates the p-th percentile (0..1) latency in ms from a histogram, returning each bucket's upper bound (the overflow bucket returns the last bound as a floor). Approximate by design — bucketed, cheap, and mergeable.

func UniquesOf 

func UniquesOf(data []byte) int64

UniquesOf returns the estimated unique visitors from a serialized sketch.

func WorkspaceIDFromRoute 

func WorkspaceIDFromRoute(name string) uint

WorkspaceIDFromRoute parses the workspace id out of a Goma route name of the form "mb-ws<id>-<slug>". Returns 0 when the name isn't in that form (a route Miabi doesn't own — e.g. the platform gateway's own route).

Types

type Aggregator 

type Aggregator struct {
	// contains filtered or unexported fields
}

Aggregator accumulates events into per-(workspace, app, route, minute) buckets in memory, and hands over "closed" buckets (older than a grace window) as models.AnalyticsRollup for persistence. Safe for concurrent Ingest/Flush.

func NewAggregator 

func NewAggregator() *Aggregator

func (*Aggregator) Flush 

func (a *Aggregator) Flush(before time.Time) []*models.AnalyticsRollup

Flush removes and returns every bucket strictly older than `before`, serializing its HLL sketch into VisitorsHLL. Call with now-grace so an in-progress minute keeps accumulating.

func (*Aggregator) Ingest 

func (a *Aggregator) Ingest(e *Event, ws, app uint)

Ingest folds one event into its bucket. ws/app are resolved by the caller (workspace from the route name, app via a route lookup); an unresolved app is 0.

func (*Aggregator) Pending 

func (a *Aggregator) Pending() int

Pending reports how many open buckets are held (for metrics/tests).

type Category 

type Category struct {
	Label string `json:"label"`
	Count int64  `json:"count"`
}

Category is one labelled count in a breakdown.

type Event 

type Event struct {
	Ts           int64  `json:"ts"` // unix millis
	Gateway      string `json:"gw"`
	Route        string `json:"name"` // "mb-ws<workspaceID>-<slug>"
	Host         string `json:"host"`
	Method       string `json:"method"`
	Status       int    `json:"status"`
	Path         string `json:"path"`
	PathTemplate string `json:"path_template"`
	ReqBytes     int64  `json:"req_bytes"`
	RespBytes    int64  `json:"resp_bytes"`
	DurationMs   int64  `json:"duration_ms"`
	UpstreamMs   int64  `json:"upstream_ms"`
	VID          string `json:"vid"`
	Country      string `json:"country"`
	UA           string `json:"ua"`
	RefererHost  string `json:"referer_host"`
}

Event is one request as emitted by Goma onto the analytics stream. Field tags mirror the gateway's AnalyticsEvent JSON.

type Performance 

type Performance struct {
	RequestP50  float64 `json:"request_p50_ms"`
	RequestP95  float64 `json:"request_p95_ms"`
	RequestP99  float64 `json:"request_p99_ms"`
	UpstreamP50 float64 `json:"upstream_p50_ms"`
	UpstreamP95 float64 `json:"upstream_p95_ms"`
	UpstreamP99 float64 `json:"upstream_p99_ms"`
	// Mean total, upstream (backend) and gateway overhead (total − upstream) in ms.
	AvgRequestMs  float64     `json:"avg_request_ms"`
	AvgUpstreamMs float64     `json:"avg_upstream_ms"`
	AvgOverheadMs float64     `json:"avg_overhead_ms"`
	SlowRoutes    []RouteStat `json:"slow_routes"`
}

Performance covers the latency pillar: request vs upstream percentiles plus the slowest routes by p95.

type Report 

type Report struct {
	Range       Window        `json:"range"`
	Granularity string        `json:"granularity"` // "minute" | "hour" | "day"
	Totals      Totals        `json:"totals"`
	Series      []SeriesPoint `json:"series"`
	Status      StatusBreak   `json:"status"`
	Performance Performance   `json:"performance"`
	Web         WebAnalytics  `json:"web"`
	// Compare holds the totals of the immediately preceding, equal-length window,
	// so the UI can render period-over-period deltas. Nil when not requested.
	Compare *Totals `json:"compare,omitempty"`
	// RetentionDays is the effective retention cap in days (-1 = unlimited), and
	// Exportable reports whether analytics export is entitled. Both are set by the
	// handler from the license so the UI can bound the range picker and show the
	// Enterprise upgrade hints. They are edition metadata, not computed from rows.
	RetentionDays int  `json:"retention_days"`
	Exportable    bool `json:"exportable"`
}

Report is the combined analytics answer for a workspace (optionally one app) over a time range: the Traffic, Performance and Web Analytics pillars share one pass over the same rollups so the three dashboard tabs come from a single query.

func BuildReport 

func BuildReport(rows []models.AnalyticsRollup, since, until time.Time) Report

BuildReport reduces the raw minute rollups of a range into the combined dashboard report. granularity is auto-selected from the span. It performs one pass for the totals/status/web breakdowns and a grouped pass for the series, merging latency histograms for percentiles and HLL sketches for uniques.

type RouteStat 

type RouteStat struct {
	Route     string  `json:"route"`
	Requests  int64   `json:"requests"`
	P95       float64 `json:"p95_latency_ms"`
	ErrorRate float64 `json:"error_rate"`
}

RouteStat is a per-route performance summary (used for the slowest-routes list).

type SeriesPoint 

type SeriesPoint struct {
	T          time.Time `json:"t"`
	Requests   int64     `json:"requests"`
	BytesIn    int64     `json:"bytes_in"`
	BytesOut   int64     `json:"bytes_out"`
	Errors     int64     `json:"errors"`     // 4xx+5xx (kept for compatibility)
	Errors4xx  int64     `json:"errors_4xx"` // client errors
	Errors5xx  int64     `json:"errors_5xx"` // server errors
	Uniques    int64     `json:"unique_visitors"`
	AvgLatency float64   `json:"avg_latency_ms"`
	P95Latency float64   `json:"p95_latency_ms"`
}

SeriesPoint is one time bucket of the request/latency time series at the chosen granularity.

type StatusBreak 

type StatusBreak struct {
	S2xx int64 `json:"s2xx"`
	S3xx int64 `json:"s3xx"`
	S4xx int64 `json:"s4xx"`
	S5xx int64 `json:"s5xx"`
}

type Totals 

type Totals struct {
	Requests    int64   `json:"requests"`
	BytesIn     int64   `json:"bytes_in"`
	BytesOut    int64   `json:"bytes_out"`
	UniqueVisit int64   `json:"unique_visitors"`
	ErrorRate   float64 `json:"error_rate"` // (4xx+5xx)/requests, 0..1
	AvgLatency  float64 `json:"avg_latency_ms"`
	P50Latency  float64 `json:"p50_latency_ms"`
	P95Latency  float64 `json:"p95_latency_ms"`
	P99Latency  float64 `json:"p99_latency_ms"`
}

Totals are the headline traffic numbers over the whole range.

type WebAnalytics 

type WebAnalytics struct {
	UniqueVisitors int64      `json:"unique_visitors"`
	BotRequests    int64      `json:"bot_requests"`
	HumanRequests  int64      `json:"human_requests"`
	TopPaths       []Category `json:"top_paths"`
	TopReferrers   []Category `json:"top_referrers"`
	TopCountries   []Category `json:"top_countries"`
	TopBrowsers    []Category `json:"top_browsers"`
	TopOS          []Category `json:"top_os"`
	TopDevices     []Category `json:"top_devices"`
	TopMethods     []Category `json:"top_methods"`
}

WebAnalytics is the cookieless web pillar: uniques plus the categorical breakdowns (paths, referrers, countries, browsers, methods).

type Window 

type Window struct {
	Since time.Time `json:"since"`
	Until time.Time `json:"until"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.