insights

package
v1.13.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 11, 2026 License: MIT Imports: 27 Imported by: 0

Documentation

Overview

Package insights lets whoever runs a Nimbus app see how it is doing from the inside: memory and the garbage collector, each route's timings, slow and repeated database queries, errors with their stacks, what holds the heap, what the goroutines wait on, and who visits.

One line turns it on, before the routes are registered:

insights.Install(app)

What it collects is read over one endpoint (/_nimbus/insights by default) by a caller that knows the app's secret, given in NIMBUS_INSIGHTS_TOKEN. Without a secret nothing is collected and the endpoint does not exist: an app nobody watches pays nothing.

It is made to be read by a control plane (Nimbus Cloud reads it for its Insights panel and for its assistant), not opened in a browser.

Index

Constants

View Source
const Version = "1"

Version is the shape of the report.

Variables

View Source
var Sections = []string{"overview", "runtime", "heap", "goroutines", "requests", "queries", "errors", "traffic"}

Sections are what the endpoint answers, by ?section=.

Functions

func Shape

func Shape(sql string) string

Shape is a query with its values taken out, so every run of it is the same text: WHERE id = 7 and WHERE id = 9 are both WHERE id = ?.

Types

type AppInfo

type AppInfo struct {
	Version   string    `json:"insights"`
	Go        string    `json:"go"`
	Env       string    `json:"env"`
	StartedAt time.Time `json:"started_at"`
	UptimeSec int64     `json:"uptime_sec"`
	Now       time.Time `json:"now"`
}

AppInfo says which program answered.

type Config

type Config struct {
	// Token is the secret a reader sends as "Authorization: Bearer …".
	// Empty takes NIMBUS_INSIGHTS_TOKEN; with neither, nothing is installed.
	Token string
	// Path is the endpoint. Default /_nimbus/insights.
	Path string
	// SlowQuery and SlowRequest are what counts as slow. Defaults 100ms
	// and 500ms.
	SlowQuery   time.Duration
	SlowRequest time.Duration
	// Sample is how often memory is read. Default 10s. Reading it stops
	// the program for a moment, so it is not done on every request.
	Sample time.Duration
	// History is how many samples are kept. Default 180 (half an hour).
	History int
	// Keep is how many slow requests, slow queries and errors are kept.
	// Default 200 of each.
	Keep int
	// TrafficDir is where each day's visits are written, so they outlive a
	// restart. Default storage/insights. "-" keeps them in memory only.
	TrafficDir string
}

Config is how the collector is set up. The zero value is right for an app on Nimbus Cloud.

type Count

type Count struct {
	Name  string `json:"name"`
	Count int    `json:"count"`
}

Count is a name and how often.

type Day

type Day struct {
	Date      string         `json:"date"` // 2026-10-09
	Views     int            `json:"views"`
	Visitors  int            `json:"visitors"`
	Hours     [24]int        `json:"hours"`
	Paths     map[string]int `json:"paths"`
	Referrers map[string]int `json:"referrers"`
	Countries map[string]int `json:"countries"`
	Devices   map[string]int `json:"devices"`
	Browsers  map[string]int `json:"browsers"`
	// Seen is the day's visitors, as the numbers they are told apart by.
	Seen []uint64 `json:"seen,omitempty"`
	// contains filtered or unexported fields
}

Day is one day's visits (UTC).

type DayCount

type DayCount struct {
	Date     string `json:"date"`
	Views    int    `json:"views"`
	Visitors int    `json:"visitors"`
}

DayCount is a day's totals.

type ErrorEntry

type ErrorEntry struct {
	At      time.Time `json:"at"`
	Method  string    `json:"method"`
	Path    string    `json:"path"`
	Route   string    `json:"route"`
	Status  int       `json:"status"`
	Kind    string    `json:"kind"` // error | panic | status
	Message string    `json:"message"`
	Stack   string    `json:"stack,omitempty"`
}

ErrorEntry is an error a handler returned, a panic, or a 5xx answer.

type Frame

type Frame struct {
	Func string `json:"func"`
	File string `json:"file"`
	Line int    `json:"line"`
}

Frame is a place in the program.

type GoroutineGroup

type GoroutineGroup struct {
	Count int     `json:"count"`
	At    Frame   `json:"at"`
	Stack []Frame `json:"stack"`
}

GoroutineGroup is goroutines that are in the same place.

type GoroutineReport

type GoroutineReport struct {
	Total  int              `json:"total"`
	Groups []GoroutineGroup `json:"groups"`
}

GoroutineReport is the goroutines by where they are: a group that keeps growing is a leak.

type HeapReport

type HeapReport struct {
	TotalInuseBytes int64      `json:"total_inuse_bytes"`
	Sites           []HeapSite `json:"sites"`
}

HeapReport is the heap by who allocated it. The runtime samples allocations (one in about every 512 KB), so small holders are estimates.

type HeapSite

type HeapSite struct {
	InuseBytes   int64   `json:"inuse_bytes"`
	InuseObjects int64   `json:"inuse_objects"`
	AllocBytes   int64   `json:"alloc_bytes"` // since the program started
	At           Frame   `json:"at"`          // the first frame that is not the runtime's
	Stack        []Frame `json:"stack"`       // a few frames up from there
}

HeapSite is memory held by what one place in the program allocated.

type Insights

type Insights struct {
	// contains filtered or unexported fields
}

Insights is a running collector.

func Install

func Install(app *nimbus.App, cfg ...Config) *Insights

Install starts collecting and mounts the endpoint. Call it once, before the app's own middleware and routes are registered, so every request passes through it. It answers nil when there is no secret.

func New

func New(cfg Config) *Insights

New starts a collector without mounting anything: for an app that wires the middleware and the handler itself. Nil when there is no secret.

func (*Insights) Close

func (in *Insights) Close()

Close stops the sampler and writes what the traffic log still holds.

func (*Insights) Collect

func (in *Insights) Collect(sections ...string) *Report

Collect gathers the sections named (all of them for "all" or none).

func (*Insights) Handler

func (in *Insights) Handler() func(c *nhttp.Context) error

Handler answers the endpoint: a Report as JSON, to a caller with the secret.

func (*Insights) Middleware

func (in *Insights) Middleware() router.Middleware

Middleware measures every request that passes through it.

type Overview

type Overview struct {
	RequestsPerMin float64 `json:"requests_per_min"`
	ErrorRate      float64 `json:"error_rate"` // share of requests answered 5xx, 0 to 1
	P50MS          float64 `json:"p50_ms"`
	P95MS          float64 `json:"p95_ms"`
	P99MS          float64 `json:"p99_ms"`
	HeapInuse      uint64  `json:"heap_inuse"`
	Sys            uint64  `json:"sys"`
	Goroutines     int     `json:"goroutines"`
	GCCPUFraction  float64 `json:"gc_cpu_fraction"`
	SlowQueries    int     `json:"slow_queries"`
	Errors         int     `json:"errors"`
	InFlight       int     `json:"in_flight"`
}

Overview is the few numbers that say how the app is doing.

type PoolStat

type PoolStat struct {
	Open         int     `json:"open"`
	InUse        int     `json:"in_use"`
	Idle         int     `json:"idle"`
	MaxOpen      int     `json:"max_open"`
	WaitCount    int64   `json:"wait_count"` // times a request had to wait for a connection
	WaitMS       float64 `json:"wait_ms"`
	MaxIdleClose int64   `json:"max_idle_closed"`
}

PoolStat is the database's connections.

type QueryReport

type QueryReport struct {
	Total  int         `json:"total"`
	Top    []QueryStat `json:"top"`    // most time spent first
	Slow   []SlowQuery `json:"slow"`   // newest first
	Failed []SlowQuery `json:"failed"` // newest first
	Pool   *PoolStat   `json:"pool,omitempty"`
	Driver string      `json:"driver"`
	SlowMS float64     `json:"slow_ms"`
}

QueryReport is the database as the app uses it.

type QueryStat

type QueryStat struct {
	Shape   string    `json:"shape"` // the query with its values taken out
	Count   int       `json:"count"`
	Errors  int       `json:"errors"`
	AvgMS   float64   `json:"avg_ms"`
	MaxMS   float64   `json:"max_ms"`
	TotalMS float64   `json:"total_ms"`
	Rows    int64     `json:"rows"`   // rows it gave or changed, in all
	Caller  string    `json:"caller"` // where in the app it was last run from
	LastAt  time.Time `json:"last_at"`
}

QueryStat is every run of one query, whatever values it was run with.

type Repeated

type Repeated struct {
	Route  string    `json:"route"`
	Shape  string    `json:"shape"`  // the query with its values taken out
	Times  int       `json:"times"`  // the most it ran in one request
	Seen   int       `json:"seen"`   // in how many requests
	Caller string    `json:"caller"` // where in the app it is run from
	LastAt time.Time `json:"last_at"`
}

Repeated is one query run many times while answering one request: the mark of a loop that asks the database once per row (N+1).

type Report

type Report struct {
	App        AppInfo          `json:"app"`
	Overview   *Overview        `json:"overview,omitempty"`
	Runtime    *RuntimeReport   `json:"runtime,omitempty"`
	Heap       *HeapReport      `json:"heap,omitempty"`
	Goroutines *GoroutineReport `json:"goroutines,omitempty"`
	Requests   *RequestReport   `json:"requests,omitempty"`
	Queries    *QueryReport     `json:"queries,omitempty"`
	Errors     []ErrorEntry     `json:"errors,omitempty"`
	Traffic    *TrafficReport   `json:"traffic,omitempty"`
}

Report is the endpoint's answer. Only the sections asked for are set.

type RequestReport

type RequestReport struct {
	Total    int            `json:"total"`
	Routes   []RouteStat    `json:"routes"` // most time spent first
	Slow     []SlowRequest  `json:"slow"`   // newest first
	Status   map[string]int `json:"status"` // "2xx", "3xx", "4xx", "5xx"
	Repeated []Repeated     `json:"repeated"`
	SlowMS   float64        `json:"slow_ms"` // what counts as slow
}

RequestReport is the routes, the slow requests and the repeated queries.

type RouteStat

type RouteStat struct {
	Route   string  `json:"route"` // "GET /posts/:id"
	Count   int     `json:"count"`
	Errors  int     `json:"errors"` // answered 5xx
	AvgMS   float64 `json:"avg_ms"`
	P95MS   float64 `json:"p95_ms"`
	MaxMS   float64 `json:"max_ms"`
	TotalMS float64 `json:"total_ms"`
	// what its requests asked of the database, on average
	Queries float64   `json:"queries"`
	QueryMS float64   `json:"query_ms"`
	LastAt  time.Time `json:"last_at"`
}

RouteStat is how one route has been answering.

type RuntimeReport

type RuntimeReport struct {
	Now          Sample   `json:"now"`
	History      []Sample `json:"history"` // oldest first
	RecentPauses []uint64 `json:"recent_pauses_ns"`
	MaxPauseNs   uint64   `json:"max_pause_ns"`
	GOMAXPROCS   int      `json:"gomaxprocs"`
	NumCPU       int      `json:"num_cpu"`
	GOGC         string   `json:"gogc"`
	MemoryLimit  int64    `json:"memory_limit"` // bytes; 0 when none is set
	// ContainerLimit is the memory the container may use (its cgroup), 0 when unknown.
	ContainerLimit uint64 `json:"container_limit"`
}

RuntimeReport is where memory and the collector stand, and how they got there.

type Sample

type Sample struct {
	At            time.Time `json:"at"`
	HeapAlloc     uint64    `json:"heap_alloc"`
	HeapInuse     uint64    `json:"heap_inuse"`
	HeapIdle      uint64    `json:"heap_idle"`
	HeapReleased  uint64    `json:"heap_released"`
	HeapSys       uint64    `json:"heap_sys"`
	HeapObjects   uint64    `json:"heap_objects"`
	StackInuse    uint64    `json:"stack_inuse"`
	Sys           uint64    `json:"sys"`
	NextGC        uint64    `json:"next_gc"`
	NumGC         uint32    `json:"num_gc"`
	PauseTotalNs  uint64    `json:"pause_total_ns"`
	GCCPUFraction float64   `json:"gc_cpu_fraction"`
	Mallocs       uint64    `json:"mallocs"`
	Frees         uint64    `json:"frees"`
	TotalAlloc    uint64    `json:"total_alloc"`
	Goroutines    int       `json:"goroutines"`
	// since the sample before: collections, time paused in them, bytes
	// allocated, requests answered and their slowest
	GCs       uint32  `json:"gcs"`
	PauseNs   uint64  `json:"pause_ns"`
	AllocRate uint64  `json:"alloc_rate"` // bytes a second
	Requests  int     `json:"requests"`
	Errors    int     `json:"errors"`
	P95MS     float64 `json:"p95_ms"`
}

Sample is memory and the collector at one moment.

type SlowQuery

type SlowQuery struct {
	At     time.Time `json:"at"`
	SQL    string    `json:"sql"` // as it was run, values and all
	Shape  string    `json:"shape"`
	MS     float64   `json:"ms"`
	Rows   int64     `json:"rows"`
	Caller string    `json:"caller"`
	Error  string    `json:"error,omitempty"`
}

SlowQuery is one run of a query that took long, or failed.

type SlowRequest

type SlowRequest struct {
	At      time.Time `json:"at"`
	Method  string    `json:"method"`
	Path    string    `json:"path"`
	Route   string    `json:"route"`
	Status  int       `json:"status"`
	MS      float64   `json:"ms"`
	Queries int       `json:"queries"`
	QueryMS float64   `json:"query_ms"`
}

SlowRequest is one request that took long.

type TrafficReport

type TrafficReport struct {
	Days      []DayCount `json:"days"` // oldest first, a day with no visit included
	Views     int        `json:"views"`
	Visitors  int        `json:"visitors"` // the days' visitors added up
	Today     DayCount   `json:"today"`
	Hours     [24]int    `json:"hours"` // today, by hour (UTC)
	Paths     []Count    `json:"paths"`
	Referrers []Count    `json:"referrers"`
	Countries []Count    `json:"countries"`
	Devices   []Count    `json:"devices"`
	Browsers  []Count    `json:"browsers"`
	Kept      bool       `json:"kept"` // written to disk, so it outlives a restart
}

TrafficReport is the visits of the last days.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL