Documentation
¶
Overview ¶
Package reqstats turns the nginx access feed into a per-site view of request timing: the typical response time and the routes that run well above their own baseline. It consumes the same syslog datagrams that drive idle-suspend, but a widened log format, and keeps a small rolling in-memory window per route. It is framework-agnostic: nothing here knows a framework name, only universal nginx timing signals.
Index ¶
- Constants
- func IsColdStart(last time.Time, seen bool, now time.Time, gap time.Duration) bool
- func IsStaticAsset(rawURI string) bool
- func NormalizeRoute(method, rawURI string) string
- func SaveSnapshot(snap []SiteStats, path string) error
- func StripQueryFragment(rawURI string) string
- type AccessRecord
- type Aggregator
- type Analytics
- type LatencyBucket
- type Record
- type RouteStat
- type SiteResolver
- type SiteStats
- type StatusCounts
- type Store
- func (s *Store) Close() error
- func (s *Store) Insert(recs []Record) error
- func (s *Store) LastSeenBySite() (map[string]time.Time, error)
- func (s *Store) Prune(before time.Time) (int64, error)
- func (s *Store) Recent(site string, limit int) ([]Record, error)
- func (s *Store) SiteAnalytics(site string, since, until time.Time) (Analytics, error)
- type ThroughputPoint
Constants ¶
const DefaultColdGap = 30 * time.Minute
DefaultColdGap is the fallback idle gap after which a request is treated as a cold start, used when the idle-suspend timeout can't be read. The watcher prefers that configured timeout.
Variables ¶
This section is empty.
Functions ¶
func IsColdStart ¶
IsColdStart reports whether a request at now is a cold start: the site has been seen before and sat idle at least gap since. The first request ever seen for a site isn't a cold start, since there's no prior time to prove it was idle; the watcher seeds the last-seen clock from the durable store on startup so a wake right after a daemon restart still counts against a real prior time.
func IsStaticAsset ¶
IsStaticAsset reports whether a request URI points at a static asset, judged by the file extension of its last path segment, so the timing view can skip it.
func NormalizeRoute ¶
NormalizeRoute turns a method and raw request URI into a stable route key by dropping the query string and collapsing id-like path segments to ":id", so "/users/123" and "/users/456" aggregate as one route. Dropping the query also keeps tokens and other query-string values out of the in-memory window.
func SaveSnapshot ¶
SaveSnapshot persists an already-computed snapshot, so a caller that also needs the snapshot (e.g. to feed slow-route notifications) can build it once and avoid recomputing it inside Save.
func StripQueryFragment ¶
StripQueryFragment returns the path portion of a raw request URI, dropping the query string and fragment, and defaulting to "/". This is the concrete path we keep as a route's openable example, so no query-string values are retained.
Types ¶
type AccessRecord ¶
type AccessRecord struct {
Host string
Status int
RequestTime float64 // seconds, as nginx $request_time
Method string
URI string // raw $request_uri, query string included
}
AccessRecord is one parsed nginx access datagram. The feed emits a pipe-delimited message "$host|$status|$request_time|$request_method|$request_uri" as the final whitespace token, so it survives syslog framing the same way the idle host-only format does.
func ParseAccessRecord ¶
func ParseAccessRecord(datagram []byte) (AccessRecord, bool)
ParseAccessRecord extracts a timing record from one access datagram. It takes the final whitespace token (skipping syslog framing, as the idle parser does) and splits it on '|'. Returns ok=false for the old host-only format, nginx's "-" host placeholder, or a line whose status/time don't parse.
func (AccessRecord) SecondsToMillis ¶
func (r AccessRecord) SecondsToMillis() float64
SecondsToMillis returns the request time in milliseconds.
type Aggregator ¶
type Aggregator struct {
// contains filtered or unexported fields
}
Aggregator holds rolling per-site, per-route timing windows. Safe for concurrent use: the access-feed reader writes while the UI reads.
func New ¶
func New(resolve SiteResolver) *Aggregator
New returns an aggregator that maps hosts to sites via resolve. A nil resolver drops every record.
func (*Aggregator) Record ¶
func (a *Aggregator) Record(r AccessRecord)
Record ingests one access record, resolving its host to a site and appending the request time to that site's overall and per-route windows. Records for an unknown host, or once a site is at its route cap for a new route, are dropped.
func (*Aggregator) Save ¶
func (a *Aggregator) Save(path string) error
Save writes the current snapshot of every site to path as JSON, via a temp file and rename so a reader never sees a half-written file. The watcher calls this on its tick; lerd-ui reads it with Load.
func (*Aggregator) SiteSnapshot ¶
func (a *Aggregator) SiteSnapshot(site string) (SiteStats, bool)
SiteSnapshot returns the current view for one site, ok=false when the site has no recorded traffic.
func (*Aggregator) Snapshot ¶
func (a *Aggregator) Snapshot() []SiteStats
Snapshot returns a view for every site with recorded traffic.
type Analytics ¶
type Analytics struct {
Site string `json:"site"`
Samples int `json:"samples"`
ColdStarts int `json:"cold_starts"` // excluded from timing, still counted
MedianMillis float64 `json:"median_millis"`
P95Millis float64 `json:"p95_millis"`
Status StatusCounts `json:"status"`
Distribution []LatencyBucket `json:"distribution"`
Throughput []ThroughputPoint `json:"throughput"`
Routes []RouteStat `json:"routes"`
}
Analytics is the full request-timing view for one site over a window.
type LatencyBucket ¶
LatencyBucket is one bar of the response-time histogram. UpperMillis is the exclusive upper bound; 0 marks the open-ended top bucket.
type Record ¶
type Record struct {
At time.Time
Site string
Route string
Method string
Status int
Millis float64
URI string
// Cold marks the first request after the site had gone idle: a cold start
// (suspended workers waking, cold caches) whose inflated time would skew the
// timing view, so it's kept out of the percentiles but still counted.
Cold bool
}
Record is one persisted request. Route is the normalized template (e.g. "GET /products/:id"); URI is the concrete path last seen, so a route stays openable and the recent list shows real paths.
func RecordFrom ¶
func RecordFrom(r AccessRecord, site string, at time.Time) Record
RecordFrom builds a store record from a parsed access record for a resolved site, normalizing the route the same way the live aggregator does so the two views agree on route identity.
type RouteStat ¶
type RouteStat struct {
Route string `json:"route"`
Method string `json:"method"`
Example string `json:"example"` // a concrete path last seen for this route, openable in a browser
P50Millis float64 `json:"p50_millis"`
P95Millis float64 `json:"p95_millis"`
// RecentP95Millis is the p95 over only the route's most recent samples, so a
// route that was slow but has since been fixed reads as fast again even while
// its old slow samples still sit in the window's overall p95.
RecentP95Millis float64 `json:"recent_p95_millis"`
Multiplier float64 `json:"multiplier"`
Samples int `json:"samples"`
}
RouteStat is one flagged route in a site snapshot. The representative time is the route's p95, not its median: a route that is only sometimes slow (a mix of fast redirects and slow renders under one path) hides in the median but shows in the tail, which is the "this request takes a second" the user actually feels.
type SiteResolver ¶
SiteResolver maps a request host to the owning site name, returning ok=false for a host that belongs to no registered site. Mirrors the idle resolver so the watcher can pass the same function.
type SiteStats ¶
type SiteStats struct {
Site string `json:"site"`
MedianMillis float64 `json:"median_millis"`
Samples int `json:"samples"`
Slow []RouteStat `json:"slow"`
UpdatedAt time.Time `json:"updated_at"`
}
SiteStats is the per-site view the UI renders: the typical response time and the routes running well above their own baseline, slowest first.
type StatusCounts ¶
type StatusCounts struct {
C2xx int `json:"c2xx"`
C3xx int `json:"c3xx"`
C4xx int `json:"c4xx"`
C5xx int `json:"c5xx"`
}
StatusCounts is the response-status breakdown, used for the error rate.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store is the durable SQLite record of requests. The watcher (the only process on the nginx access feed) writes to it; lerd-ui opens it read-only to build the request-timing analytics view over any window. Pure-Go driver, so the CGO-free build is unaffected.
func OpenStore ¶
OpenStore opens (creating if needed) the SQLite store at path with WAL enabled so the watcher can write while lerd-ui reads across processes.
func (*Store) Insert ¶
Insert writes a batch of records in one transaction, so the watcher can buffer a tick's worth of requests and flush them together rather than per request.
func (*Store) LastSeenBySite ¶
LastSeenBySite returns the most recent request time for every site in the store, so the watcher can seed its cold-start clock on startup. Without it a daemon restart forgets the last request and judges the next wake as warm, letting the cold boot's inflated time dominate the route p95.
func (*Store) Prune ¶
Prune deletes rows older than before, returning how many were removed, so the watcher can bound the store to a retention window.
func (*Store) SiteAnalytics ¶
SiteAnalytics aggregates every request for a site in [since, until) into the full analytics view. Percentiles are computed in Go from the window's rows, reusing the same helpers as the live aggregator, since SQLite has no native percentile and local request volumes are small.
type ThroughputPoint ¶
ThroughputPoint is the request count in one minute bucket, keyed by the bucket's start as unix milliseconds.
Source Files