analytics

package
v0.19.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 3, 2026 License: AGPL-3.0 Imports: 39 Imported by: 0

Documentation

Overview

Package analytics is GoRun's built-in, cookieless site analytics: a page view and form-submit beacon, stored in its own analytics.db beside the main database, with daily rollups and an admin dashboard. Visitors are counted by a daily-salted hash; the raw IP, the user agent and query strings are never stored (docs/superpowers/specs/2026-11-03-site-analytics-design.md).

Index

Constants

View Source 
const (
	BufferCap  = 10000           // events held in memory before Add drops
	FlushEvery = 2 * time.Second // Run's flush interval
	FlushBatch = 500             // rows per Insert call (and per transaction)

)

Write buffer defaults (plan, Global Constraints).

View Source 
const (
	ScriptPath = "/_a/s.js"
	BeaconPath = "/_a/e"

	// SettingEnabled turns tracking on or off (JSON bool, default true).
	SettingEnabled = "analytics.enabled"
	// SettingExclude lists excluded path prefixes, one per line (JSON string).
	SettingExclude = "analytics.exclude"
)

Collection endpoints and settings.

View Source 
const (
	ModuleName = "analytics"
	PermView   = "analytics.view"
	CronRollup = "analytics.rollup"
)

Module name, permission and cron.

View Source 
const (
	DimTotal   = "total"
	DimPage    = "page"
	DimRef     = "ref"
	DimUTM     = "utm"
	DimBrowser = "browser"
	DimOS      = "os"
	DimDevice  = "device"
	DimCountry = "country"
	DimForm    = "form"
)

daily dims (spec §3). Each row is one (day, dim, key):

  • total: key ""; views, visitors and submits of the day.
  • page: key is the path; views and visitors of that page's views, and the submits made on that page.
  • ref, browser, os, device, country: the column's value; views only (a submit carries none of these), submits 0.
  • utm: key "source/medium/campaign" (each part may be empty, e.g. "//spring" for a campaign without source and medium); views carrying no utm value at all have no utm row. Top(dim="utm") uses the same key.
  • form: key formKey(form, page); submits only, views and visitors 0 (Forms reads the page's views from the page dim).

visitors is COUNT(DISTINCT visitor) over the day's views with a non-empty visitor. Summing it across days double counts a returning visitor, so a range's visitors are always counted on events (query.go).

View Source 
const (
	KindView   = "view"
	KindSubmit = "submit"
)

Event kinds (events.kind).

View Source 
const DBFile = "analytics.db"

DBFile is the analytics database file name inside the data directory. internal/backup finds the optional backup member by this name (ruling A2).

View Source 
const GeoFile = "GeoLite2-Country.mmdb"

GeoFile is the optional MaxMind country database the operator drops into the data dir. Without it every country is unknownCountry.

View Source 
const InsertBatch = 500

InsertBatch is the largest number of rows Insert writes in one transaction.

View Source 
const (
	// SettingBackupExclude leaves analytics.db out of backups (JSON bool,
	// default false). internal/backup's callers read it by this name.
	SettingBackupExclude = "analytics.backup_exclude"
)

Variables

View Source 
var ErrClosed = errors.New("analytics: store closed")

ErrClosed is returned by Store methods called after Close.

View Source 
var ErrDim = errors.New("analytics: unknown dim")

ErrDim is returned by Top for a dim it does not know.

View Source 
var ErrRange = errors.New("analytics: empty or invalid range")

ErrRange is returned for a Filter whose From is not before its To.

Functions

This section is empty.

Types

type Buffer 

type Buffer struct {
	// contains filtered or unexported fields
}

Buffer queues events in memory and writes them to the store in batches, so a burst of page views costs one transaction per batch instead of one per request. When it holds cap events, Add drops new ones. It is safe for concurrent use.

func NewBuffer 

func NewBuffer(s *Store, cap int, every time.Duration, batch int, log *slog.Logger) *Buffer

NewBuffer returns a buffer of capacity cap that Run flushes every every, batch events per Insert (at most InsertBatch). Non-positive values take the defaults above; log may be nil (slog.Default).

func (*Buffer) Add 

func (b *Buffer) Add(e Event) bool

Add queues e and reports whether it was kept. A zero TS is set to the time of the call, so an event keeps the day it happened on however late it is flushed. An event whose Kind is neither KindView nor KindSubmit is refused (the events CHECK would fail its whole batch), as is every event once Run has stopped. When the buffer is full the event is dropped and a warning is logged at most once per minute, with the number dropped since the last one.

func (*Buffer) Flush 

func (b *Buffer) Flush(ctx context.Context) error

Flush writes every queued event, batch events per Insert. Events of a batch that fails go back to the front of the queue as far as the capacity allows, and the error is returned; after maxFlushFailures failed flushes in a row they are dropped instead, with an error log.

func (*Buffer) Len 

func (b *Buffer) Len() int

Len returns the number of queued events.

func (*Buffer) Run 

func (b *Buffer) Run(ctx context.Context)

Run flushes every interval until ctx is cancelled, then flushes what is left (with a fresh deadline, since ctx is already done) and returns.

type Event 

type Event struct {
	TS                                time.Time
	Kind, Path, Visitor, RefHost      string
	UTMSource, UTMMedium, UTMCampaign string
	Browser, OS, Device, Country      string
	Locale, Form                      string
}

Event is one stored row of events. TS is stored in UTC at second precision (db.FormatTime); the other fields are stored as given, so the caller strips query strings, caps the path and hashes the visitor before the event reaches the buffer.

type Filter 

type Filter struct {
	From, To                   time.Time
	Page, Ref, Country, Device string
}

Filter selects the events a query counts: those with From <= ts < To (UTC; day boundaries are UTC midnights), narrowed by the non-empty fields. Page is a path, Ref a referrer host ("direct" for none), Country an ISO code ("--" unknown), Device mobile, tablet, desktop or unknown.

Submit events carry only a page, so with Ref, Country or Device set they all drop out: submissions (Summary.Submits, Forms) are 0 under those filters.

type FormRow 

type FormRow struct {
	Form, Page     string
	Views, Submits int
	Rate           float64
}

FormRow is one form on one page: the page's views, the form's submissions there and Submits/Views (0 without views).

type Module 

type Module struct {
	// contains filtered or unexported fields
}

Module is the analytics kernel.Module.

func New 

func New() *Module

New returns the analytics module.

func (*Module) Catalogs 

func (*Module) Catalogs() fs.FS

Catalogs implements kernel.CatalogProvider (the analytics.* strings).

func (*Module) Enabled 

func (*Module) Enabled(ctx context.Context, d *sql.DB) (bool, error)

Enabled implements kernel.OptionalModule: on unless the operator turned it off (addons.analytics in the config or the addons.analytics.enabled setting). Disabled, it opens no analytics.db and registers nothing; an existing file is kept.

func (*Module) Migrations 

func (*Module) Migrations() fs.FS

Migrations implements kernel.Module. The module has no tables in the main database: its migrations (migrations/) belong to analytics.db and OpenStore applies them there.

func (*Module) Name 

func (*Module) Name() string

Name implements kernel.Module.

func (*Module) Register 

func (m *Module) Register(d *kernel.Deps) error

Register opens analytics.db, starts the write buffer (stopped and flushed when the app closes), serves the beacon script and endpoint, injects the script into public HTML, records form submissions (site.form.submitted), declares analytics.view, adds the hourly rollup cron and registers the admin pages, the dashboard widget and the summary API operation. A store that fails to open is logged and the module registers nothing else, so the site keeps serving with collection off.

type Point 

type Point struct {
	At              time.Time
	Visitors, Views int
}

Point is one bucket of Series, starting At (UTC).

type PointJSON 

type PointJSON struct {
	At       time.Time `json:"at"`
	Visitors int       `json:"visitors"`
	Views    int       `json:"views"`
}

PointJSON is one series bucket (hourly for a range of a day or less, else daily), starting at At (UTC).

type Row 

type Row struct {
	Key             string
	Visitors, Views int
}

Row is one key of Top.

type RowJSON 

type RowJSON struct {
	Key      string `json:"key"`
	Visitors int    `json:"visitors"`
	Views    int    `json:"views"`
}

RowJSON is one key of a top list. Paths are percent-encoded as stored; campaign keys are source/medium/campaign.

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store owns analytics.db. It is safe for concurrent use.

Every use of the handle (Insert, VisitorHash, With) holds the read lock for its whole duration, and Reopen takes the write lock, so the old handle is closed only once no connection of it is in use. Closing an in-use connection later would let SQLite's WAL close unlink the -wal and -shm files by path, which by then belong to the recreated database.

func OpenStore 

func OpenStore(dataDir string) (*Store, error)

OpenStore opens (creating it when missing) <dataDir>/analytics.db and applies the module's migrations to it.

func (*Store) Close 

func (s *Store) Close() error

Close closes analytics.db, waiting for in-flight uses. The store stays closed: later calls fail with ErrClosed instead of recreating the file.

func (*Store) DB 

func (s *Store) DB() *sql.DB

DB returns the current handle without holding any lock: a Reopen may close it at any moment. It exists for tests and diagnostics; everything else uses With. It is never nil (a closed handle reports "sql: database is closed").

func (*Store) DeleteAll 

func (s *Store) DeleteAll(ctx context.Context) (deleted, error)

DeleteAll empties events and daily and drops every salt but today's (UTC), then VACUUMs analytics.db. Today's salt is kept so a visitor seen again today keeps the same hash.

func (*Store) DeleteBefore 

func (s *Store) DeleteBefore(ctx context.Context, day time.Time) (deleted, error)

DeleteBefore removes the events before day's UTC midnight and the daily rows of the days before it, then VACUUMs analytics.db.

func (*Store) Forms 

func (s *Store) Forms(ctx context.Context, f Filter) ([]FormRow, error)

Forms returns, for every form submitted in f's range, per page it was submitted on: that page's views, the submissions and the conversion rate, most submissions first. A form with no submission in the range is not listed (the module does not know the site's forms).

func (*Store) Insert 

func (s *Store) Insert(ctx context.Context, evs []Event) error

Insert stores evs, InsertBatch rows per transaction, holding the read lock throughout. A batch that fails stops the call; earlier batches stay committed.

func (*Store) Now 

func (s *Store) Now(ctx context.Context, at time.Time) (int, error)

Now returns the distinct visitors with a view in the 5 minutes up to at (inclusive at both ends, at second precision).

func (*Store) Path 

func (s *Store) Path() string

Path is the analytics.db file path.

func (*Store) Reopen 

func (s *Store) Reopen() error

Reopen waits for in-flight uses, closes the handle and opens the file again, creating and migrating it when it is missing. It is the recreate path for a file deleted while running, and what a restore uses to pick up a replaced file.

func (*Store) Rollup 

func (s *Store) Rollup(ctx context.Context, now time.Time) error

Rollup recomputes daily, with replace semantics, from the earlier of yesterday and the newest day already in daily, through today (now's UTC day). That is today and yesterday on every regular hourly run, and it also fills the days a stopped server missed. When daily is empty but events is not (first start, or daily was cleared), it backfills from the first event's day. Every recomputed day gets a total row, zero when the day had no events, so reads can tell a quiet day from one not rolled up. Running it twice gives the same daily.

func (*Store) Series 

func (s *Store) Series(ctx context.Context, f Filter) ([]Point, error)

Series returns visitors and views per hour when f spans at most one day, else per UTC day, with a zero point for every empty bucket. A day point's visitors are that day's distinct visitors.

func (*Store) Summary 

func (s *Store) Summary(ctx context.Context, f Filter) (Summary, error)

Summary returns f's visitors, views, submissions and views per visitor, with Prev for the equal-length period before it.

func (*Store) Top 

func (s *Store) Top(ctx context.Context, f Filter, dim string, limit int) ([]Row, error)

Top returns f's keys of dim (page, ref, utm, browser, os, device or country) by visitors, then views, then key; limit <= 0 returns all. A key's views come from daily for settled days; its visitors are counted distinct on events over the whole range. The utm key is "source/medium/campaign", as in daily, and views without utm are left out.

func (*Store) VisitorHash 

func (s *Store) VisitorHash(ctx context.Context, day time.Time, siteID, ip, ua string) (string, error)

VisitorHash returns hex(sha256(salt || siteID || ip || ua))[:16], where salt is the 32 random bytes of day's UTC date. It creates that salt on first use and then deletes the salts of earlier days, so a visitor cannot be linked across days and the raw IP and user agent are never stored. Pass the time of the request. A day older than the newest stored salt (a request straddling midnight) uses the newest salt: a past day's salt is never recreated once it has been rotated out.

func (*Store) With 

func (s *Store) With(ctx context.Context, fn func(*sql.DB) error) error

With runs fn with the analytics.db handle while holding the store's read lock, recreating the file first when it was deleted, so a read after a file loss sees an empty database. fn must not keep the handle, and must not call other Store methods (the lock is not reentrant).

type Summary 

type Summary struct {
	Visitors, Views, Submits int
	ViewsPerVisitor          float64
	Prev                     *Summary
}

Summary is a period's headline numbers. Prev is the same numbers for the previous period of equal length, ending where this one starts (nil on Prev itself).

type SummaryJSON 

type SummaryJSON struct {
	Visitors        int          `json:"visitors"`
	Views           int          `json:"views"`
	Submits         int          `json:"submits"`
	ViewsPerVisitor float64      `json:"views_per_visitor"`
	Prev            *SummaryJSON `json:"prev,omitempty"`
}

SummaryJSON is a period's headline numbers in the API.

type SummaryResponse 

type SummaryResponse struct {
	From    time.Time            `json:"from"`
	To      time.Time            `json:"to"`
	Summary SummaryJSON          `json:"summary"`
	Series  []PointJSON          `json:"series"`
	Top     map[string][]RowJSON `json:"top"`
}

SummaryResponse is the body of GET /api/v1/analytics/summary.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.