perfharness

package
v0.3.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 9, 2026 License: GPL-3.0 Imports: 39 Imported by: 0

Documentation

Index

Constants

View Source
const Password = "etherperf-password"

Credentials minted into the throwaway copy. Every viewer signs in with Password through the real protocol sign-in routes; Plex and the owner console use a minted Plex token; Stremio uses a minted surface grant.

View Source
const RestrictedViewer = 2

RestrictedViewer is the fixture viewer with a rating cap and a plex.tv share filter, so restricted listings are measured on a full library.

Variables

View Source
var Budgets = map[string]Budget{
	// contains filtered or unexported fields
}
View Source
var LargeScale = Scale{Movies: 250_000, Shows: 10_000, SeasonsPerShow: 4, EpisodesPerSeason: 6, Viewers: 2, HistoryPerViewer: 20_000, InProgressPerViewer: 40}

LargeScale is the CI gate's library: 500k+ catalogue rows and a household with deep watch history.

View Source
var OffMirror800k = OffMirror{Movies: 680_000, Shows: 120_000, DetailMovies: 3_000, DetailShows: 1_500, DetailSeasons: 3, DetailEpisodes: 8, Viewers: 4, XtreamIDs: true}

OffMirror800k is the upgrade fixture: about 800k titles.

View Source
var SmallScale = LargeScale.Divide(10)

SmallScale is LargeScale's household over a tenth of its library; the gate compares the two, so only the library differs.

View Source
var XLargeScale = Scale{Movies: 1_073_442, Shows: 205_326, SeasonsPerShow: 2, EpisodesPerSeason: 13, EpisodeShows: 300, Viewers: 2, HistoryPerViewer: 20_000, InProgressPerViewer: 40}

XLargeScale is the nightly gate's library: a full TMDB mirror, 1.28M titles (1,073,442 movies and 205,326 series, TMDB's dated, non-adult counts), each series with two seasons stored and the episodes of 300 opened series, as the long-tail universe (longtail.go) mirrors it. Its pair is XLargeScale.Divide(10).

Functions

func ApplyPageBound

func ApplyPageBound(rows []*Row, results []Result, trace map[string]TraceRecord, caps map[string]int)

ApplyPageBound sets each listing row's page bound: the titles its requests rendered and by how much its worst request read past what they render (caps from DetailCaps).

func Audit

func Audit(ctx context.Context, dbPath string, trace map[string]TraceRecord, w io.Writer) error

audit runs EXPLAIN QUERY PLAN for every statement the trace attributes to a request and ranks them by the time they cost, flagging full scans of large tables, temporary sort trees, automatic indexes, correlated subqueries and statements repeated within one request.

func AuditPostgres

func AuditPostgres(ctx context.Context, url string, trace map[string]TraceRecord, w io.Writer) error

AuditPostgres audits the same statements against a PostgreSQL database (a cluster node's, or a copy of one) with EXPLAIN (GENERIC_PLAN), which plans a parameterized statement without values (PostgreSQL 16+). It flags sequential scans of large tables, sorts and correlated subplans.

func CgroupMemory

func CgroupMemory(dir string) (current, peak int64)

CgroupMemory reads a cgroup's memory.current and memory.peak (0 when unavailable).

func DetailCaps

func DetailCaps(ctx context.Context, db *sql.DB) (map[string]int, error)

DetailCaps is, for each title type, the most rows a page's hydration of one title of that type reads in the generated library: its detail rows (the facts catalog/store.go hydrateFactBatch reads) plus pageItemRows, and for a series two rows per season (aired counts). Every stored type participates, including titles with no own facts. Children also read their parent's identity, keywords and alternative titles. Later episodes read all positive-season counts for absolute numbering; seasons do not. These are per-title upper bounds: hydration shares a parent's reads when multiple rendered children have the same parent, so repeating its cost is conservative. Parent credits/genres and unrelated shows' seasons are not inherited and must not inflate a child's cap.

func DockerCgroup

func DockerCgroup(containerID string) (string, error)

DockerCgroup is the cgroup v2 directory of a running container under the systemd cgroup driver, or under the cgroupfs driver as a fallback.

func Generate

func Generate(ctx context.Context, db *sql.DB, s Scale, now time.Time) error

Generate fills an empty, migrated database with a synthetic library and household at scale s. The output is deterministic for a given scale and now.

func LoadTMDBYearCounts

func LoadTMDBYearCounts(raw []byte, capped bool) ([2]map[int]int, error)

LoadTMDBYearCounts reads per-year TMDB discover totals ({"movie":{"years": {"2024":[total,released]}},"tv":…}). Capped, each year keeps the 10,000 titles a discover slice returns, which is what a catalogue sync with "Hide obscure titles" off can mirror.

Uncapped, it is the whole of TMDB: a probe that stopped at 20,001 is replaced by the exact count in movie_exact_truncated_years.

func RunRankBench

func RunRankBench(ctx context.Context, db *sql.DB, path string) ([]RankBenchStep, []RankBenchPage, error)

func WriteBaselineReport

func WriteBaselineReport(w io.Writer, title string, meta []string, rows []BaselineRow)

WriteBaselineReport writes one server's measured replay as markdown.

func WriteComparison

func WriteComparison(w io.Writer, targets []Target, client string)

WriteComparison writes per-step tables comparing targets: latency, CPU and response size. Steps are matched by client and step name.

func WriteRankBench

func WriteRankBench(w io.Writer, steps []RankBenchStep, pages []RankBenchPage)

func WriteReport

func WriteReport(w io.Writer, title string, meta []string, rows []*Row) (failed int)

func WriteSizeReport

func WriteSizeReport(w io.Writer, sizes []ObjectSize, detailed int64, minBytes int64)

WriteSizeReport writes sizes grouped by table: the table, each of its indexes, and the group's total, with bytes per detailed title when detailed is positive.

func WriteSyncBench

func WriteSyncBench(w io.Writer, phases []SyncBenchPhase)

WriteSyncBench renders phases as a table.

func WriteTailBench

func WriteTailBench(w io.Writer, rows int, vs []TailVariant)

WriteTailBench renders the variants.

Types

type BackgroundWriter

type BackgroundWriter struct {
	// contains filtered or unexported fields
}

backgroundWriter imitates a catalogue refresh against the same database: write transactions of 500 title rows, back to back with a short pause, so request paths meet a busy writer the way they do while the mirror syncs.

func StartBackgroundWriter

func StartBackgroundWriter(dbPath string) (*BackgroundWriter, error)

func (*BackgroundWriter) Close

func (w *BackgroundWriter) Close() string

type BaselineRow

type BaselineRow struct {
	Client string `json:"client"`
	Step   string `json:"step"`
	Class  string `json:"class"`
	// Status is the HTTP status of the warm requests (of the cold one when
	// there were none); 0 when the step was skipped or failed to connect.
	Status  int    `json:"status"`
	Skipped string `json:"skipped,omitempty"`
	Error   string `json:"error,omitempty"`

	Cold      time.Duration `json:"cold"`
	ColdCPU   time.Duration `json:"coldCpu"`
	ColdRead  int64         `json:"coldReadBytes"`
	ColdWrite int64         `json:"coldWriteBytes"`

	WarmN     int           `json:"warmN"`
	WarmP50   time.Duration `json:"warmP50"`
	WarmP95   time.Duration `json:"warmP95"`
	WarmCPU   time.Duration `json:"warmCpuMedian"`
	WarmRead  int64         `json:"warmReadBytesMedian"`
	WarmWrite int64         `json:"warmWriteBytesMedian"`

	Bytes     int `json:"bytes"`
	GzipBytes int `json:"gzipBytes"`
	WireBytes int `json:"wireBytes"`
	Items     int `json:"items"`
	TotalSize int `json:"totalSize"`

	// Queries is the median statement count per request, when the server
	// wrote a perf trace (Ether only).
	Queries int64 `json:"queries,omitempty"`
}

BaselineRow is one step's measurements against one server: its cold request (the first a freshly restarted server answered), warm latency percentiles, response sizes, and the server's CPU time and block IO per request. It is what etherperf baseline writes, for PMS and for Ether alike, and what etherperf compare reads.

func SummarizeBaseline

func SummarizeBaseline(results []Result, trace map[string]TraceRecord, order []Client) []BaselineRow

SummarizeBaseline groups a measured replay's results by client and step. Only the cold and warm phases count; others (discovery, priming) do not.

type Budget

type Budget struct {
	Warm, Cold, Loaded time.Duration
	Gated              bool
}

budget is the latency a class must meet: p95 of the warm phase, the single cold request, and p95 under concurrent viewers or a busy writer.

type CgroupProbe

type CgroupProbe struct{ Dir string }

CgroupProbe reads a cgroup v2 directory: cpu.stat's usage_usec (every process in the group, microsecond resolution) and io.stat's rbytes and wbytes summed over devices. Block IO counts only what missed the page cache; writes are charged when the kernel writes them back, which may be after the request that dirtied them.

func (CgroupProbe) Sample

func (p CgroupProbe) Sample() (Usage, error)

type Client

type Client struct {
	Name    string
	Surface string
	// Port offset from the base port (0 main, 1 Jellyfin, 2 Emby, 3 Silo),
	// or SharedPrefix on the main port when the runner shares one port.
	Port         int
	SharedPrefix string
	Prefix       string
	Headers      func(s *session) http.Header
	SignIn       *Step
	Steps        []Step
}

client is one real client's request sequence against one protocol.

func Clients

func Clients() []Client

Clients is every replayed client, in report order.

func Only

func Only(cs []Client, names []string) []Client

Only keeps the clients named in names (all of them when names is empty).

func OnlySteps

func OnlySteps(cs []Client, names []string) []Client

OnlySteps keeps the steps whose name contains one of names (case insensitive); clients left with no steps are dropped. Sign-in always runs.

type LongTailConfig

type LongTailConfig struct {
	// Titles is the number of movies plus shows in the TMDB universe.
	Titles int
	// YearCounts, when set, fixes how many movies ([0]) and shows ([1]) the
	// universe has per release year, replacing Titles and MovieShare: for
	// example TMDB's own counts, capped at the 10,000 a discover year slice
	// can return (LoadTMDBYearCounts).
	YearCounts [2]map[int]int
	// MovieShare is the movies' share of Titles (0 = 0.7).
	MovieShare float64
	Seed       uint64
	// Now is the catalogue's clock and the universe's present.
	Now time.Time
	// HideObscure keeps the default browsable bar (popularity 5 or 100
	// votes). Off is "Hide obscure titles" turned off: everything released
	// is browsable, so the catalogue details the whole universe.
	HideObscure bool
	// OpenShows is how many of the most popular shows a household opened,
	// so their seasons and episodes are stored (0 = 300, -1 = none). It
	// does not grow with the universe: viewers open what they watch.
	OpenShows int
	// SkipSync goes straight to detailing (a database a sync already filled).
	SkipSync bool
	// DetailLimit stops warming after this many detail fetches (0 = all).
	DetailLimit int
	// Progress, if set, is called as phases advance.
	Progress func(phase string, done int)
	Log      *slog.Logger
}

LongTailConfig sizes and drives a long-tail generation.

type LongTailReport

type LongTailReport struct {
	Universe       int
	Browsable      int // titles in the universe meeting the default bar
	Synced         time.Duration
	Filled         int // released titles detailed through Get after the sync
	FillWaited     int // of those, Gets that returned before their fetch landed
	FillTook       time.Duration
	Warmed         int
	WarmTook       time.Duration
	SeasonsOpened  int
	DetailRequests int
}

LongTailReport is what a generation did.

func GenerateLongTail

func GenerateLongTail(ctx context.Context, db *sql.DB, cfg LongTailConfig) (LongTailReport, error)

GenerateLongTail mirrors a long-tail TMDB universe into db through the catalogue: a sync, then detail warming until nothing browsable is left undetailed, then the seasons a household opened. db must be migrated.

type LongTailTMDB

type LongTailTMDB struct {
	// contains filtered or unexported fields
}

LongTailTMDB is an in-process TMDB serving a seeded long-tail universe. It satisfies catalog.TMDB. A title's payload is rebuilt from its seed on every request, so the universe costs a few bytes per title.

func NewLongTailTMDB

func NewLongTailTMDB(cfg LongTailConfig) *LongTailTMDB

NewLongTailTMDB builds the universe cfg describes.

func (*LongTailTMDB) CollectionDetails

func (u *LongTailTMDB) CollectionDetails(_ context.Context, id int64) (*tmdb.CollectionDetails, error)

func (*LongTailTMDB) DiscoverMovies

func (u *LongTailTMDB) DiscoverMovies(_ context.Context, p tmdb.DiscoverParams) (*tmdb.Paged[tmdb.Movie], error)

func (*LongTailTMDB) DiscoverTV

func (*LongTailTMDB) EpisodeExternalIDs

func (u *LongTailTMDB) EpisodeExternalIDs(_ context.Context, tvID int64, season, episode int) (*tmdb.ExternalIDs, error)

func (*LongTailTMDB) FindByExternalID

func (u *LongTailTMDB) FindByExternalID(context.Context, string, string) (*tmdb.FindResult, error)

func (*LongTailTMDB) Genres

func (u *LongTailTMDB) Genres(_ context.Context, mediaType string) ([]tmdb.Genre, error)

func (*LongTailTMDB) MovieDetails

func (u *LongTailTMDB) MovieDetails(_ context.Context, id int64) (*tmdb.MovieDetails, error)

func (*LongTailTMDB) MovieDetailsFresh

func (u *LongTailTMDB) MovieDetailsFresh(ctx context.Context, id int64) (*tmdb.MovieDetails, error)

MovieDetailsFresh and TVDetailsFresh serve catalog.RefreshTitle.

func (*LongTailTMDB) MovieList

func (u *LongTailTMDB) MovieList(ctx context.Context, _ string, page int) (*tmdb.Paged[tmdb.Movie], error)

func (*LongTailTMDB) MovieRecommendations

func (u *LongTailTMDB) MovieRecommendations(ctx context.Context, id int64, _ int) (*tmdb.Paged[tmdb.Movie], error)

func (*LongTailTMDB) MovieSimilar

func (u *LongTailTMDB) MovieSimilar(context.Context, int64, int) (*tmdb.Paged[tmdb.Movie], error)

func (*LongTailTMDB) SearchMovies

func (u *LongTailTMDB) SearchMovies(context.Context, string, int) (*tmdb.Paged[tmdb.Movie], error)

func (*LongTailTMDB) SearchMulti

func (*LongTailTMDB) SearchTV

func (*LongTailTMDB) SeasonDetails

func (u *LongTailTMDB) SeasonDetails(_ context.Context, tvID int64, season int) (*tmdb.SeasonDetails, error)

func (*LongTailTMDB) TVDetails

func (u *LongTailTMDB) TVDetails(_ context.Context, id int64) (*tmdb.TVDetails, error)

func (*LongTailTMDB) TVDetailsFresh

func (u *LongTailTMDB) TVDetailsFresh(ctx context.Context, id int64) (*tmdb.TVDetails, error)

func (*LongTailTMDB) TVList

func (u *LongTailTMDB) TVList(ctx context.Context, _ string, page int) (*tmdb.Paged[tmdb.TVShow], error)

func (*LongTailTMDB) TVRecommendations

func (u *LongTailTMDB) TVRecommendations(ctx context.Context, id int64, _ int) (*tmdb.Paged[tmdb.TVShow], error)

func (*LongTailTMDB) TVSimilar

func (u *LongTailTMDB) TVSimilar(context.Context, int64, int) (*tmdb.Paged[tmdb.TVShow], error)

func (*LongTailTMDB) TrendingAll

func (*LongTailTMDB) TrendingMovies

func (u *LongTailTMDB) TrendingMovies(_ context.Context, _ string, page int) (*tmdb.Paged[tmdb.Movie], error)

The live lists are the most popular titles.

func (*LongTailTMDB) TrendingTV

func (u *LongTailTMDB) TrendingTV(_ context.Context, _ string, page int) (*tmdb.Paged[tmdb.TVShow], error)

type ObjectSize

type ObjectSize struct {
	Name  string `json:"name"`
	Table string `json:"table"` // the table it belongs to (itself for a table)
	Index bool   `json:"index"`
	Bytes int64  `json:"bytes"`
	Rows  int64  `json:"rows,omitempty"` // tables only
}

ObjectSize is the on-disk size of one table or index.

func MeasureSizes

func MeasureSizes(ctx context.Context, db *sql.DB, withRows bool) ([]ObjectSize, error)

MeasureSizes reports every table's and index's size: dbstat pages on SQLite, pg_relation_size (with TOAST for tables) on PostgreSQL. FTS5 shadow tables count toward their virtual table.

type OffMirror

type OffMirror struct {
	MovieExport, TVExport string
	// Movies and Shows are the most popular export ids stored (0.3.5's Off
	// mirror held about 680k movies and 120k series).
	Movies, Shows int
	// DetailMovies and DetailShows were opened or warmed (detail=1); each
	// opened show has DetailSeasons seasons of DetailEpisodes episodes.
	DetailMovies, DetailShows     int
	DetailSeasons, DetailEpisodes int
	// Viewers each watched, listed and favourited titles, a share of them
	// in the long tail.
	Viewers int
	// XtreamIDs assigns an Xtream compact id to every stored movie and
	// series, as a client that loaded the VOD lists has.
	XtreamIDs bool
	// Stride keeps every Stride-th title of the popularity-ranked export
	// (a smaller mirror with the same share of long tail); 0 or 1 keeps
	// the first Movies and Shows.
	Stride int
	Now    time.Time
}

OffMirror describes a catalogue as a 0.3.5 server stored it with the Off preset: every released title the /discover walk reached, as summary rows (title, popularity, votes, genres, country), with the household's touches in the stores that hold them. Titles, ids, original names and popularity are TMDB's (the daily exports); vote counts are drawn so the share of titles at or above the library tier's floor matches the live /discover counts (votes >= 10: ~107k movies, ~20k series).

type OffMirrorSummary

type OffMirrorSummary struct {
	Movies, Shows, Seasons, Episodes int
	// AtFloor titles meet the Off floor on their stored values; TailTouched
	// are below it and named by a touch source.
	AtFloor, TailTouched int
	Rows                 map[string]int
}

OffMirrorSummary counts what GenerateOffMirror wrote.

func GenerateOffMirror

func GenerateOffMirror(ctx context.Context, db *sql.DB, o OffMirror) (OffMirrorSummary, error)

GenerateOffMirror fills an empty, migrated database with an Off-preset 0.3.5 catalogue and household. The output is deterministic for o.

type Probe

type Probe interface {
	Sample() (Usage, error)
}

Probe samples a measured server's resource counters, so a sequential replay can charge each request with the CPU and disk work the server did while answering it.

type RankBenchPage

type RankBenchPage struct {
	Name       string        `json:"name"`
	Rank       bool          `json:"rank"`
	Statements int64         `json:"statements"`
	Rows       int64         `json:"rows"`
	DB         time.Duration `json:"db"`
}

type RankBenchStep

type RankBenchStep struct {
	Name       string        `json:"name"`
	Took       time.Duration `json:"took"`
	WAL        int64         `json:"wal"`
	WriterWait time.Duration `json:"writerWait,omitempty"`
}

RankBench measures the popularity rank (catalog/rank.go) on a catalogue copy: the first build and the index swap with a concurrent writer, a day's popularity refresh (every movie and show, 250 per transaction, as the sync writes them) and the rebuild after it, and popularity browse pages before and after.

type Result

type Result struct {
	SchemaVersion int           `json:"schemaVersion,omitempty"`
	Phase         string        `json:"phase"`
	Client        string        `json:"client"`
	Surface       string        `json:"surface"`
	Step          string        `json:"step"`
	Class         string        `json:"class"`
	Viewer        string        `json:"viewer"`
	Tag           string        `json:"tag"`
	Status        int           `json:"status"`
	Bytes         int           `json:"bytes"`
	BodySHA       string        `json:"bodySha256,omitempty"`
	Dur           time.Duration `json:"dur"`
	Err           string        `json:"err,omitempty"`
	Skipped       string        `json:"skipped,omitempty"`
	// RenderedItems counts the titles a listing response rendered, by type.
	RenderedItems map[string]int `json:"renderedItems,omitempty"`
	// Set when the runner measures (Measure): the bytes on the wire, the
	// body's size gzipped at the default level, and the server's CPU time
	// and block IO while it answered.
	WireBytes  int           `json:"wireBytes,omitempty"`
	GzipBytes  int           `json:"gzipBytes,omitempty"`
	CPU        time.Duration `json:"cpu,omitempty"`
	ReadBytes  int64         `json:"readBytes,omitempty"`
	WriteBytes int64         `json:"writeBytes,omitempty"`
	Probed     bool          `json:"probed,omitempty"`
	// Items and TotalSize are a Plex MediaContainer's size and totalSize
	// (measure mode), so an empty page is not mistaken for a fast one.
	Items     int `json:"items,omitempty"`
	TotalSize int `json:"totalSize,omitempty"`
}

result is one timed request.

type Row

type Row struct {
	Client, Step, Class     string
	RowsBound, QueriesBound int
	// Export and Bytes (the median response size): an export step's work
	// is compared per response byte (Step.Export).
	Export bool
	Bytes  int64
	// WritesMax is the most statements that wrote in one request.
	WritesMax                              int64
	Cold, WarmP50, WarmP95, ConcP95, BgP95 time.Duration
	Queries, Writes, Rows                  int64
	AllocBytes, Allocs                     int64
	// The fewest and most statements and rows over the cold and warm
	// passes. A pass may or may not be answered by a cache, so the gate
	// compares the large library's least work with the small one's most.
	QueriesMin, QueriesMax, RowsMin, RowsMax int64
	DBTime, LockWait                         time.Duration
	// Cold is the owner's first pass after the start, First another
	// viewer's first pass (that viewer's caches cold, the server's warm);
	// the most work one request of each took.
	ColdWork, FirstWork Work
	// The page bound of a listing (ApplyPageBound): its worst request's
	// rows, the rows its rendered titles account for, and their number.
	PageCounted         bool
	PageRows, PageBound int64
	PageItems           int
	Status              string
	Fails               []string
	Skipped             string
	Errors              []string
	// contains filtered or unexported fields
}

row is one endpoint's line in the report.

func Summarize

func Summarize(results []Result, trace map[string]TraceRecord, order []Client) []*Row

summarize groups results by client and step and checks each against its class budget. A step with no samples in a phase is not judged for it.

func (*Row) PageOver

func (r *Row) PageOver() (string, bool)

PageOver reports, for a listing row with countable titles, whether its worst request read more than its titles render.

type Runner

type Runner struct {
	// contains filtered or unexported fields
}

func NewRunner

func NewRunner(basePort int, shared bool, viewers []Viewer) *Runner

NewRunner drives a server whose main port is basePort. With shared set, Jellyfin, Emby and Silo are reached under their path prefixes on that port; otherwise on basePort+1, +2 and +3.

func (*Runner) ColdEach

func (r *Runner) ColdEach(ctx context.Context, phase string, v Viewer, cs []Client, restart func(ctx context.Context, c Client, st Step) error) error

ColdEach replays each client's steps for viewer v one at a time, calling restart before every step, so each step's request is the first a freshly started server answers. The steps' variables must already be discovered (a prior pass under another phase); a step that discovers variables still records them.

func (*Runner) Concurrent

func (r *Runner) Concurrent(ctx context.Context, phase string, n, rounds int, cs []Client)

concurrent runs n workers, each a viewer working through every client in its own order, rounds times.

func (*Runner) Measure

func (r *Runner) Measure(probe Probe)

Measure records each response's size on the wire and gzipped, and, when probe is not nil, the server's CPU time and block IO during each request. The probe's counters are server-wide, so it is meaningful only while one request runs at a time (Sequential, ColdEach).

func (*Runner) Results

func (r *Runner) Results() []Result

Results returns every timed request so far.

func (*Runner) SaveBodies added in v0.3.3

func (r *Runner) SaveBodies(dir string)

SaveBodies writes the last warm response of each viewer, client and step to dir, so two builds' answers can be compared once personalization has settled.

func (*Runner) Sequential

func (r *Runner) Sequential(ctx context.Context, phase string, viewers []Viewer, cs []Client)

sequential runs every client for each viewer, one request at a time.

func (*Runner) SetBefore

func (r *Runner) SetBefore(fn func(context.Context))

SetBefore runs fn before each request: the gate waits there for the server's background work to finish, so every request starts from the same state.

func (*Runner) SetHost

func (r *Runner) SetHost(host string)

SetHost sends requests to host instead of 127.0.0.1, for a server in a container or on another machine.

func (*Runner) SetTimeout

func (r *Runner) SetTimeout(d time.Duration)

SetTimeout caps each request (60 s by default).

type Scale

type Scale struct {
	Movies            int
	Shows             int
	SeasonsPerShow    int
	EpisodesPerSeason int
	// EpisodeShows is how many series have their episodes stored (0: all).
	// The catalogue stores a series' seasons with its details and its
	// episodes only once a season is opened, so on a full mirror they are
	// the household's series, not the catalogue's.
	EpisodeShows int
	// Viewers beyond the owner, each with HistoryPerViewer watched titles
	// and InProgressPerViewer partly watched ones.
	Viewers             int
	HistoryPerViewer    int
	InProgressPerViewer int
}

Scale sizes a synthetic library. Every title is browsable, detailed and fresh, so a request never needs TMDB: the fixture measures Ether, not a provider.

func (Scale) Divide

func (s Scale) Divide(n int) Scale

Divide shrinks the library by n, keeping the household.

func (Scale) Rows

func (s Scale) Rows() int

Rows is the number of catalogue rows the scale produces.

func (Scale) Titles

func (s Scale) Titles() int

Titles is the number of movies and series.

type Shape

type Shape struct {
	Type       core.MediaType
	Popularity float64
	Votes      int
	Rating     float64
	// Tier is 0 for the obscure tail and 1 for a blockbuster. It scales
	// every detail count below.
	Tier float64

	Cast, Directors, Writers, Crew int
	Keywords                       int
	Genres                         int
	AltTitles                      int
	Studios                        int
	Countries                      int
	// ReleaseCountries is how many countries list release dates (movies).
	ReleaseCountries int
	// Collection is true for a movie in a franchise collection.
	Collection bool
	// CrowdRecs is the length of the first recommendations page.
	CrowdRecs int
	// Seasons and EpisodesPerSeason describe a show.
	Seasons, EpisodesPerSeason int
	// SummaryLen is the overview's length in characters (0: none).
	SummaryLen                   int

Shape is the counts-level description of one title: what its TMDB detail payload carries. LongTailShape draws it; LongTailTMDB builds the payload from it, and the CI fixture writes rows from it directly.

func LongTailShape

func LongTailShape(rng *rand.Rand, typ core.MediaType) Shape

LongTailShape draws one title's shape. Popularity and votes are the first draws, so a universe can sort by them cheaply.

func (Shape) Browsable

func (s Shape) Browsable() bool

Browsable reports whether the shape meets the default browsable bar (popularity 5 or 100 votes).

func (Shape) Credits

func (s Shape) Credits() int

Credits is the number of credit rows the catalogue keeps for the shape: cast is capped at 30 by the catalogue, crew is not.

type Step

type Step struct {
	Name  string
	Class string
	// RowsBound, when set, is a hard bound on the rows the step reads,
	// whatever the library's size: a search reads at most its candidate
	// window, so its rows grow with the matches only until the window
	// fills. The gate checks the bound instead of comparing scales.
	RowsBound int
	// QueriesBound, when set, is a hard bound on the statements one
	// request of the step runs, whatever the library's size, and the step
	// must write nothing: a read that loops over its titles or rows, or
	// saves what it shows, breaks it before the scale comparison would.
	QueriesBound int
	// Export marks a step whose response is the whole library by design
	// (an IPTV app's VOD list): its work grows with what it renders, so
	// the gate compares work per response byte instead of per request.
	Export   bool
	Method   string
	Path     string
	Body     string
	Discover func(vars map[string]string, body []byte)
}

type SyncBenchConfig

type SyncBenchConfig struct {
	// Titles is the universe: movies and shows the list phases store.
	Titles int
	// Details is how many of them the detail phases fetch (0 = 2,000).
	Details int
	// IDBase puts the universe's TMDB ids above every id the catalogue
	// already holds (0 = 50,000,000).
	IDBase int64
	Seed   uint64
	// Path is the SQLite file, for WAL size sampling; empty on PostgreSQL.
	Path string
	Log  *slog.Logger
	// Progress, if set, gets a line per phase as it starts.
	Progress func(string)
}

SyncBench measures how fast the catalogue's real write paths take titles into an existing (large) catalogue when TMDB costs nothing: the database-bound ceiling of a sync. The fake TMDB serves a long-tail universe of new titles, with ids above IDBase, and four phases run in turn:

  • list fill: catalog.Sync stores the universe's discover pages as new browsable summaries;
  • list refresh: the same sync again, every title's popularity and votes moved by 15%;
  • detail fill: Get on Details of the new titles (details, seasons and crowd recommendations), eight at a time as WarmDetails' callers do;
  • detail refresh: RefreshTitle on the same titles, one credit and one keyword changed.

The server's background WAL checkpointer runs throughout.

type SyncBenchOperations

type SyncBenchOperations struct {
	Attempted int `json:"attempted"`
	Succeeded int `json:"succeeded"`
	Rejected  int `json:"rejected"`
}

SyncBenchOperations counts completed detail API calls, independently of any later audit of whether successfully touched titles survived an upgrade. A rejected call is not a successful touch, even if other calls succeeded.

type SyncBenchPhase

type SyncBenchPhase struct {
	Name         string        `json:"name"`
	Titles       int           `json:"titles"`
	Wall         time.Duration `json:"wall"`
	TitlesPerSec float64       `json:"titlesPerSec"`
	// TMDB is time the fake TMDB spent building payloads; DBBound is
	// titles per second with it subtracted.
	TMDB       time.Duration `json:"tmdb"`
	DBBound    float64       `json:"dbBoundTitlesPerSec"`
	DBTime     time.Duration `json:"dbTime"`
	DBPerTitle time.Duration `json:"dbPerTitle"`
	Statements int64         `json:"statements"`
	StmtsTitle float64       `json:"statementsPerTitle"`
	LockWait   time.Duration `json:"lockWait"`
	// WrittenPerTitle is bytes the process wrote to disk per title (WAL
	// and checkpoints; Linux), WALPerTitle PostgreSQL's WAL bytes.
	WrittenPerTitle float64              `json:"writtenPerTitle,omitempty"`
	WALPerTitle     float64              `json:"walPerTitle,omitempty"`
	PeakWAL         int64                `json:"peakWal,omitempty"`
	Gate            store.WriteGateHolds `json:"gate"`
	// Detail operations are counted individually. List sync is one bulk call
	// and does not expose per-title outcomes, so its Operations is absent.
	Operations *SyncBenchOperations `json:"operations,omitempty"`
	Error      string               `json:"error,omitempty"`
}

SyncBenchPhase is one phase's measurement.

func RunSyncBench

func RunSyncBench(ctx context.Context, db *sql.DB, cfg SyncBenchConfig) ([]SyncBenchPhase, error)

RunSyncBench runs the four phases against db, which holds a catalogue (typically a large fixture clone).

type TailRow

type TailRow struct {
	Key           int64
	Title         string
	OriginalTitle string
	Year          int
	Poster        string
	Popularity    float64
}

TailRow is one compact long-tail title.

func LoadTailExports

func LoadTailExports(movies, shows string) ([]TailRow, error)

LoadTailExports reads TMDB daily id exports (gzip JSON lines) for movies and shows, skipping adult titles. Exports carry the original title and popularity only; title, year and poster are drawn the way a /discover row would carry them (about 60% of originals are also the title, posters on 60% of titles), seeded by id.

type TailSearch

type TailSearch struct {
	Query string        `json:"query"`
	Rows  int           `json:"rows"`
	Took  time.Duration `json:"took"`
}

TailSearch is one query: matches read and the time to the top 20 by popularity (popularity held in memory, from the day's export).

type TailVariant

type TailVariant struct {
	Name       string        `json:"name"`
	Bytes      int64         `json:"bytes"`
	IndexBytes int64         `json:"indexBytes"`
	Load       time.Duration `json:"load"`
	LoadWAL    int64         `json:"loadWal"`
	Searches   []TailSearch  `json:"searches"`
}

TailVariant is one layout's measurement.

func RunTailBench

func RunTailBench(ctx context.Context, rows []TailRow, dir, pgURL string) ([]TailVariant, error)

RunTailBench builds each compact-tier layout on SQLite in dir (a file per layout) and, when pg is set, on PostgreSQL, and measures it.

type Target

type Target struct {
	Label string
	Rows  []BaselineRow
}

Target is one server's baseline rows under a label, for WriteComparison.

type TraceRecord

type TraceRecord struct {
	Tag        string          `json:"tag"`
	Surface    string          `json:"surface"`
	Method     string          `json:"method"`
	Path       string          `json:"path"`
	Status     int             `json:"status"`
	Duration   time.Duration   `json:"duration"`
	Queries    int64           `json:"queries"`
	Writes     int64           `json:"writes"`
	Rows       int64           `json:"rows"`
	DBTime     time.Duration   `json:"dbTime"`
	LockWait   time.Duration   `json:"lockWait"`
	AllocBytes uint64          `json:"allocBytes"`
	Allocs     uint64          `json:"allocs"`
	Statements []statementStat `json:"statements"`
}

traceRecord mirrors app.PerfTraceRecord (one line of ETHER_PERF_TRACE).

func ReadTrace

func ReadTrace(path string) (map[string]TraceRecord, []TraceRecord, error)

type Usage

type Usage struct {
	CPU        time.Duration // user + system CPU time
	ReadBytes  int64         // bytes read from block devices
	WriteBytes int64         // bytes written to block devices
}

Usage is a server's cumulative resource counters at one instant.

func (Usage) Sub

func (u Usage) Sub(earlier Usage) Usage

Sub is u minus an earlier sample.

type Viewer

type Viewer struct {
	ID         int64
	Username   string
	Owner      bool
	Restricted bool
	PlexToken  string
	Stremio    string
	// Xtream is an IPTV app password (the username is derived from it);
	// MCP is a read-scoped MCP grant, minted for the owner only.
	Xtream string
	MCP    string
}

func MintCredentials

func MintCredentials(ctx context.Context, db *sql.DB, max int) ([]Viewer, error)

MintCredentials gives up to max active viewers of a throwaway database a known password, a Plex token and a Stremio grant. The owner comes first. Never run it against a real server's database.

type Work

type Work struct {
	Queries, Rows int64
	DBTime        time.Duration
}

Work is the database work of one request.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL