Documentation
¶
Overview ¶
Package stats maintains the killboard's aggregate counters.
Three tables, all keyed the same way — (entity_type, entity_id, period_type, period_start):
- stats one row per entity per period, the headline counters
- stats_breakdowns the same, split by a dimension (ship flown, region, who they died to) — the large one, hundreds of millions of rows
- stats_leaderboards the top N entities per metric per period
Every killmail touches a lot of this. A single kill with fifty attackers updates the victim and fifty attackers, each across three period types, each with several breakdown dimensions — which is why the writer accumulates in memory and merges once rather than issuing an upsert per counter.
The enum values are stored as smallints and are a wire contract with the frontend and the API. They are never renumbered; the reserved gaps below are gaps on purpose.
Index ¶
- Constants
- Variables
- func Load(ctx context.Context, pool *pgxpool.Pool, killmailID int64) (Killmail, []Attacker, error)
- type Accumulator
- type Attacker
- type BackfillOptions
- type BackfillResult
- type Breakdown
- type BreakdownKey
- type CatchupResult
- type Derived
- type DimCategory
- type EntityType
- type Killmail
- type LeaderboardMetric
- type PeriodType
- type PipelineResult
- type Row
- type StatsKey
- type WriteResult
- func Write(ctx context.Context, pool *pgxpool.Pool, a *Accumulator, day time.Time) (WriteResult, error)
- func WritePeriod(ctx context.Context, pool *pgxpool.Pool, a *Accumulator, periodStart time.Time, ...) (WriteResult, error)
- func WritePeriodTx(ctx context.Context, tx pgx.Tx, a *Accumulator, periodStart time.Time, ...) (WriteResult, error)
Constants ¶
const ( // DailyRetentionDays is how long per-day rows are kept. Every rolling // window the site offers is 365 days or less, so a daily row older than // that answers no question — and daily is by far the largest slice. DailyRetentionDays = 365 // MonthlyRetentionMonths keeps a year and a half, enough for // year-over-year comparison with a margin. MonthlyRetentionMonths = 18 // MonthlyRebuildMonths is how far back monthly rows are rebuilt. Six // months including the current one: older months cannot change, because // the daily rows behind them have been purged. MonthlyRebuildMonths = 5 // BreakdownTopN caps how many dimensions survive per rolled-up period. // Without it a busy alliance's monthly row would carry every system it has // ever fought in, and the breakdown table is already the largest in the // database. BreakdownTopN = 20 // LeaderboardTopN is how deep each ranking goes. LeaderboardTopN = 100 )
Retention and rollup windows.
const CatchupBatch = 5_000
CatchupBatch is how many killmails are held in memory at once.
The accumulator for a whole day of a busy killboard is large but bounded; the killmails and their attackers are not, so they are streamed in batches rather than read in one slice.
Variables ¶
var ErrNotStored = errors.New("killmail is not stored")
ErrNotStored means the killmail the job names is not in the database.
Routine rather than exceptional: the stats job is dispatched alongside other follow-up work, and a killmail can be deleted between the dispatch and the run. Nothing to count, and nothing to retry.
Functions ¶
Types ¶
type Accumulator ¶
type Accumulator struct {
Stats map[StatsKey]*Row
Breakdowns map[BreakdownKey]*Breakdown
}
Accumulator collects the counter updates for one or more killmails.
Reusable across a batch: accumulating a day of killmails and merging once is dramatically cheaper than merging per kill, because the same character appears on many kills and collapses to a single row.
func NewAccumulator ¶
func NewAccumulator() *Accumulator
NewAccumulator returns an empty accumulator.
func (*Accumulator) Add ¶
func (a *Accumulator) Add(km Killmail, attackers []Attacker)
Add folds one killmail into the accumulator.
func (*Accumulator) AddFactions ¶ added in v0.1.1
func (a *Accumulator) AddFactions(km Killmail, attackers []Attacker)
AddFactions folds only faction headline counters into the accumulator.
A faction-only backfill deliberately avoids constructing every other entity and breakdown row just to discard them. Keep this in lockstep with the faction portions of Add: one attacker-side kill per distinct faction on the mail, plus one victim-side loss.
func (*Accumulator) Empty ¶
func (a *Accumulator) Empty() bool
Empty reports whether anything was accumulated.
func (*Accumulator) KeepEntityTypes ¶ added in v0.1.1
func (a *Accumulator) KeepEntityTypes(entityTypes []EntityType)
KeepEntityTypes removes aggregate rows outside an explicitly selected set. A nil/empty selection means all entity types, preserving the normal live and all-entity backfill behavior.
type Attacker ¶
type Attacker struct {
CharacterID int32
CorporationID int32
AllianceID int32
FactionID int32
ShipTypeID int32
DamageDone int64
FinalBlow bool
}
Attacker is one participant.
type BackfillOptions ¶
type BackfillResult ¶
type BackfillResult struct {
Months int `json:"months"`
DailyMonths int `json:"daily_months"`
MonthlyMonths int `json:"monthly_months"`
Killmails int64 `json:"killmails"`
Stats int64 `json:"stats"`
Breakdowns int64 `json:"breakdowns"`
MonthlyStats int64 `json:"monthly_stats"`
MonthlyBreakdowns int64 `json:"monthly_breakdowns"`
YearlyStats int64 `json:"yearly_stats"`
YearlyBreakdowns int64 `json:"yearly_breakdowns"`
}
func Backfill ¶
func Backfill(ctx context.Context, pool *pgxpool.Pool, opts BackfillOptions) (BackfillResult, error)
Backfill performs the same authoritative initial population as the TypeScript command. Old months are written directly at monthly granularity; recent months retain daily rows, then monthly and yearly rows are rebuilt from their lower-resolution source.
type Breakdown ¶
type Breakdown struct {
Kills int64 `json:"kills"`
Losses int64 `json:"losses"`
IskDestroyed float64 `json:"isk_destroyed"`
IskLost float64 `json:"isk_lost"`
// LastKillmail records the most recent kill contributing to this
// breakdown, which is what "last seen flying" is read from.
LastKillmailID int64 `json:"last_killmail_id"`
LastKillmailTime int64 `json:"last_killmail_time"`
}
Breakdown is one dimension's counters.
type BreakdownKey ¶
type BreakdownKey struct {
EntityType EntityType
EntityID int32
DimCategory DimCategory
DimID int32
}
BreakdownKey identifies a breakdown row within one period.
type CatchupResult ¶
type CatchupResult struct {
Days int `json:"days"`
Killmails int64 `json:"killmails"`
Deleted int64 `json:"deleted"`
Stats int64 `json:"stats"`
Breakdowns int64 `json:"breakdowns"`
}
CatchupResult reports what a catchup replaced.
func Catchup ¶
Catchup rebuilds the daily rows for [from, to).
One day at a time, each in its own transaction. A range that fails partway leaves the days it finished correct rather than rolling back the lot — re-running covers the rest, and the days already done are replaced again to the same values.
func CatchupTargets ¶
func CatchupTargets( ctx context.Context, pool *pgxpool.Pool, from, to time.Time, wantStats, wantBreakdowns bool, ) (CatchupResult, error)
CatchupTargets rebuilds either or both daily aggregate tables for [from, to). The selector exists for CLI parity with the TypeScript command, where operators can repair stats and breakdowns independently.
type Derived ¶
type Derived struct {
Efficiency int `json:"efficiency"`
IskEfficiency int `json:"isk_efficiency"`
SoloRatio int `json:"solo_ratio"`
SoloLossRatio int `json:"solo_loss_ratio"`
NPCLossRatio int `json:"npc_loss_ratio"`
BlobFactor float64 `json:"blob_factor"`
AvgDamagePerKill int64 `json:"avg_damage_per_kill"`
// DangerRatio is kills per loss. Zero losses with kills is reported as -1
// rather than an infinity, because JSON has no infinity and encoding one
// produces a document the frontend cannot parse.
DangerRatio float64 `json:"danger_ratio"`
}
Derived are the metrics computed from a Row rather than stored.
Kept here so every consumer renders the same efficiency and the same blob factor; two endpoints computing "efficiency" slightly differently is exactly the sort of thing nobody notices until the numbers are compared.
type DimCategory ¶
type DimCategory int16
DimCategory is what a breakdown row splits by.
The gaps are reserved rather than free. 20, 30 and 40–42 were character-level relationship dimensions that moved to Memgraph, where the graph shape fits better; rows may still exist for them, so the ids must not be reused for something else or old rows would be read as the new meaning.
const ( DimShipFlown DimCategory = 0 DimShipLost DimCategory = 1 DimSystem DimCategory = 10 DimConstellation DimCategory = 11 DimRegion DimCategory = 12 DimDiesToCorporation DimCategory = 21 DimDiesToAlliance DimCategory = 22 DimKilledCorporation DimCategory = 31 DimKilledAlliance DimCategory = 32 )
type EntityType ¶
type EntityType int16
EntityType is what a stats row is about.
const ( EntityCharacter EntityType = 0 EntityCorporation EntityType = 1 EntityAlliance EntityType = 2 EntityShip EntityType = 3 EntitySystem EntityType = 4 EntityConstellation EntityType = 5 EntityRegion EntityType = 6 EntityFaction EntityType = 7 )
func ParseEntityType ¶ added in v0.1.1
func ParseEntityType(value string) (EntityType, bool)
ParseEntityType resolves the stable name used by maintenance commands. "all" and an empty value deliberately mean no filter.
type Killmail ¶
type Killmail struct {
KillmailID int64
KillmailTime time.Time
SolarSystemID int32
ConstellationID int32
RegionID int32
VictimCharacterID int32
VictimCorporationID int32
VictimAllianceID int32
VictimFactionID int32
VictimShipTypeID int32
VictimDamageTaken int64
TotalValue float64
Points int64
AttackerCount int64
IsNPC bool
IsSolo bool
}
Killmail is what the accumulator needs about a kill.
Re-read from the database rather than carried on the job, so the job payload stays a single id and the handler is trivially replay-safe.
type LeaderboardMetric ¶
type LeaderboardMetric int16
LeaderboardMetric is what a leaderboard ranks by.
const ( MetricKills LeaderboardMetric = 0 MetricLosses LeaderboardMetric = 1 MetricIskDestroyed LeaderboardMetric = 2 MetricIskLost LeaderboardMetric = 3 MetricSoloKills LeaderboardMetric = 4 MetricPoints LeaderboardMetric = 5 MetricFinalBlows LeaderboardMetric = 6 )
type PeriodType ¶
type PeriodType int16
PeriodType is the granularity of a row.
Daily rows are the source of truth and everything else is rebuilt from them: monthly from daily, yearly from monthly. That is why a purge of old daily rows has to run after the rebuild rather than before it.
const ( PeriodDaily PeriodType = 0 PeriodMonthly PeriodType = 1 PeriodYearly PeriodType = 2 )
type PipelineResult ¶
type PipelineResult struct {
PurgedDaily int64 `json:"purged_daily"`
PurgedMonthly int64 `json:"purged_monthly"`
MonthlyStats int64 `json:"monthly_stats"`
MonthlyBreakdowns int64 `json:"monthly_breakdowns"`
YearlyStats int64 `json:"yearly_stats"`
YearlyBreakdowns int64 `json:"yearly_breakdowns"`
Leaderboards int64 `json:"leaderboards"`
}
PipelineResult reports what the nightly run did.
func RunPipeline ¶
RunPipeline rebuilds the rolled-up periods and the leaderboards.
type Row ¶
type Row struct {
Kills int64 `json:"kills"`
Losses int64 `json:"losses"`
SoloKills int64 `json:"solo_kills"`
SoloLosses int64 `json:"solo_losses"`
NPCLosses int64 `json:"npc_losses"`
FinalBlows int64 `json:"final_blows"`
Points int64 `json:"points"`
IskDestroyed float64 `json:"isk_destroyed"`
IskLost float64 `json:"isk_lost"`
DamageDealt int64 `json:"damage_dealt"`
DamageTaken int64 `json:"damage_taken"`
SumAttackerCount int64 `json:"sum_attacker_count"`
}
Row is the headline counters for one entity in one period.
type StatsKey ¶
type StatsKey struct {
EntityType EntityType
EntityID int32
}
StatsKey identifies a stats row within one period.
type WriteResult ¶
WriteResult reports what a merge wrote.
func Write ¶
func Write(ctx context.Context, pool *pgxpool.Pool, a *Accumulator, day time.Time) (WriteResult, error)
Write merges the accumulated counters for one day.
The period is always daily here. Monthly and yearly rows are rebuilt from these by the pipeline rather than written incrementally — incrementing three period types per counter would triple the write cost for aggregates nobody reads until the next day.
func WritePeriod ¶
func WritePeriod( ctx context.Context, pool *pgxpool.Pool, a *Accumulator, periodStart time.Time, period PeriodType, wantStats bool, wantBreakdowns bool, ) (WriteResult, error)
WritePeriod merges an accumulator into one explicit period. It is used by the historical backfill to write old months directly as monthly rows rather than creating years of daily rows that the retention job would immediately purge.
func WritePeriodTx ¶
func WritePeriodTx( ctx context.Context, tx pgx.Tx, a *Accumulator, periodStart time.Time, period PeriodType, wantStats bool, wantBreakdowns bool, ) (WriteResult, error)
WritePeriodTx is WritePeriod inside a caller-owned transaction.
The stats worker uses this so the additive counter updates and the killmail_processing ledger bit commit atomically. A River retry after the counters committed but before the bit did would otherwise count the same killmail twice.