killmail

package
v0.1.19 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 30, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package killmail turns an ESI killmail into the rows the killboard stores.

Index

Constants

AllEffects is every bit above.

View Source
const AllEffectsExceptWar = AllEffects &^ EffectWarInteractions

AllEffectsExceptWar marks a killmail that has been through the pipeline but has only just been associated with a war.

The war queue finds historical kills and attaches a war_id to them years after the fact. Everything else about such a killmail was aggregated long ago; only the war interaction is genuinely outstanding. Marking the rest complete is what stops the repair path re-running a full pipeline over a killmail whose only new fact is which war it belongs to.

View Source
const KillmailColumns = 26

KillmailColumns is the width of the killmails table, quoted in the compare output so "identical" states what was actually checked.

View Source
const NoParent int32 = -1

NoParent marks a top-level item. It is stored as NULL.

View Source
const SingletonBPC int16 = 2

SingletonBPC marks a blueprint copy. Copies carry no market value regardless of what the original sells for, so they are priced at the floor rather than looked up — otherwise a hauler full of copied battleship blueprints would read as a fortune.

Variables

View Source
var ErrNotStored = errors.New("killmail is not stored")

ErrNotStored is returned by Load for a killmail the database does not have.

View Source
var ErrWarConflict = errors.New("killmail already assigned to a different war")

ErrWarConflict means a killmail is already attached to a different war.

Functions

func AssignWars

func AssignWars(ctx context.Context, pool *pgxpool.Pool, batch []*Parsed) (int64, error)

AssignWars sets war_id on killmails that are already stored.

The bulk insert ignores conflicts, so a kill the live queue picked up before the war archive named it keeps a null war_id. Only a null is filled: a killmail already attributed to a war must not be moved to a different one, and the archives do occasionally disagree.

func BumpRollup

func BumpRollup(ctx context.Context, tx pgx.Tx, types []string, killmailTime string) error

BumpRollup adds this killmail to every daily subset it belongs to.

Rows are sorted into primary-key order before the insert. Two workers bumping overlapping subsets in different orders would deadlock, and at the rate killmails arrive that is a certainty rather than a risk.

func Delay

func Delay(ctx context.Context, pool *pgxpool.Pool, km Ref, hours int) error

Delay parks a killmail until now + hours.

When a row already exists the earlier of the two deadlines wins. Two sources can observe the same kill with different delays, and the shorter one is the one we are entitled to publish at — taking the later would hold back a kill somebody is already allowed to see.

func Delete

func Delete(ctx context.Context, pool *pgxpool.Pool, killmailID int64) error

Delete removes a killmail and its children, for reprocessing.

func Insert

func Insert(ctx context.Context, pool *pgxpool.Pool, p *Parsed) (bool, error)

Insert writes a killmail and everything hanging off it.

Returns false when the killmail already existed, which is not an error: two sources routinely deliver the same kill within seconds of each other, and the loser of that race has nothing left to do.

func InsertUntracked

func InsertUntracked(ctx context.Context, pool *pgxpool.Pool, p *Parsed) (bool, error)

InsertUntracked writes only the killmail and child rows, without creating a derived-effect ledger entry. It exists for debug:killmail, whose TypeScript implementation invokes selected effects inline with allowUntracked=true.

func IsComplete

func IsComplete(mask, effect Effect) bool

IsComplete reports whether an effect has already run.

func IsDelayed

func IsDelayed(ctx context.Context, pool *pgxpool.Pool, killmailID int64) (bool, error)

IsDelayed reports whether a killmail is currently held back.

func Points

func Points(cache *eve.Cache, km PointsInput) int32

Points is the zKillboard score for one killmail.

func RunDBEffect

func RunDBEffect(
	ctx context.Context,
	pool *pgxpool.Pool,
	killmailID int64,
	effect Effect,
	work func(ctx context.Context, tx pgx.Tx) (bool, error),
	opts EffectOptions,
) (bool, error)

RunDBEffect runs an effect that writes to Postgres, recording completion in the same transaction.

Committing the work and the bit together is what makes the guarantee hold: there is no window in which the counter has been incremented but the ledger does not say so, and therefore no way for a retry to increment it twice.

Returns true only when the work actually ran. A work function returning false means "not yet" — the effect stays pending and is retried later, which is how a killmail belonging to a war we have not imported waits for the war rather than being marked done without it.

func RunExternalEffect

func RunExternalEffect(
	ctx context.Context,
	pool *pgxpool.Pool,
	killmailID int64,
	effect Effect,
	work func(ctx context.Context) error,
) (bool, error)

RunExternalEffect runs an effect that cannot share a Postgres transaction — enqueueing a job, publishing to the relay, calling ESI.

The bit is set after the work succeeds, so a crash in between re-runs it. That is acceptable only because every such effect is addressed by a stable downstream id: the same killmail always produces the same job key, and River's uniqueness collapses the duplicate. An effect without that property does not belong here.

func Subject

func Subject(cache *eve.Cache, km Killmail) killtype.Subject

Subject builds the classifier input from a parsed killmail.

The security rating and the victim hull's meta group are not on the killmail row, so they come from the SDE cache. When the cache does not know the system — a newly added one, or a partial load — the region on the killmail row stands in, which is enough for the region-based subsets even though the security-based ones have to be skipped.

func UTCDateKey

func UTCDateKey(p Parsed) string

UTCDateKey is the calendar day a killmail counts towards.

UTC because the whole game runs on it: the client's "Today" header is EVE time, and a rollup keyed to anything else would disagree with it for part of every day.

func Undelay

func Undelay(ctx context.Context, pool *pgxpool.Pool, killmailIDs []int64) error

Undelay removes killmails from the delay table.

Called whenever a kill arrives from a source with no delay — the zKillboard feed, a manual post, an unrestricted key. Seeing it publicly means the delay no longer applies, whatever some other observation said.

func UpdateLastActive

func UpdateLastActive(ctx context.Context, tx pgx.Tx, p Parsed) error

UpdateLastActive moves each participating character's last_active forward.

It also refreshes security_status from the attacker rows, which is not a tangent: ESI reports it on every attacker, so taking it here costs nothing and saves a /characters/{id}/ call per ganker — the population whose security status changes most and who would otherwise be refetched constantly.

Only moves forward. Killmails arrive out of order and an old one must not drag a character's activity backwards.

Types

type Attacker

type Attacker struct {
	KillmailID int64 `json:"killmail_id"`

	// AttackerIndex is the position in the ESI attackers array. It is the only
	// stable identity an attacker has — mails carry no per-attacker ID — so it
	// doubles as half the primary key and must preserve ESI's ordering.
	AttackerIndex int32 `json:"attacker_index"`

	CharacterID   int32 `json:"character_id"`
	CorporationID int32 `json:"corporation_id"`
	AllianceID    int32 `json:"alliance_id"`
	FactionID     int32 `json:"faction_id"`
	ShipTypeID    int32 `json:"ship_type_id"`
	ShipGroupID   int32 `json:"ship_group_id"`
	WeaponTypeID  int32 `json:"weapon_type_id"`
	DamageDone    int32 `json:"damage_done"`
	FinalBlow     bool  `json:"final_blow"`

	// SecurityStatus is nullable and legitimately zero. See ESIAttacker.
	SecurityStatus *float64 `json:"security_status"`

	// KillmailTime is denormalised onto every attacker row so that "what did
	// this character kill in March" needs no join back to killmails.
	KillmailTime time.Time `json:"killmail_time"`
}

Attacker is one row of the killmail_attackers table.

type BatchResult

type BatchResult struct {
	Killmails int64
	Attackers int64
	Items     int64
}

BatchResult reports what a bulk insert wrote.

func InsertBatch

func InsertBatch(ctx context.Context, pool *pgxpool.Pool, batch []*Parsed) (BatchResult, error)

InsertBatch writes many parsed killmails at once.

Conflicts are ignored rather than updated: a killmail is immutable once stored, and the archives overlap heavily with what the live queue has already picked up. That makes re-running an import safe and cheap.

The three tables go in together under one transaction, so a failure never leaves attackers without their killmail.

Archive imports deliberately do not create killmail_processing rows. The TS importers treat historical rows as already imported data, not newly-arrived queue work. Creating a zeroed ledger here would make the entire archive look like a pending derived-effects backlog.

type Difference

type Difference struct {
	Field  string `json:"field"`
	Mine   string `json:"mine"`
	Theirs string `json:"theirs"`
}

Difference is one column that disagrees between two copies of a killmail.

func Diff

func Diff(mine, theirs *Parsed, tolerance float64) []Difference

Diff compares two copies of a killmail column by column.

tolerance is relative and applies only to the float columns — the ISK values. It exists for one specific case: hulls priced from custom_prices, which are regenerated from current market data and therefore move by a fraction of a percent between imports. Everything else compares exactly, because everything else is either an ID or a computation over IDs, and "close" would be wrong.

type ESIAttacker

type ESIAttacker struct {
	CharacterID   int32 `json:"character_id"`
	CorporationID int32 `json:"corporation_id"`
	AllianceID    int32 `json:"alliance_id"`
	FactionID     int32 `json:"faction_id"`
	ShipTypeID    int32 `json:"ship_type_id"`
	WeaponTypeID  int32 `json:"weapon_type_id"`
	DamageDone    int32 `json:"damage_done"`
	FinalBlow     bool  `json:"final_blow"`

	// SecurityStatus is a pointer because 0 is a real security status — a
	// freshly created character has exactly 0.0 — and only NPC attackers omit
	// the field entirely.
	SecurityStatus *float64 `json:"security_status"`
}

ESIAttacker is one participant on the kill.

type ESIItem

type ESIItem struct {
	ItemTypeID        int32     `json:"item_type_id"`
	Flag              int32     `json:"flag"`
	Singleton         int16     `json:"singleton"`
	QuantityDropped   int64     `json:"quantity_dropped"`
	QuantityDestroyed int64     `json:"quantity_destroyed"`
	Items             []ESIItem `json:"items"`
}

ESIItem is a fitted module, a cargo item, or the contents of either. Items nest one level: a container or a fitted launcher carries its own items array.

type ESIKillmail

type ESIKillmail struct {
	KillmailID    int64         `json:"killmail_id"`
	KillmailTime  time.Time     `json:"killmail_time"`
	SolarSystemID int32         `json:"solar_system_id"`
	Victim        ESIVictim     `json:"victim"`
	Attackers     []ESIAttacker `json:"attackers"`

	// WarID is absent from the public endpoint. The war queue supplies it out
	// of band, so the parser takes it as an argument rather than reading it
	// here. EVE Ref's war archives do carry it on the document, which is why
	// the field is decoded at all.
	WarID int32 `json:"war_id"`

	// KillmailHash is likewise absent from ESI's response — there the hash is
	// half the request — but present on every document in EVE Ref's archives,
	// which is the only way an archive import can store one.
	KillmailHash string `json:"killmail_hash"`
}

ESIKillmail is the response body of /killmails/{id}/{hash}/.

type ESIPosition

type ESIPosition struct {
	X float64 `json:"x"`
	Y float64 `json:"y"`
	Z float64 `json:"z"`
}

ESIPosition is where in the system the ship died. Absent on structure kills and on some older mails, so it is a pointer: the three coordinates are stored together or not at all.

type ESIVictim

type ESIVictim struct {
	CharacterID   int32        `json:"character_id"`
	CorporationID int32        `json:"corporation_id"`
	AllianceID    int32        `json:"alliance_id"`
	FactionID     int32        `json:"faction_id"`
	ShipTypeID    int32        `json:"ship_type_id"`
	DamageTaken   int32        `json:"damage_taken"`
	Position      *ESIPosition `json:"position"`
	Items         []ESIItem    `json:"items"`
}

ESIVictim is the loss half of a killmail.

type Effect

type Effect int32

Effect is one bit in the ledger.

const (
	EffectLastActive             Effect = 1 << 0
	EffectWarInteractions        Effect = 1 << 1
	EffectDailyKillRollup        Effect = 1 << 2
	EffectAchievementsDispatched Effect = 1 << 3
	EffectFitDispatched          Effect = 1 << 4
	EffectStatsDispatched        Effect = 1 << 5
	EffectGraphDispatched        Effect = 1 << 6
	EffectEntitiesEnsured        Effect = 1 << 7
	EffectEventPublished         Effect = 1 << 8
	EffectTickerEvaluated        Effect = 1 << 9
	EffectStatsWritten           Effect = 1 << 10
)

Effect bits.

These are a stored contract, not an internal enum: production rows hold these values indefinitely and the TypeScript backend reads the same column with the same meanings. A bit may be retired but must never be reused for a different effect — doing so would silently mark old killmails as having completed work they have never seen.

type EffectOptions

type EffectOptions struct {
	// AllowUntracked runs the work even when no ledger row exists.
	//
	// Without a row there is nothing to record completion against, so the work
	// cannot be replay-protected — only for effects that are naturally
	// idempotent.
	AllowUntracked bool

	// BeforeLedgerLock runs inside the transaction before the ledger row is
	// locked. The war aggregation uses it to take a shared advisory lock, so
	// that a concurrent atomic rebuild cannot swap the table out from under an
	// in-flight increment.
	BeforeLedgerLock func(ctx context.Context, tx pgx.Tx) error
}

EffectOptions tunes how one effect runs.

type Item

type Item struct {
	KillmailID int64 `json:"killmail_id"`

	// ItemIndex is assigned by the flattening walk, not by ESI. It is depth-first
	// in ESI order, which is what makes ParentIndex meaningful.
	ItemIndex int32 `json:"item_index"`

	TypeID            int32 `json:"type_id"`
	FlagID            int32 `json:"flag_id"`
	QuantityDropped   int64 `json:"quantity_dropped"`
	QuantityDestroyed int64 `json:"quantity_destroyed"`
	Singleton         int16 `json:"singleton"`

	// ParentIndex is the ItemIndex of the container or fitted module holding
	// this item, and NoParent for anything at the top level. Zero cannot mean
	// "none" here: item 0 is a real item and is frequently a parent.
	ParentIndex int32 `json:"parent_index"`
}

Item is one row of the killmail_items table.

type Killmail

type Killmail struct {
	KillmailID   int64     `json:"killmail_id"`
	KillmailTime time.Time `json:"killmail_time"`
	KillmailHash string    `json:"killmail_hash"`

	SolarSystemID   int32 `json:"solar_system_id"`
	ConstellationID int32 `json:"constellation_id"`
	RegionID        int32 `json:"region_id"`

	// Position is stored as three columns but is present or absent as a unit.
	Position *ESIPosition `json:"position"`

	VictimCharacterID   int32 `json:"victim_character_id"`
	VictimCorporationID int32 `json:"victim_corporation_id"`
	VictimAllianceID    int32 `json:"victim_alliance_id"`
	VictimFactionID     int32 `json:"victim_faction_id"`
	VictimShipTypeID    int32 `json:"victim_ship_type_id"`
	VictimShipGroupID   int32 `json:"victim_ship_group_id"`
	VictimDamageTaken   int32 `json:"victim_damage_taken"`

	TotalValue     float64 `json:"total_value"`
	FittedValue    float64 `json:"fitted_value"`
	DroppedValue   float64 `json:"dropped_value"`
	DestroyedValue float64 `json:"destroyed_value"`

	Points        int32 `json:"points"`
	AttackerCount int32 `json:"attacker_count"`
	IsNPC         bool  `json:"is_npc"`
	IsSolo        bool  `json:"is_solo"`

	WarID int32 `json:"war_id"`

	// Blob marks a mail whose item list was too large to store inline. Nothing
	// sets it yet; it exists so the row matches the table.
	Blob bool `json:"blob"`
}

Killmail is one row of the killmails table.

The JSON names are the column names, not Go-idiomatic ones: this shape is diffed against the TypeScript parser's output during the port, and later serves the API, where the column name is what a consumer expects.

type Parsed

type Parsed struct {
	Killmail  Killmail
	Attackers []Attacker
	Items     []Item
}

Parsed is the complete set of rows one killmail produces.

func Load

func Load(ctx context.Context, pool *pgxpool.Pool, killmailID int64) (*Parsed, error)

Load reads a killmail back out of the database in the same shape the parser produces, so a stored mail and a freshly parsed one can be compared field by field.

func Parse

func Parse(
	ctx context.Context,
	cache *eve.Cache,
	prices *eve.Prices,
	km *ESIKillmail,
	hash string,
	warID int32,
) (*Parsed, error)

Parse turns an ESI killmail into the rows the killboard stores.

It performs exactly one database query — the batched price lookup — and does everything else from the in-memory cache.

type PointsAttacker

type PointsAttacker struct {
	CharacterID int32
	ShipTypeID  int32
}

PointsAttacker is one attacker, as the scorer reads it.

type PointsInput

type PointsInput struct {
	VictimShipTypeID int32

	// Items are the victim's top-level items only, never the flattened tree.
	// Modules inside containers are cargo, not a fit, and counting them would
	// let a hauler full of spare guns score as a brawler.
	Items     []PointsItem
	Attackers []PointsAttacker
}

PointsInput is everything the score depends on.

Neutral between the two sources that produce it: a freshly fetched ESI document during ingest, and the stored rows during a backfill. They must score identically — a backfill that disagreed with ingest would rewrite history to a different number than the live path would have produced — so there is one implementation and two thin adapters rather than two copies.

type PointsItem

type PointsItem struct {
	TypeID            int32
	Flag              int32
	QuantityDropped   int64
	QuantityDestroyed int64
}

PointsItem is one fitted item, as the scorer reads it.

type Ref

type Ref struct {
	KillmailID   int64  `json:"killmail_id"`
	KillmailHash string `json:"killmail_hash"`
}

Ref is the id and hash pair needed to fetch a killmail.

func ClaimExpired

func ClaimExpired(ctx context.Context, pool *pgxpool.Pool, limit int) ([]Ref, error)

ClaimExpired atomically takes up to limit killmails whose delay has passed.

One DELETE ... RETURNING rather than a SELECT followed by a DELETE, so two concurrent runs cannot both claim the same rows and dispatch the killmail twice. The ordering is oldest first, so a backlog drains in the order the delays actually expired.

type Tracked

type Tracked struct {
	Exists    bool
	Tracked   bool
	Completed Effect
	WarID     int32
}

Tracked is what the pipeline needs to know about a killmail already stored.

func Prepare

func Prepare(ctx context.Context, pool *pgxpool.Pool, killmailID int64, requestedWarID int32) (Tracked, error)

Prepare inspects a stored killmail and, when a war id is offered, attaches it.

The war attachment is the interesting half. A killmail that has been in the database for years can be discovered to belong to a war, and when that happens exactly one effect becomes outstanding again. Clearing that single bit — rather than reprocessing — is what keeps the war repair path cheap enough to run over a backlog.

func (Tracked) Done

func (t Tracked) Done() bool

Done reports a killmail with nothing left to do.

func (Tracked) Pending

func (t Tracked) Pending() Effect

Pending is the set of effects still outstanding.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL