entities

package
v0.1.78 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 2, 2026 License: MIT Imports: 8 Imported by: 0

Documentation

Overview

Package entities turns ESI responses into the character, corporation and alliance rows the killboard stores.

A killmail names entities by id and nothing else. Everything a page shows — names, tickers, corp membership, alliance history — comes from fetching those ids separately and keeping the result. This package is that half: fetch, persist, and report what else needs fetching as a result.

It deliberately does not enqueue anything. A refresh returns the follow-up work it implies and leaves dispatching to the caller, so the same code serves a CLI walking the cascade synchronously and, later, a queue driving it.

Index

Constants

View Source
const PlayerCorporationIDMin int32 = 10_000_000

PlayerCorporationIDMin is where player corporations start. Below it are NPC corporations, which every new character belongs to and which no amount of fetching will make interesting — refreshing them would spend the entire corporation budget on the same few hundred ids.

View Source
const StaleAfter = 7 * 24 * time.Hour

StaleAfter is how long an entity stays usable before it is refetched. Names and tickers change rarely; a week is a compromise between a stale killboard and spending the whole ESI budget re-reading things that did not change.

Variables

View Source
var ErrTransient = errors.New("ESI request failed")

ErrTransient marks a failure worth retrying — ESI was unreachable or unwell, as opposed to answering that the entity does not exist.

Functions

func HighestCharacterID

func HighestCharacterID(ctx context.Context, pool *pgxpool.Pool) (int32, error)

HighestCharacterID is the top of the known id space, which is where a trailing scan starts.

func IsPlayerCorporation

func IsPlayerCorporation(id int32) bool

IsPlayerCorporation reports whether an id is worth fetching.

Types

type Affiliation

type Affiliation struct {
	CharacterID   int32
	CorporationID int32
	AllianceID    int32
}

Affiliation is what a killmail asserts about one character at kill time.

type Cascade

type Cascade struct {
	Characters           []int32 `json:"characters,omitempty"`
	Corporations         []int32 `json:"corporations,omitempty"`
	Alliances            []int32 `json:"alliances,omitempty"`
	CharacterHistories   []int32 `json:"character_histories,omitempty"`
	CorporationHistories []int32 `json:"corporation_histories,omitempty"`
}

Cascade is the follow-up work a refresh implies.

Returned rather than enqueued so this package has no opinion about how work is scheduled. Everything here is already deduplicated and filtered — an id in this struct is one the caller should actually fetch.

func Stale

func Stale(ctx context.Context, pool *pgxpool.Pool, ref Referenced) (Cascade, error)

Stale reports which of the referenced entities need fetching.

Three queries regardless of how many entities the killmail names, because a large fight names thousands and per-entity checks would cost more than the fetches they save.

func (Cascade) Empty

func (c Cascade) Empty() bool

Empty reports whether there is nothing left to do.

type CascadeTotal

type CascadeTotal struct {
	Characters int `json:"characters,omitempty"`
	Histories  int `json:"histories,omitempty"`
}

CascadeTotal counts the follow-up work discovery implied.

type HoleOptions

type HoleOptions struct {
	// From and To bound which known ids are walked. To of zero means no bound.
	From, To int32

	// MaxGap skips gaps wider than this.
	//
	// The very large gaps are CCP's unallocated allocation blocks — tens of
	// millions of ids with nobody in them. Walking one exhaustively would spend
	// weeks of error budget to find nothing, so they are left alone.
	MaxGap int

	// ProbeBlock is how many consecutive ids are tried at each step, and how far
	// a step advances when any of them hit.
	ProbeBlock int

	// Skip is how far to jump when a whole block missed. Larger than
	// ProbeBlock: an empty block is evidence the neighbourhood is empty, and
	// stepping one block at a time through a million-id void is what makes an
	// exhaustive scan infeasible.
	Skip int

	// MaxMisses ends the run.
	MaxMisses int
}

HoleOptions tunes a gap scan.

type HoleResult

type HoleResult struct {
	GapsScanned int          `json:"gaps_scanned"`
	GapsSkipped int          `json:"gaps_skipped"`
	Probed      int          `json:"probed"`
	Hits        int          `json:"hits"`
	Misses      int          `json:"misses"`
	LastID      int32        `json:"last_id"`
	Stopped     StopReason   `json:"stopped,omitempty"`
	Cascade     CascadeTotal `json:"cascade,omitzero"`
}

HoleResult reports a scan of the gaps between known ids.

type Probe

type Probe struct {
	ID      int32
	Outcome ProbeOutcome
	Name    string
}

Probe reports one id's result to a caller that wants to log it.

type ProbeOutcome

type ProbeOutcome int

ProbeOutcome is what one probe found.

const (
	// ProbeHit means a character exists at that id.
	ProbeHit ProbeOutcome = iota
	// ProbeMiss means the id is unallocated or biomassed.
	ProbeMiss
	// ProbeError means ESI could not answer, which is not evidence either way.
	ProbeError
)

type Prober

type Prober struct {
	Pool *pgxpool.Pool
	ESI  *esi.Client

	// DryRun probes without storing anything. The ESI calls still happen — a
	// dry run that skipped them would report nothing useful — so it still
	// spends the error budget.
	DryRun bool

	// OnProbe, when set, is called for every id attempted.
	OnProbe func(Probe)

	// OnCascade dispatches the history and corporation work implied by a hit.
	// The entity package deliberately does not know which queue implementation
	// the caller uses; scan commands provide the River adapter here.
	OnCascade func(context.Context, Cascade) error
}

Prober probes character ids and ingests what it finds.

func (*Prober) ScanHoles

func (p *Prober) ScanHoles(ctx context.Context, opts HoleOptions) (HoleResult, error)

ScanHoles walks the gaps between known character ids.

Sampling rather than exhaustive: each step probes a small block and then either advances by that block, if anything was there, or jumps ahead. The assumption is that allocated ids cluster, so a hit means keep looking here and a miss means look elsewhere. It will not find every character in a gap, and it is not meant to — it is meant to find the populated regions of a space too large to enumerate.

func (*Prober) ScanTrailing

func (p *Prober) ScanTrailing(ctx context.Context, start int32, ahead, maxMisses int, keepGoing bool) (TrailingResult, error)

ScanTrailing walks forward from an id looking for newly allocated characters.

Stops on cumulative misses rather than consecutive ones. CCP does not allocate ids densely, so a run of gaps inside live territory is normal and stopping at the first would give up immediately; a high cumulative count is the signal that the allocated range has genuinely been left behind.

type Referenced

type Referenced struct {
	Affiliations []Affiliation
	Corporations []int32
	Alliances    []int32
	// KillmailTime is when the kill happened, which decides whether the
	// killmail's view of an affiliation is newer than the stored one.
	KillmailTime time.Time
}

Referenced is every entity a killmail mentions.

type Refresher

type Refresher struct {
	Pool *pgxpool.Pool
	ESI  *esi.Client
}

Refresher fetches entities and writes them.

func (*Refresher) Alliance

func (r *Refresher) Alliance(ctx context.Context, id int32) (Result, error)

Alliance fetches an alliance and stores it.

func (*Refresher) Character

func (r *Refresher) Character(ctx context.Context, id int32) (Result, error)

Character fetches a character and stores it.

A 404 or 400 means biomassed or never existed. That is recorded as deleted rather than left absent, so the id is not retried forever — a killboard accumulates references to dead characters indefinitely.

func (*Refresher) CharacterHistory

func (r *Refresher) CharacterHistory(ctx context.Context, id int32, force bool) (Result, error)

CharacterHistory fetches and stores a character's corporation history.

force overrides the sync marker, which otherwise suppresses a fetch whose answer is already known.

func (*Refresher) Corporation

func (r *Refresher) Corporation(ctx context.Context, id int32) (Result, error)

Corporation fetches a corporation and stores it.

func (*Refresher) CorporationHistory

func (r *Refresher) CorporationHistory(ctx context.Context, id int32, force bool) (Result, error)

CorporationHistory fetches and stores a corporation's alliance history.

type Result

type Result struct {
	ID      int32   `json:"id"`
	Kind    string  `json:"kind"`
	Name    string  `json:"name,omitempty"`
	Status  int     `json:"status"`
	Deleted bool    `json:"deleted,omitempty"`
	Rows    int64   `json:"rows,omitempty"`
	Cascade Cascade `json:"cascade,omitzero"`
}

Result reports one refresh.

type StopReason

type StopReason string

StopReason says why a scan ended.

const (
	StopMaxMisses StopReason = "max-misses"
	StopReached   StopReason = "range-covered"
	StopESIError  StopReason = "esi-error"
)

type TrailingResult

type TrailingResult struct {
	Probed  int          `json:"probed"`
	Hits    int          `json:"hits"`
	Misses  int          `json:"misses"`
	NewIDs  []int32      `json:"new_ids,omitempty"`
	LastID  int32        `json:"last_id"`
	Stopped StopReason   `json:"stopped"`
	Cascade CascadeTotal `json:"cascade,omitzero"`
}

TrailingResult reports a scan above the highest known id.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL