Documentation
¶
Overview ¶
Package indexlanethroughput measures how a durable index build scales as its crawl is divided across journal-writing lanes.
The measurement exists to answer one question with evidence rather than argument: does giving each lane its own journal actually raise ingest throughput, and does it leave a listing-bound crawl alone? Both halves matter. A writer-bound case that does not improve means the lanes are not worth their complexity; a listing-bound case that changes means lanes are taking credit for prefix fan-out that already existed.
Figures produced here are machine-local and directional. They are not a product benchmark and must not be quoted as one.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Corpus ¶
type Corpus struct {
Prefixes int
ObjectsPerPrefix int
PageSize int
// PageLatency is the simulated per-LIST-page round trip. Zero produces the
// writer-bound case; a large value produces the listing-bound control.
PageLatency time.Duration
}
Corpus describes the synthetic estate a run crawls.
type Instrument ¶
type Instrument struct {
// contains filtered or unexported fields
}
Instrument records aggregate provider pressure and lane activity for one run.
Every counter here is deliberately aggregate rather than per-lane: the ceilings a shared request budget promises are run-global, and a per-lane reading would look correct even when N lanes together breach them.
func (*Instrument) CrawlWall ¶
func (in *Instrument) CrawlWall() time.Duration
CrawlWall is the crawl phase measured directly by this harness: from the first provider listing to the last observation delivered. It does not depend on the engine reporting its own duration.
func (*Instrument) ListStarts ¶
func (in *Instrument) ListStarts() int
ListStarts is the total number of provider listing requests issued.
func (*Instrument) PeakConcurrentLanes ¶
func (in *Instrument) PeakConcurrentLanes() int
PeakConcurrentLanes reports the largest number of lanes whose delivery windows overlapped at one instant.
Headline throughput alone cannot distinguish a genuinely parallel run from a scheduler that happened to interleave lanes; this can. A value of 1 means the lanes ran one after another whatever the wall clock says.
func (*Instrument) PeakListConcurrency ¶
func (in *Instrument) PeakListConcurrency() int
PeakListConcurrency is the highest number of simultaneous provider listings observed across every lane.
func (*Instrument) RequestWindow ¶
func (in *Instrument) RequestWindow() time.Duration
RequestWindow is the interval a sustained-rate ceiling applies to: from before the first request could have been admitted, to the last one observed.
It starts at the run rather than at the first observed listing on purpose. A listing is timestamped on provider entry, which is after the limiter admitted it, so scheduler delay after that first admission would shrink the window and report a breach the limiter did not commit.
type RunResult ¶
type RunResult struct {
Spec RunSpec
JournalsSealed int
ObjectsObserved int64
CrawlDuration time.Duration
TotalDuration time.Duration
PeakListConcurrency int
PeakConcurrentLanes int
ListStarts int
RequestWindow time.Duration
}
RunResult is what one cell reports.
func (RunResult) CrawlObjectsPerSec ¶
CrawlObjectsPerSec is the throughput the lane feature is meant to move. It is deliberately the crawl phase alone: publication runs after every lane is terminal and is unaffected by how many lanes produced the journals.
The duration is this harness's own measurement of the crawl interval, not a figure the engine reported about itself.
Source Files