ledger

package
v0.0.0-...-ea53456 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 10, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Overview

Package ledger records what a run cost: a tree of spans per run, written by one collector goroutine into a database of its own beside the index store, and readable by any other process while the run is still going.

It imports internal/model and the pinned engine, and nothing else of the product. The store, the coordinator and the providers open spans, so this package can never import them.

Index

Constants

View Source 
const (
	// CPUAttributed is the value of a span whose CPU columns are populated.
	CPUAttributed = ""
	// CPUOverlapped is in-process work that ran beside other goroutines. The
	// process-wide counters measure the process, and attributing them to one
	// stage would be a guess; a null is the honest answer.
	CPUOverlapped = "overlapped"
	// CPUUnsampled is a platform that does not expose the counters at all.
	CPUUnsampled = "unsampled"
)

Why a span carries no CPU figures. The empty value means the figures are there; the others name what the process could not measure, so a reader never has to guess whether a null is a zero.

View Source 
const FileName = "ledger.db"

FileName is the ledger database's name beside the index store's own file in the workspace directory. It lives under the same directory lock and is deleted with the same cache.

View Source 
const FootprintBytes int64 = (writerConnections + readerHandles*readerConnections) * cacheKiB << 10

FootprintBytes is the memory the run ledger's page caches can hold in one process: the writer's connection and every connection of every reader it can hold open at once. The process's base footprint counts it (config.BaseFootprint), because nothing else reserves it.

View Source 
const LiveWindow = 120 * flushInterval

LiveWindow is how far ahead of itself a run's collector stamps the liveness deadline it publishes on the run row, and so how long after a process dies its run still reads as live. It is a multiple of the flush interval rather than a duration chosen on its own: the collector renews the stamp on the flush it already performs, and the window has only to cover the longest a flush can legitimately be late -- a write waiting up to busyTimeout for the single writer that retention also takes, and a subscriber running on the collector's own goroutine. A hundred and twenty intervals is thirty seconds, six times that wait.

It is exported because it is also the answer to the question a process waiting for the workspace lock asks: how long a holder may legitimately show no fresh stamp before it is treated as no longer progressing. That waiter reuses this window rather than choosing a second number, because the two questions are the same question.

View Source 
const ReasonInterrupted = "the span was still running when its run ended"

ReasonInterrupted is the reason of a span that was still running when its run ended: nobody measured how it finished, because it did not. A span with no reason at all would be read as a failure whose cause was never recorded, which is the one thing this account exists to replace -- so the cut-off says so in the same two columns every other non-ok row uses.

View Source 
const ReasonNotAdmitted = "the unit was planned but never reached admission"

ReasonNotAdmitted is the reason of a unit whose span was still 'planned' when its run ended: nothing ever started it, so it was never admitted to the work its plan named. It is the ledger's own reason rather than a provider's, because no provider was ever reached to state one.

Variables

View Source 
var Fingerprint = func() string {
	sum := sha256.Sum256([]byte(schemaSQL))
	return hex.EncodeToString(sum[:])
}()

Fingerprint is the SHA-256 of the embedded DDL. It is this file's own and is deliberately not folded into anything the index store keys on: a change here must never re-key an analysis unit.

Functions

func Path 

func Path(dir string) string

Path is the ledger database beside the index store's own file.

Types

type Kind 

type Kind string

A Kind is what opened a run: an index run, the deferred publication of a generation whose run had already ended, or the per-process overlay a language server's startup hangs under when no run is live. 

const (
	KindIndex    Kind = "index"
	KindDeferred Kind = "deferred"
	KindOverlay  Kind = "overlay"
)

type Ledger 

type Ledger struct {
	// contains filtered or unexported fields
}

A Ledger is a process's stable handle on the run ledger. It is composed once and lives as long as the workspace; what comes and goes underneath it is its COLLECTOR -- the database, the bus and the one goroutine that writes -- which ATTACHES when this process takes the cross-process workspace lock for a building operation and DETACHES when that hold ends.

The split is the one-writer-per-ledger invariant made structural. A process that answers questions and occasionally builds -- a server -- would otherwise hold a writer on this file for its whole session while holding the workspace for the beat that builds, and the person's own index beside it would be a second writer on a file whose separate-database design exists so there is only ever one.

A DETACHED handle records nothing, exactly as a nil *Ledger does: NewRun returns a nil run, published events are dropped, and every query answers empty. That is what lets the surfaces that are handed this handle at composition -- the language-server manager's overlay run among them -- be instrumented unconditionally and degrade to recording nothing rather than writing without the lock.

A nil *Ledger is a valid ledger that records nothing, exactly as a nil *Span is a valid span that records nothing: it opens nil runs, and every method on it does nothing and reports no failure. This is what lets a composition without a ledger -- a test of a package that records, or a process where opening the file failed -- be instrumented at all. The alternative is a guard at every one of the dozens of stage sites, where the first one anybody forgets panics the run it was meant to measure.

func New 

func New(dir string) *Ledger

New composes the handle on the run ledger in dir -- the directory the index store lives in, under the same lock. It opens NOTHING: the file is opened, and the collector that writes it started, only by Attach.

func (*Ledger) Attach 

func (l *Ledger) Attach(ctx context.Context) error

Attach opens the ledger file for writing and starts its collector. The caller must already hold the cross-process workspace lock for this workspace, and must Detach before it gives that lock up.

A handle that already has a collector, or one that has been stopped, is a composition defect rather than a condition to recover from: the first is two writers in one process and the second is a hold taken after Close.

func (*Ledger) DeleteOverlayRuns 

func (l *Ledger) DeleteOverlayRuns(ctx context.Context, runIDs []string) error

DeleteOverlayRuns removes the overlay runs of processes that are no longer running: an overlay run has no generation, so nothing else would ever collect it. A process that exits cleanly removes its own through DiscardRun, which is ordered against its own writer; this is the collection pass's call for the ones that did not.

func (*Ledger) Detach 

func (l *Ledger) Detach() error

Detach finishes this handle's collector and closes the ledger file: it drains what is buffered, flushes it, writes every span still open as interrupted and every run's finish, and closes the pool. The handle survives and records nothing until the next Attach.

It is the LAST thing a hold does before it releases the workspace lock. Release the lock first and the waiter acquiring on its next poll opens its collector while this one is still flushing -- two writers on the file again, which is the whole defect this split exists to remove.

func (*Ledger) DiscardRun 

func (l *Ledger) DiscardRun(ctx context.Context, run *Run) error

DiscardRun deletes a run and everything recorded under it, and stops this process recording it at all. It is what a run that exists only for the life of one process does at clean exit: the per-process overlay a language server's start hangs under belongs to no generation, so the retention that deletes a generation's runs would never reach it and the file would keep one run and its spans per process for ever.

The run is forgotten BEFORE the delete, and forgotten for good: an event of its still on the bus, and the interrupted-marking pass a stopping ledger makes over every run it knows, would otherwise write the row back in behind the delete. The delete itself takes the writer, so a flush already in progress finishes first and this sees the row it wrote.

func (*Ledger) Flush 

func (l *Ledger) Flush(ctx context.Context) error

Flush returns once the collector has written everything published before the call: the spans that ended, the run rows whose totals or outcome moved, and the rows a finished run left planned, which the collector closes in batches and this waits out in full.

It exists because finishing a run moves in-memory state, and the row turns terminal only at the next flush: a read straight after a run returns would otherwise render a finished run as still running, with its last stages missing. It is a barrier and not a wait: nothing is slept on and nothing is polled, the collector answers the event when the write it was already going to do has committed.

Nothing a run records goes through here, so the bus keeps its non-blocking discipline: the barrier's own send waits, because its caller is a reader that asked to wait, while every producer's publish still drops rather than block. A ledger that records nothing, and one whose collector has already stopped and therefore written everything it had, return at once.

func (*Ledger) IndexRunID 

func (l *Ledger) IndexRunID() (string, bool)

IndexRunID is the identifier of the indexing run this process is recording at this moment, and false when it is recording none. It exists because a process records more than one run at a time -- a language server's start hangs under a per-process overlay run that can be open while an index run is going -- so a surface that follows an indexing run has to be able to say which of them a span it is handed came from.

Runs are held in the order they opened and a process indexes under one cross-process owner, so the last one still running is the one a caller is asking about. A run that has finished is not it: its rows are complete, and answering with it would hand a later caller a run that is over.

func (*Ledger) NewRun 

func (l *Ledger) NewRun(kind Kind, repositoryID string) (*Run, error)

NewRun records a new run and returns its handle. The run id is 32 random bytes, allocated before any generation exists, so a run that fails before publication still has a complete ledger. A ledger that records nothing -- nil, or a handle no hold has attached a collector to -- opens a run that records nothing, and validates nothing, because there is nothing to record.

The run belongs to the attachment that was current when it opened. When that attachment detaches the run is finalized with it and records nothing further, so a caller that cached one across a hold degrades to the same no-op a composition with no ledger already gets.

func (*Ledger) OverlayRuns 

func (l *Ledger) OverlayRuns(ctx context.Context) ([]string, error)

OverlayRuns lists the overlay runs whose writer is gone: an overlay run that finished, and one whose process died without stopping its ledger and whose published deadline has since elapsed. A live overlay run -- another process serving a language server right now -- is never listed, so the caller can hand everything it gets straight to DeleteOverlayRuns.

func (*Ledger) RetirePeaks 

func (l *Ledger) RetirePeaks(ctx context.Context, repositoryID string, keep []string) error

RetirePeaks deletes this repository's high-water rows whose scope keys are not in keep, which is what bounds that table by what the repository holds instead of by a count: keep is every heavy scope a plan just derived for the repository, in ascending order, so a scope that no longer exists -- a removed project, a renamed root, a subdivision part or any other key a span measured that no plan asks about -- leaves the table at the next plan, and every scope that still exists keeps its measurement however many runs have passed since one last touched it.

The caller calls it only with a plan that derived the complete heavy scope set; a plan that did not -- the provider that plans heavy units was not active -- names nothing, and retiring against it would forget every measurement a later plan will need. The table is read in pages of model.MaxRecordsPerResult, one write transaction per page, so no statement carries a repository-sized parameter list and the writer is never held for longer than one page.

func (*Ledger) Stop 

func (l *Ledger) Stop() error

Stop retires the handle for good: it detaches whatever collector is still attached -- draining, flushing, writing every span still open as interrupted and every run's finish, and closing the database -- and refuses any further attach.

It is what the composition's Close calls, and it is the backstop for a hold that ended without detaching: a collector a leaked hold left open still holds a writer on this file, and the process must not exit leaving it there. It is safe to call more than once and returns the first failure.

func (*Ledger) Subscribe 

func (l *Ledger) Subscribe(fn func(SpanRow))

Subscribe registers a function the collector calls with every span that finishes, in the order the collector writes them. It is the one place a finished span is published: the log line, the progress notification and the per-span message all come from here, never from the caller's own goroutine, so every surface sees the same event once.

The function runs on the collector goroutine. It must not block: a subscriber that waits delays every later flush, and with it every live reader's view.

func (*Ledger) SweepRuns 

func (l *Ledger) SweepRuns(ctx context.Context, repositoryID string,
	retained func(ctx context.Context, generationID int64) (bool, error)) error

SweepRuns deletes the runs no retained generation stands behind, which is the whole of the run history's collection. The history follows what the store keeps rather than a count: the cutoff is the start of the oldest run whose generation retained still reports held, a run that started at or after it is kept whatever it produced, and every run that started before it is deleted -- a run whose generation retention has since swept, a tick that published nothing, and a run whose process died before it reached a generation all leave the file the same way. So the run behind each retained generation keeps its account for as long as the store keeps what it built, and so does every run since the oldest of them, including the failures an operator is investigating.

The window is the store's, so it shares the store's consequences. An index run whose generation a deferred publication supersedes on the same ref is deleted once retention sweeps that generation, unless the run behind an older retained ref precedes it: the deferred run is then what built the active store, and it is the run `status --resources` shows.

When no run in the file names a retained generation, nothing is deleted. That is the first run of a workspace before the generation it published has been flushed to its row, and a file recreated after the store was built: in both, every run is newer than the store's next cutoff, and deleting them would erase the account of runs the next pass keeps. It cannot grow without bound, because the sweep runs after a publication, and the run that published names the active generation from its next flush on.

Two classes of row are never deleted:

  • a live run -- one that says 'running' and whose writer has renewed the deadline it published. That is the run this pass is part of, and it is also another process's run in flight.
  • an overlay run, which DeleteOverlayRuns collects by its own writer's liveness. Overlay runs are per-process and belong to no generation.

retained answers whether the store still holds a generation as a published result, which is the caller's to know. The caller holds the lock every publication takes, so no run older than the cutoff can publish a retained generation between the scan and the delete. It is asked about each run before the cutoff that names a generation, and about the cutoff's own run, so a pass asks about the runs it and the passes after it delete and one more. The scan reads model.MaxRecordsPerResult rows at a time, and one pass deletes at most that many runs; it is called from the periodic collection, so the next pass takes the rest.

type MarkedUnit 

type MarkedUnit struct {
	ScopeKey       string
	RunID          string
	DiagnosticCode string
	Failure        string
}

MarkedUnit is one named unit and the marker that names its run: the run's id and the marker's diagnostic code and failure text.

type MarkedUnits 

type MarkedUnits struct {
	Provider string
	Units    int64
	Named    []MarkedUnit
}

MarkedUnits is one provider's unit spans in the runs a marker names: how many there are, and the first of them by scope key. A unit span is a top-level span whose stage is its provider's id, which is how a run records every unit it planned.

type Measured 

type Measured struct {
	CPUUserMS *int64
	CPUSysMS  *int64
	// CPUUnattributed names why CPUUserMS and CPUSysMS are nil. It must be
	// empty when they are set and one of CPUOverlapped or CPUUnsampled when
	// they are not.
	CPUUnattributed string
	PeakRSSBytes    *uint64
	ReadBytes       *uint64
	WriteBytes      *uint64
	// ItemsIn and ItemsOut override the span's own counters for a stage that
	// only knows its totals at the end. A stage that counted as it went leaves
	// them nil and the counters stand.
	ItemsIn  *int64
	ItemsOut *int64
	// DiagnosticCode and Failure carry a failed span's typed error. They are
	// filled from the error End is given and a caller normally leaves them
	// empty.
	DiagnosticCode string
	Failure        string
}

Measured is what a stage knows about its own cost when it ends. Every field is a pointer because the platform cannot give all of them everywhere: a field it cannot give is absent, never zero, so a reader is never shown a measurement nobody made.

The child-process fields are filled by whatever ran the child from the accounting it already holds -- the resource usage at exit, the tree peak the sampler observed, the byte counters read before exit. An in-process stage leaves them nil.

type NeedClass 

type NeedClass struct {
	Key           NeedKey
	Observations  int64
	Overruns      int64
	MaxDriftBytes int64
}

A NeedClass is one learned need model of a repository as a reader sees it: how many files it was learned from, how many of them used more than they were reserved, and the largest drift -- need minus reservation -- any of them showed, which is below zero when every file fitted.

type NeedKey 

type NeedKey struct {
	Language    string
	Fingerprint string
	Build       string
	SizeClass   int
}

NeedKey names one learned need model within a repository: the language, the grammar fingerprint the files were parsed under, the build identity of the worker that parsed them, and the file-size class. A worker of a new build learns afresh: what it holds of a file's tree beside the parse -- the flattened array, the lowering's structures -- changes with the build, and a model learned under another would reserve for work the file no longer does.

type NeedObservation 

type NeedObservation struct {
	Key NeedKey
	// ReservedBytes is the per-file increment the file was admitted with, and
	// NeedBytes what the worker measured it to use above its base.
	ReservedBytes int64
	NeedBytes     int64
	State         []byte
}

NeedObservation is one parsed file's measured need, with the model state after it was folded in. State is the encoded model, opaque to this package.

type Outcome 

type Outcome string

An Outcome is what became of a run or a span. 'planned' is a unit that has a row before anything ran it, so a unit that never reaches its work still exists; 'running' is the value a span carries between its start and its end; 'interrupted' is what a span that was still running when its run ended is left as, which is neither a success nor a recorded failure but a measurement that was cut off; 'unavailable' is work that reached no output because something it needed was absent, and it carries the reason. 

const (
	OutcomePlanned     Outcome = "planned"
	OutcomeRunning     Outcome = "running"
	OutcomeOK          Outcome = "ok"
	OutcomeFailed      Outcome = "failed"
	OutcomeSubdivided  Outcome = "subdivided"
	OutcomeReused      Outcome = "reused"
	OutcomeSkipped     Outcome = "skipped"
	OutcomeInterrupted Outcome = "interrupted"
	OutcomeUnavailable Outcome = "unavailable"
)

type Reader 

type Reader struct {
	// contains filtered or unexported fields
}

A Reader is a read-only view of a ledger file. It is a separate connection with query_only set, so it reads while a run writes and can disturb nothing: this is what makes the live view cost the run nothing.

func OpenReader 

func OpenReader(ctx context.Context, dir string) (*Reader, bool, error)

OpenReader opens the ledger beside the index store in dir for reading, and verifies its schema fingerprint so a reader never half-reads a file written by another schema.

It reports false, and no error, for a workspace that has never recorded a run: there is nothing to read and nothing to report as broken. The connection is read-only, so asking never creates the file.

func (*Reader) Close 

func (r *Reader) Close() error

Close releases the reader's connections.

func (*Reader) LatestRun 

func (r *Reader) LatestRun(ctx context.Context, repositoryID string, generationID int64) (RunView, bool, error)

LatestRun is the run a status surface shows for a repository: the one that is still live if one is, and otherwise the one that produced generationID. A generationID of zero or less means the repository has no active generation, in which case the most recently started run answers -- which is what an operator asking about the run that just failed is asking for.

Live means the run's writer has renewed the deadline it published on the run row, not merely that the row says 'running'. A process that died without stopping its ledger leaves 'running' behind for ever, and preferring it would show that dead run instead of the run that produced the active generation -- exactly when an operator is investigating the crash. Such a run reads as interrupted, with its still-open spans interrupted too, which is what the collector itself writes for a ledger that stops; the reader states the same fact without touching the file, because a status surface must never write.

It returns false, and no error, when the repository has no recorded run. Spans are bounded to one page; a run with more says so.

func (*Reader) LiveStage 

func (r *Reader) LiveStage(ctx context.Context) (stage string, live bool, err error)

LiveStage is what a process waiting for the workspace lock needs of this file and nothing more: whether SOME run in it is still live by the liveRun rule above, and the stage that run is inside right now.

It is not scoped to a repository, and deliberately so. The lock and this ledger are both per workspace directory, the waiter is asking about the one process holding that directory, and the repository identity is derived far below the point where a waiter must decide whether to keep waiting.

The stage is the innermost span the live run still has open, which is the most specific thing that can be said about where the holder is. It is empty for a run that has opened no span yet, which is a live holder that has not reached a stage rather than a holder that is doing nothing.

One statement, so the liveness and the stage cannot be read from two different moments of a file another process is writing.

func (*Reader) MarkedUnits 

func (r *Reader) MarkedUnits(ctx context.Context, repositoryID string, generationID int64, stage string,
	named int) ([]MarkedUnits, error)

MarkedUnits reads the unit spans of every deferred run of a repository that holds a top-level span of this stage scoped to generationID in decimal -- the marker a run writes to say what it did to the units it planned over that generation -- and that started no earlier than the run that produced the generation. It answers per provider, in provider order, with the count and the first named of them by scope key.

The runs it reads are ones retention keeps for as long as the generation is retained: a sweep deletes only runs that started before the oldest run behind a retained generation, and every run read here started at or after the run behind this one (SweepRuns). A generation whose producing run the file no longer holds -- a ledger recreated after the store was built -- answers nothing, since no marker can be told apart from one about an earlier generation of the same id.

The markers are found through the runs index by repository and start, and each run's spans through the run's own key, so the read covers the runs since the generation's own and never the whole span table. The answer is bounded by named per provider.

func (*Reader) NeedClasses 

func (r *Reader) NeedClasses(ctx context.Context, repositoryID string) ([]NeedClass, int64, error)

NeedClasses is one page of a repository's learned need models in key order, the disclosure of how often each class's reservation fell short, and how many classes past the page the repository holds. A repository nothing was ever measured in answers none. The page is model.MaxRecordsPerResult wide and is read, and a truncated one counted, exactly as a run's spans are.

func (*Reader) RecordedPeak 

func (r *Reader) RecordedPeak(ctx context.Context, repositoryID, scopeKey, familyPrefix string) (int64, bool, error)

RecordedPeak is the largest process-tree peak this ledger holds for one scope of a repository, and whether one is held at all. It is two indexed reads of the high-water table and never a scan: the scope's own row by its key, and only where that row is absent and familyPrefix is not empty, the largest row whose key begins with familyPrefix -- a range over the same key, bounded by that family's own scopes.

The second result is the answer. A scope nothing ever sampled, a family with no measured scope and a repository that has never run a heavy unit all answer false with a zero the caller must not read: the table holds only measurements above zero, so a zero here can only mean "no row", and a peak of zero answered as observed would be a reservation of nothing.

func (*Reader) Run 

func (r *Reader) Run(ctx context.Context, runID string) (RunView, bool, error)

Run is the run with this identifier, and false when the file holds none: the answer for a caller that knows which run it is asking about. A run reports on itself through this and not through LatestRun, which answers with the live run of the repository -- and a process can have a second run of its own live, a deferred publication beside an index, which would then answer for the run that asked.

It judges liveness, bounds its spans and reports what it omitted exactly as LatestRun does; the two differ in which row they select and in nothing else.

type Run 

type Run struct {
	// contains filtered or unexported fields
}

A Run is one recorded run: an index attempt, a deferred publication, or the per-process overlay. It allocates its spans' ordinals and carries the run row's own totals, which the collector writes at every flush.

A nil *Run is the run of a ledger that records nothing: its methods do nothing, and the context it returns carries no run, so every span opened under it is a nil span.

func RunFromContext 

func RunFromContext(ctx context.Context) *Run

RunFromContext returns the run a context's work belongs to, or nil.

func (*Run) AttachGeneration 

func (r *Run) AttachGeneration(id int64)

AttachGeneration records the generation this run produced, once publication has opened one. A run that never reaches it leaves the column null.

func (*Run) Context 

func (r *Run) Context(ctx context.Context) context.Context

Context returns a context whose spans belong to this run. Every package that records is handed one of these and calls Start; none of them holds a *Run.

func (*Run) Dropped 

func (r *Run) Dropped() int64

Dropped is how many of this run's events the bus refused. It is on the run and not on an event because a drop happens exactly when no event gets through: an event carrying the count would be lost at the only moment it matters.

func (*Run) Finish 

func (r *Run) Finish(outcome Outcome)

Finish records how the run ended, and tells the collector so, because the end of a run is itself an ending to record: the units the plan named that nothing ever started are closed here, through the same publication every other span end goes through. Spans still open when the ledger stops are written interrupted regardless of what the run says about itself.

func (*Run) ID 

func (r *Run) ID() string

ID is the run's public identifier: the same 32 bytes as lowercase hex, and empty for a run that records nothing.

func (*Run) NeedModel 

func (r *Run) NeedModel(ctx context.Context, key NeedKey) ([]byte, bool, error)

NeedModel is the persisted model state for key in this run's repository, and false when none is recorded. A nil run and a run whose collector has detached read nothing and answer false, not an error, exactly as they record nothing: the store is read through the run's own attachment.

It first waits for the collector to write everything published before the call, so an observation still on the bus -- one a model dropped at the end of a stage published, or one a failed run left behind -- is in the answer, and a model loaded again never forgets it. It waits on the collector and reads through its one connection, so it is never called from a subscriber, which runs while a flush holds that connection.

func (*Run) ObserveNeed 

func (r *Run) ObserveNeed(o NeedObservation)

ObserveNeed records one file's observation. The model state it carries is written to the observation store in the same transaction as the observation, whatever becomes of the generation it was taken under.

It crosses the bus like every other event, so it never waits: a full bus drops it and counts the drop on the run row, and the next observation's state, which folds in everything before it, supersedes what was lost. The state is copied, so the caller may reuse its buffer at once.

An observation the store would refuse -- no language, no fingerprint, no build, no state, a negative class, or a negative figure, which no measurement is -- is counted as dropped too rather than published: written, it would fail the transaction it shares with every other event of the batch and lose them all. A nil or stopped run records nothing.

func (*Run) Report 

func (r *Run) Report(t Totals)

Report records the run row's totals as the run learns them. The last call before the collector's next flush is what a reader sees.

func (*Run) RepositoryID 

func (r *Run) RepositoryID() string

RepositoryID is the repository this run records, as the hex the ledger's readers take, and empty for a run that records nothing.

type RunRow 

type RunRow struct {
	RunID               string
	Kind                Kind
	RepositoryID        string
	GenerationID        *int64
	StartedAt           time.Time
	FinishedAt          *time.Time
	WallMS              int64
	Outcome             Outcome
	FileCount           int64
	SourceBytes         int64
	UnitsPlanned        int64
	UnitsSucceeded      int64
	UnitsFailed         int64
	UnitsSubdivided     int64
	EventsDropped       int64
	ProcessPeakRSSBytes *uint64
}

A RunRow is one recorded run as a reader sees it. WallMS is the run's elapsed time: the time between its start and its finish, or between its start and now while it is still going, so a live run reads as how far it has got rather than as a zero. It is zero on a run whose writer stopped refreshing its liveness, because that run's end was never measured; such a run reads as interrupted and carries no FinishedAt.

type RunView 

type RunView struct {
	Run          RunRow
	Spans        []SpanRow
	SpansOmitted int64
}

A RunView is one run and its spans. SpansOmitted is how many of the run's spans this page does not carry, so a caller reports a partial view as partial and says by how much: "some stages are missing" does not tell an operator whether the page dropped three of them or three hundred.

type Span 

type Span struct {
	// contains filtered or unexported fields
}

A Span is one bracketed piece of work: a run's root, a stage, a unit under a stage, a part of a subdivided unit. A nil *Span is valid and does nothing, which is what a caller that was handed no run gets, so instrumenting a code path never requires the caller to know whether a ledger is open.

func FromContext 

func FromContext(ctx context.Context) *Span

FromContext returns the innermost open span, or nil where none is open.

func Plan 

func Plan(ctx context.Context, stage, scopeKey, provider string) *Span

Plan opens a span for work that has been decided on but not started. The row exists from the moment the plan names the unit, so a unit that is never run -- because nothing admitted it, or because the run ended first -- still has a row that says so, instead of leaving no trace at all.

It returns no context: a planned span has no work under it yet. Begin makes it the parent of what runs.

func Start 

func Start(ctx context.Context, stage, scopeKey string) (context.Context, *Span)

Start opens a span under whatever span the context already carries, and returns a context a nested Start will find it in. With no run in the context it returns the context unchanged and a nil span whose methods do nothing.

It allocates the span's ordinal and does one non-blocking send. It never waits: a full bus drops the event and counts it on the run row.

func StartProvider 

func StartProvider(ctx context.Context, stage, scopeKey, provider string) (context.Context, *Span)

StartProvider is Start for a span a provider owns, which records which provider produced it.

func (*Span) AddIn 

func (s *Span) AddIn(n int64)

AddIn and AddOut count what went into and came out of this span -- files walked, records emitted, rows staged, bytes imported. They are one atomic add per item and never touch the bus, so a stage can count per file while a span is a thing that costs seconds. The collector snapshots them at every flush, which is how a reader in another process sees a live span advance.

func (*Span) AddOut 

func (s *Span) AddOut(n int64)

func (*Span) Begin 

func (s *Span) Begin(ctx context.Context) context.Context

Begin moves a planned span to running and returns a context a nested Start finds it in. The span's wall is measured from here and not from the plan, so a unit's cost stays the cost of running it and never the time it waited in a queue; what the row says about the wait is that it was 'planned' until now.

func (*Span) End 

func (s *Span) End(outcome Outcome, m Measured, err error)

End closes the span. A second call does nothing: a stage that ends its span on both a normal return and a deferred cleanup records one end, not two.

err fills the failed span's diagnostic code and message when the caller did not fill them itself. The message is the error's own text; no source bytes and no raw analyzer output reach this row.

func (*Span) In 

func (s *Span) In() int64

In and Out read the counters back, for a caller that reports its own totals.

func (*Span) Out 

func (s *Span) Out() int64

type SpanRow 

type SpanRow struct {
	RunID string
	Seq   int64
	// ParentSeq is the ordinal of the span this one nests under, absent for a
	// span the run opened at its top level. It is the ordinal rather than a
	// row id because the ordinal is what both sides of the bus know.
	ParentSeq *int64
	Stage     string
	ScopeKey  string
	Provider  string
	StartedAt time.Time
	// FinishedAt is absent while the span is running and on a span that was
	// still open when its run ended.
	FinishedAt *time.Time
	// WallMS is the span's measured wall once it has finished, and its elapsed
	// time so far while it is still running, so a live reader sees a stage
	// that has been going for a minute as a minute rather than as nothing.
	WallMS int64
	// Running is whether WallMS is elapsed-so-far rather than a final
	// measurement, so a caller never presents the two as the same thing.
	Running         bool
	CPUUserMS       *int64
	CPUSysMS        *int64
	CPUUnattributed string
	PeakRSSBytes    *uint64
	ReadBytes       *uint64
	WriteBytes      *uint64
	ItemsIn         int64
	ItemsOut        int64
	Outcome         Outcome
	DiagnosticCode  string
	Failure         string
	// ShareOfWall is this span's wall as a fraction of its run's, computed
	// when the row is read. It is zero on a row published to a subscriber,
	// where the run's own wall is not yet known.
	ShareOfWall float64
}

A SpanRow is one span as a reader sees it, and also what the collector hands a subscriber when a span finishes. Every measured field is a pointer: a figure the platform could not give is absent, never zero.

type Totals 

type Totals struct {
	FileCount           int64
	SourceBytes         int64
	UnitsPlanned        int64
	UnitsSucceeded      int64
	UnitsFailed         int64
	UnitsSubdivided     int64
	ProcessPeakRSSBytes *uint64
}

Totals are the run row's own counts, which the run reports as it learns them. ProcessPeakRSSBytes is nil where the platform does not expose the process's peak resident size.

ProcessPeakRSSBytes is the kernel's high-water mark for THIS PROCESS, read once the run has ended, and not a mark taken over the run alone: in a one-shot command the two are the same figure, while in a long-lived process serving a fourth refresh the mark may have been set by the first.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.