Documentation
¶
Overview ¶
Package views is CSF's measurements and the Prometheus and Grafana that show them: CSF owns both and uses no other project's.
Views exports every family of catalog.json at /metrics. The per-run families (turns, tokens by kind, spend, gate decisions, reply-gate refusals, merge checks) and the struggle rate are folded from the state directory's run records by one goroutine, which refolds only a run whose event log changed and publishes each measurement whole; the live families (sessions by phase, admission, the dispatcher's slices and limits) are read from their sources at each scrape; merges and the ontology alignment score come from main's merged pull requests.
Given a Stack, Start provisions Prometheus and Grafana through the container capability, with the dashboard in dashboard/csf.json, and backfills the history the records reconstruct. /views/panels/<metric> redirects to the panel that shows a family, on the host the browser used, which is how the Workbench links its tiles.
No metric without a panel, no panel without a metric: CheckPanels is the check tools/metricpanels runs on the merge path.
Index ¶
- Constants
- Variables
- func Bounded(value *float64, low *float64, high *float64) map[string]float64
- func ByClass(corrections []Correction) map[string]int64
- func CheckPanels(catalog Catalog, dashboard Dashboard) []string
- func CostsThrough(events []CostEvent, at time.Time) map[CostKey]CostTally
- func DashboardDocument() []byte
- func HistogramOpts(name string, buckets []float64) prometheus.HistogramOpts
- func Totals(runs []Run, assignments map[string]string) map[RunKey]*Tally
- func WriteBackfill(writer io.Writer, history History, from time.Time, through time.Time) (int, error)
- type Admission
- type AdmissionSource
- type Catalog
- type CheckKey
- type Correction
- type CorrectionSource
- type CostEvent
- type CostKey
- type CostTally
- type Dashboard
- type Dispatch
- type DispatchSource
- type FreeBytes
- type GateKey
- type GitHubPulls
- type GrafanaAccess
- type HandLabeled
- type History
- type IContainers
- type LabEntry
- type LabSample
- type Measurement
- type MergedPull
- type Metric
- type MetricType
- type Option
- func WithAdmission(source AdmissionSource) Option
- func WithClock(source clock.IClock) Option
- func WithCollectors(collectors ...prometheus.Collector) Option
- func WithCorpus(files iofs.IFiles) Option
- func WithCorrections(source CorrectionSource) Option
- func WithDispatch(source DispatchSource) Option
- func WithLogger(logger *slog.Logger) Option
- func WithPulls(source PullSource) Option
- func WithSessions(source SessionSource) Option
- func WithStack(stack *Stack) Option
- type Panel
- type PullSource
- type PullState
- type Retention
- type Run
- type RunKey
- type SessionSource
- type SessionState
- type Stack
- type StackOption
- func WithContainers(containers IContainers) StackOption
- func WithFreeBytes(free FreeBytes) StackOption
- func WithListener(listener ionet.IListener) StackOption
- func WithPublishedAddresses(addresses ...netip.Addr) StackOption
- func WithScrapeTarget(target string) StackOption
- func WithStackClock(source clock.IClock) StackOption
- func WithStateDirectory(directory string) StackOption
- type StackRecord
- type Struggle
- type Tally
- type TokenKind
- type Views
Constants ¶
const ( MetricSessions = "csf_sessions" MetricTurns = "csf_session_turns_total" MetricTokens = "csf_tokens_total" MetricCost = "csf_cost_usd_total" MetricWorkerCap = "csf_admission_worker_cap" MetricFreeDisk = "csf_admission_free_disk_bytes" MetricDiskFloor = "csf_admission_disk_floor_bytes" MetricAdmissionHeld = "csf_admission_held" MetricDispatchSlices = "csf_dispatch_slices" MetricDispatchLimit = "csf_dispatch_limit_remaining" MetricDispatchStage = "csf_dispatch_stage_slices" MetricDispatchStageAge = "csf_dispatch_stage_age_seconds" MetricDispatchLeadTime = "csf_dispatch_lead_time_seconds" MetricGateDecisions = "csf_gate_decisions_total" MetricReplyRefusals = "csf_reply_gate_refusals_total" MetricStruggleRate = "csf_struggle_rate_per_1k_tool_calls" MetricCompounding = "csf_compounding_factor" MetricMerges = "csf_merges_total" MetricOntologyScore = "csf_ontology_alignment_score" MetricOntologySignal = "csf_ontology_alignment_signal" MetricCorrections = "csf_operator_corrections_total" MetricAttentionHours = "csf_operator_attention_hours_total" MetricHypervisorCostEvents = "csf_hypervisor_cost_events_total" MetricHypervisorCostUSD = "csf_hypervisor_cost_usd_total" MetricInformationDensity = "csf_information_density" MetricMergeChecks = "csf_merge_checks_total" MetricBazelWall = "csf_bazel_build_wall_seconds" MetricBazelMemory = "csf_bazel_peak_memory_bytes" MetricHostPressure = "csf_host_pressure_some_avg10" MetricResumesHeld = "csf_resumes_held" )
The metric families, by name; catalog.json defines each.
const ( PeriodWeek = "week" PeriodDay = "day" BoundRate = "rate" BoundLow = "low" BoundHigh = "high" StateQueued = "queued" StateHeld = "held" StateRunning = "running" )
The values of the period, bound and state labels.
const ( OutcomeOK = "ok" OutcomeFailed = "failed" )
The outcomes of a control action, as csf_merge_checks_total labels them.
const ( SourceHandLabeled = "hand_labeled" SourceRuns = "runs" SourceAffect = "affect" )
The values of the source label of the orchestrator-quality families.
const ( // DashboardFile is the dashboard's path in this package, which Grafana // is provisioned from and the metric panel check reads. DashboardFile = "dashboard/csf.json" // DashboardUID is the dashboard's Grafana identifier, which panel links // address. DashboardUID = "csf" )
const ( // PrometheusRecordFile and GrafanaRecordFile are the two owned // containers' records under the state directory, mode 0600: where each // is and on which port. PrometheusRecordFile = "prometheus.json" GrafanaRecordFile = "grafana.json" // Directory holds, under the state directory, the files the containers // mount (Prometheus's configuration and backfill, Grafana's provisioning // and dashboard) and the backfill mark. Directory = "views" // BackfillMarkFile records how far the backfill reached, in Directory. BackfillMarkFile = "backfill.json" // PrometheusImage and GrafanaImage are the pinned images, by digest. PrometheusImage = "prom/prometheus:v3.5.0@sha256:63805ebb8d2b3920190daf1cb14a60871b16fd38bed42b857a3182bc621f4996" GrafanaImage = "grafana/grafana:12.2.1@sha256:35c41e0fd0295f5d0ee5db7e780cf33506abfaf47686196f825364889dee878b" // ScrapeInterval is how often Prometheus reads /metrics. The fold behind // it refreshes on the same period, so a shorter scrape would only repeat // samples; sessions run for minutes to hours, so a minute resolves them. ScrapeInterval = time.Minute // provisioning that Prometheus may keep; the measured value and the // derivation are recorded in its record. RetentionShare = 100 )
const ( // MetricsPath is where the families are served. MetricsPath = "/metrics" // PanelPath is the prefix of a family's panel link: PanelPath + name. PanelPath = "/views/panels/" // DashboardPath redirects to the dashboard. DashboardPath = "/views" // PullRefresh is how often main's merge history is read again: merges // land minutes apart, and each read lists every merged pull request. PullRefresh = 15 * time.Minute )
const ( // BackfillFile is the OpenMetrics history, written beside Prometheus's // configuration where its container reads it, and removed once loaded. BackfillFile = "backfill.om" )
const CatalogFile = "catalog.json"
CatalogFile is the catalog's path in this package, which the metric panel check reads from a checkout.
const ( // CostRefresh is how often the hypervisor's cost events are derived // again: the derivation reads every run whole, so it runs on a slower // cadence than the fold, and the cost model changes over hours. CostRefresh = 15 * time.Minute )
const MetricChatSettle = "csf_chat_settle_seconds"
MetricChatSettle is the family services/harness/chat observes: how long a session's event takes to reach the transcripts watching it.
Variables ¶
var ( // ErrInvalidOption reports a nil option or a value the views cannot use. ErrInvalidOption = errors.New("views: invalid option") // ErrMissingCapability reports views or a stack built without what they // need: the corpus; or the state directory, containers, listener, // free-disk measure, scrape target or an address a container reaches. ErrMissingCapability = errors.New("views: a required capability is missing") )
var ( // ErrNoGrafana is the answer to a panel link while no Grafana is // provisioned. ErrNoGrafana = errors.New("views: no Grafana is provisioned on this host") // ErrStarted reports a second Start: the views own their goroutines once. ErrStarted = errors.New("views: already started") )
var ( // ErrNoRepository reports a pull request source built without the // repository it lists. ErrNoRepository = errors.New("views: the repository whose merges are measured is required, as OWNER/NAME") )
Functions ¶
func ByClass ¶
func ByClass(corrections []Correction) map[string]int64
ByClass counts corrections by class.
func CheckPanels ¶
CheckPanels is the metric panel check: every catalog family has a panel that queries it, every panel queries only catalog families, and every panel has a title, a unit and, in its description, the definition of each family it queries. It returns one finding per gap, empty when there is none.
func CostsThrough ¶
CostsThrough adds up the cost events at or before at, by label set.
func DashboardDocument ¶
func DashboardDocument() []byte
DashboardDocument is the dashboard as the repository holds it, which the stack provisions Grafana with.
func HistogramOpts ¶
func HistogramOpts(name string, buckets []float64) prometheus.HistogramOpts
HistogramOpts are a histogram family's options, built from the catalog so the service observing it cannot drift from its name and definition.
func WriteBackfill ¶
func WriteBackfill(writer io.Writer, history History, from time.Time, through time.Time) (int, error)
WriteBackfill writes, as OpenMetrics, every sample the records reconstruct at the end of each hour after from and up to through: the per-run counters as they stood, the struggle rate as measured through the previous day, and the merges, score and signals as main's history stood. It returns how many samples it wrote.
Types ¶
type Admission ¶
type Admission struct {
WorkerCap int64
FreeBytes uint64
DiskFloorBytes uint64
Held bool
// Pressure is the percent of the last 10 s some task waited, by
// resource; a resource the kernel does not report is absent.
Pressure map[string]float64
// ResumesHeld is how many open runs the last restart still holds.
ResumesHeld int
}
Admission is the harness launch check.
type AdmissionSource ¶
AdmissionSource reads the harness launch check.
type Catalog ¶
type Catalog struct {
Metrics []Metric `json:"metrics"`
}
Catalog is every family CSF exports at /metrics.
func ReadCatalog ¶
ReadCatalog decodes a catalog document.
type Correction ¶
type Correction struct {
Class string `json:"class"`
What string `json:"what"`
At time.Time `json:"at"`
}
Correction is one operator-flagged mistake, classed by the ticket or gate that owns its class.
type CorrectionSource ¶
type CorrectionSource func(ctx context.Context) ([]Correction, error)
CorrectionSource lists the typed corrections AFFECT (#124) finds, which export as source affect.
type CostEvent ¶
CostEvent is one hypervisor cost event (#353), as the cost families count and price it.
type Dashboard ¶
type Dashboard struct {
UID string `json:"uid"`
Title string `json:"title"`
Panels []Panel `json:"panels"`
}
Dashboard is the part of the Grafana dashboard model the check and the panel links read.
func ReadDashboard ¶
ReadDashboard decodes a dashboard document.
type Dispatch ¶
type Dispatch struct {
Queued int
Held int
Running int
// Remaining is the launches each bounded limit still allows, by name.
Remaining map[string]int64
// Slices maps a launched slice's assignment to the slice.
Slices map[string]string
// Stages is how many slices stand in each stage, and StageAges how long
// the oldest of them has stood there, in seconds, by stage.
Stages map[string]int
StageAges map[string]float64
// LeadTime is the mean seconds from enqueue to merge of the slices merged
// in the last day; nil when none merged.
LeadTime *float64
}
Dispatch is the dispatcher's snapshot as the collector reads it.
type DispatchSource ¶
DispatchSource reads the dispatcher's snapshot.
type GitHubPulls ¶
type GitHubPulls struct {
// contains filtered or unexported fields
}
GitHubPulls lists a repository's merged pull requests through gh, run by the process capability.
func NewGitHubPulls ¶
func NewGitHubPulls(launcher proc.ILauncher, repository string) (*GitHubPulls, error)
NewGitHubPulls grants the merged pull requests of repository, OWNER/NAME.
func (*GitHubPulls) Merged ¶
func (pulls *GitHubPulls) Merged(ctx context.Context) ([]MergedPull, error)
Merged lists main's merged pull requests, oldest first.
type GrafanaAccess ¶
type GrafanaAccess struct {
Password string `json:"password"`
}
GrafanaAccess is Grafana's own part of its record: the admin password.
type HandLabeled ¶
type HandLabeled struct {
Source string `json:"source"`
Provenance string `json:"provenance"`
From time.Time `json:"from"`
Through time.Time `json:"through"`
AttentionHours int64 `json:"attention_hours"`
MergedSlices []int `json:"merged_slices"`
Corrections []Correction `json:"corrections"`
}
HandLabeled is corrections.json: the window labeled by hand before AFFECT types corrections, with the operator-attention hours counted for it.
type History ¶
type History struct {
Measurement *Measurement
Pulls []MergedPull
Costs []CostEvent
Slices map[string]string
}
History is what the backfill reconstructs from: the corpus, main's merge history and the slice each assignment ran for.
type IContainers ¶
type IContainers interface {
docker.IServices
SignalService(ctx context.Context, name string, signal string) error
Exec(ctx context.Context, name string, spec docker.ExecSpec) (docker.ExecResult, error)
}
IContainers is the part of the container capability the stack uses: the owned-service lifecycle, a signal to reread configuration, and the promtool run the backfill is.
type LabEntry ¶
type LabEntry struct {
Entry string `json:"entry"`
Slice string `json:"slice"`
At time.Time `json:"at"`
Note string `json:"note"`
Series []LabSample `json:"series"`
}
LabEntry is a dated lab entry's results, as lab.json records them: each a sample of one family, at the time the entry landed.
type LabSample ¶
type LabSample struct {
Metric string `json:"metric"`
Labels map[string]string `json:"labels"`
Value float64 `json:"value"`
}
LabSample is one result: a family, its labels and the value.
type Measurement ¶
type Measurement struct {
At time.Time
Runs []Run
Days []ouroboros.Rate
// OperatorHours are the distinct UTC hours holding an operator-authored
// message in any run, oldest first: an hour two runs share counts once.
OperatorHours []time.Time
}
Measurement is the corpus folded at one instant: every run and the struggle rate by day. It is published whole and never written again.
type MergedPull ¶
type MergedPull struct {
Number int
MergedAt time.Time
// Slice is the title's leading slice name, or empty.
Slice string
// Score is the ontology alignment score after the change, when the body
// reports one; Signals are the signal table's after column.
Score *float64
Signals map[string]float64
}
MergedPull is one pull request merged into main, as its title and body report it.
type Metric ¶
type Metric struct {
Name string `json:"name"`
Type MetricType `json:"type"`
Row string `json:"row"`
Labels []string `json:"labels"`
Definition string `json:"definition"`
Pending string `json:"pending,omitempty"`
}
Metric is one family CSF exports: its name, type, labels, the dashboard row its panel sits in, and its definition, which is both the family's help text and what its panel's description must carry. Pending names the slice whose landing gives the family its first samples; until then it is described and panelled but empty.
type MetricType ¶
type MetricType string
MetricType is a metric's Prometheus type as the catalog spells it.
const ( Gauge MetricType = "gauge" Counter MetricType = "counter" Histogram MetricType = "histogram" )
The metric types the catalog uses. A histogram's panel queries its _bucket, _sum or _count series.
type Option ¶
Option configures Views.
func WithAdmission ¶
func WithAdmission(source AdmissionSource) Option
WithAdmission grants the launch check the admission families are read from.
func WithCollectors ¶
func WithCollectors(collectors ...prometheus.Collector) Option
WithCollectors adds collectors another package owns, whose families the catalog also lists, to /metrics.
func WithCorpus ¶
WithCorpus grants the state directory whose run records are folded. Required.
func WithCorrections ¶
func WithCorrections(source CorrectionSource) Option
WithCorrections grants the typed operator corrections AFFECT (#124) finds, which export beside the hand-labeled window as source affect.
func WithDispatch ¶
func WithDispatch(source DispatchSource) Option
WithDispatch grants the dispatcher's snapshot, which the dispatch families and every slice label are read from.
func WithLogger ¶
WithLogger names the logger refresh failures are reported to.
func WithPulls ¶
func WithPulls(source PullSource) Option
WithPulls grants main's merge history, which the merge and ontology families are read from.
func WithSessions ¶
func WithSessions(source SessionSource) Option
WithSessions grants the session list csf_sessions is read from.
type Panel ¶
type Panel struct {
ID int `json:"id"`
Type string `json:"type"`
Title string `json:"title"`
Description string `json:"description"`
FieldConfig struct {
Defaults struct {
Unit string `json:"unit"`
} `json:"defaults"`
} `json:"fieldConfig"`
Targets []struct {
Expr string `json:"expr"`
} `json:"targets"`
}
Panel is one dashboard panel, or a row.
type PullSource ¶
type PullSource func(ctx context.Context) ([]MergedPull, error)
PullSource lists main's merged pull requests, oldest first.
type PullState ¶
PullState is main's merge history folded through one instant: merges by slice, and the latest reported score and signals.
func PullsThrough ¶
func PullsThrough(pulls []MergedPull, at time.Time) PullState
PullsThrough folds the merges at or before at.
type Retention ¶
Retention is Prometheus's own part of its record: how many bytes it may keep, and how that was derived.
type Run ¶
type Run struct {
Assignment string
Agent string
Executor string
Model string
// Hours maps the start of each UTC hour to what the run added in it.
Hours map[time.Time]*Tally
Total *Tally
Reading ouroboros.Reading
// OperatorHours are the UTC hours holding an operator-authored message
// sent into the run.
OperatorHours map[time.Time]bool
}
Run is one run directory folded: who ran it, what its records added by the hour they fell in, and its struggle reading. A published Run is never written again.
type SessionSource ¶
type SessionSource func(ctx context.Context) ([]SessionState, error)
SessionSource lists the host's agent sessions.
type SessionState ¶
SessionState is one agent session as the session service reports it.
type Stack ¶
type Stack struct {
// contains filtered or unexported fields
}
Stack is the Prometheus and Grafana a state directory owns: two docker.OwnedService containers and the files they mount.
func NewStack ¶
func NewStack(options ...StackOption) (*Stack, error)
NewStack validates the whole option set and defines the two owned containers.
func (*Stack) Backfill ¶
func (stack *Stack) Backfill(ctx context.Context, record StackRecord, history History, through time.Time) (int, error)
Backfill writes the history after record.BackfilledThrough and up to through into the Prometheus container's mounted directory, has promtool turn it into blocks in Prometheus's data directory, which Prometheus loads on its next block reload, and removes the file. It returns the samples loaded.
func (*Stack) Ensure ¶
func (stack *Stack) Ensure(ctx context.Context) (StackRecord, error)
Ensure writes the files the containers mount and brings Prometheus, then Grafana, up: created on their first start, reused on every later one. The files are rewritten each time, so a new binary's dashboard and scrape target take effect; Prometheus is told to reread its configuration when that changed. It returns once both health checks pass.
func (*Stack) MarkBackfilled ¶
MarkBackfilled records how far the backfill reached.
func (*Stack) Record ¶
func (stack *Stack) Record() (StackRecord, error)
Record reads the stack as recorded, without Grafana's password, or reports docker.ErrServiceNotOwned.
type StackOption ¶
StackOption configures a Stack.
func WithContainers ¶
func WithContainers(containers IContainers) StackOption
WithContainers grants the container capability. Required.
func WithFreeBytes ¶
func WithFreeBytes(free FreeBytes) StackOption
WithFreeBytes grants the free-disk measure retention is derived from. Required.
func WithListener ¶
func WithListener(listener ionet.IListener) StackOption
WithListener grants the network capability the first Ensure probes free ports with. Required.
func WithPublishedAddresses ¶
func WithPublishedAddresses(addresses ...netip.Addr) StackOption
WithPublishedAddresses names the host addresses Prometheus and Grafana are published on: loopback and the address csf serve already listens on beyond it. Required; the first non-loopback one is also how Grafana's container reaches Prometheus.
func WithScrapeTarget ¶
func WithScrapeTarget(target string) StackOption
WithScrapeTarget names csf serve's host:port as a container reaches it, where Prometheus scrapes /metrics. Required.
func WithStackClock ¶
func WithStackClock(source clock.IClock) StackOption
WithStackClock replaces the clock the health wait is paced by.
func WithStateDirectory ¶
func WithStateDirectory(directory string) StackOption
WithStateDirectory names the state directory the records and the mounted files live in. Required.
type StackRecord ¶
type StackRecord struct {
Prometheus docker.OwnedRecord[Retention] `json:"prometheus"`
Grafana docker.OwnedRecord[GrafanaAccess] `json:"grafana"`
Addresses []string `json:"addresses"`
ScrapeTarget string `json:"scrape_target"`
BackfilledThrough time.Time `json:"backfilled_through"`
}
StackRecord is the stack as Ensure leaves it: both owned containers' records, the addresses they are published on, csf serve's address as Prometheus scrapes it, and how far the backfill reached.
func (StackRecord) GrafanaURL ¶
func (record StackRecord) GrafanaURL(host string) string
GrafanaURL is Grafana's address on host, one of the addresses it is published on.
type Struggle ¶
type Struggle struct {
Week ouroboros.Rate
Day ouroboros.Rate
Compounding ouroboros.Trend
CompoundingDaily ouroboros.Trend
}
Struggle is the struggle rate and its compounding as of one day.
type Tally ¶
type Tally struct {
Turns int64
Tokens map[TokenKind]int64
CostUSD float64
Gates map[GateKey]int64
Refusals map[string]int64
Checks map[CheckKey]int64
}
Tally is what a run's records added: in one hour, or in all of them.
type Views ¶
type Views struct {
// contains filtered or unexported fields
}
Views exports CSF's measurements and serves the panel links.
func NewViews ¶
NewViews validates the whole option set and returns the views, their registry holding the collector and every added one.
func (*Views) Registry ¶
func (views *Views) Registry() *prometheus.Registry
Registry is the registry /metrics serves.