Documentation
¶
Overview ¶
Package kvwindow is the shape in which a model server's KV-cache activity is reported, and the fold that sums windows for the judged eval board (internal/evals/harness).
Every window is fleet-level and time-correlated: the server's whole traffic between two readings (core-models kvmetrics), never a session's share of it, and never folded into cost.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Fold ¶
type Fold struct {
// contains filtered or unexported fields
}
Fold sums windows: counts added (absent until one reports them), the hit rate recomputed from the sums, the peak the highest, the time span the union, and Reset, EndFailed and Approximate if any window's were. The zero value is an empty fold. Not safe for concurrent use.
type Window ¶
type Window struct {
// Fleet is always true: the server's whole traffic in the window.
Fleet bool `json:"fleet"`
Start time.Time `json:"start"`
End time.Time `json:"end"`
// Queries and Hits are prefix-cache tokens queried and hit.
Queries *float64 `json:"queries_tokens,omitempty"`
Hits *float64 `json:"hits_tokens,omitempty"`
// HitRate is Hits/Queries from the counters; never from a gauge.
HitRate *float64 `json:"hit_rate,omitempty"`
// HitRateGauge is the server's own instantaneous hit-rate gauge at
// the window's last reading, for servers without counters. A fold
// does not carry it: an instantaneous value is not a sum.
HitRateGauge *float64 `json:"hit_rate_gauge,omitempty"`
// PeakKVUsage is the highest KV utilization (0–1) seen.
PeakKVUsage *float64 `json:"peak_kv_usage,omitempty"`
Readings int `json:"readings"`
// Reset: an engine's counters went backwards inside the window.
Reset bool `json:"reset,omitempty"`
// EndFailed: the closing reading could not be taken, so the window
// ends at the last good one.
EndFailed bool `json:"end_failed,omitempty"`
// Approximate: a bracketing reading was not taken at the window's
// edge, so the window reaches up to one poll interval past it (see
// the producer's docs).
Approximate bool `json:"approximate,omitempty"`
// Windows is how many windows a fold summed; zero on a single one.
Windows int `json:"windows,omitempty"`
}
Window is one window of KV-cache activity, or a fold of several. Counts are tokens as the server reports them; a field the server did not expose is nil and omitted, never zero.