kvwindow

package
v3.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 6, 2026 License: Apache-2.0 Imports: 2 Imported by: 0

Documentation

Overview

Package kvwindow is the shape in which a model server's KV-cache activity is reported, and the fold that sums windows for the judged eval board (internal/evals/harness).

Every window is fleet-level and time-correlated: the server's whole traffic between two readings (core-models kvmetrics), never a session's share of it, and never folded into cost.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Fold

type Fold struct {
	// contains filtered or unexported fields
}

Fold sums windows: counts added (absent until one reports them), the hit rate recomputed from the sums, the peak the highest, the time span the union, and Reset, EndFailed and Approximate if any window's were. The zero value is an empty fold. Not safe for concurrent use.

func (*Fold) Add

func (f *Fold) Add(w *Window)

Add folds one window in; a nil window is skipped.

func (*Fold) Window

func (f *Fold) Window() *Window

Window returns the fold so far, or nil when nothing was added.

type Window

type Window struct {
	// Fleet is always true: the server's whole traffic in the window.
	Fleet bool      `json:"fleet"`
	Start time.Time `json:"start"`
	End   time.Time `json:"end"`
	// Queries and Hits are prefix-cache tokens queried and hit.
	Queries *float64 `json:"queries_tokens,omitempty"`
	Hits    *float64 `json:"hits_tokens,omitempty"`
	// HitRate is Hits/Queries from the counters; never from a gauge.
	HitRate *float64 `json:"hit_rate,omitempty"`
	// HitRateGauge is the server's own instantaneous hit-rate gauge at
	// the window's last reading, for servers without counters. A fold
	// does not carry it: an instantaneous value is not a sum.
	HitRateGauge *float64 `json:"hit_rate_gauge,omitempty"`
	// PeakKVUsage is the highest KV utilization (0–1) seen.
	PeakKVUsage *float64 `json:"peak_kv_usage,omitempty"`
	Readings    int      `json:"readings"`
	// Reset: an engine's counters went backwards inside the window.
	Reset bool `json:"reset,omitempty"`
	// EndFailed: the closing reading could not be taken, so the window
	// ends at the last good one.
	EndFailed bool `json:"end_failed,omitempty"`
	// Approximate: a bracketing reading was not taken at the window's
	// edge, so the window reaches up to one poll interval past it (see
	// the producer's docs).
	Approximate bool `json:"approximate,omitempty"`
	// Windows is how many windows a fold summed; zero on a single one.
	Windows int `json:"windows,omitempty"`
}

Window is one window of KV-cache activity, or a fold of several. Counts are tokens as the server reports them; a field the server did not expose is nil and omitted, never zero.

func From

func From(w kvmetrics.Window) *Window

From converts a kvmetrics window.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL