llm

type Backpressure struct {
	// Reason is the proxy's own classification. corrallm: "rejected" (queue full),
	// "queue-timeout" (waited and never got a slot), "spill" (routed elsewhere),
	// "exhausted" (budget/quota gone).
	Reason string
	// RetryAfter is when the server says to come back. Ok=false from
	// backpressureFrom means it did not say.
	RetryAfter time.Duration
	// Capacity / InFlight / Waiting describe the queue: total slots, slots busy,
	// and requests already waiting. Zero = not reported.
	Capacity, InFlight, Waiting int
	// Message is the server's human-readable line, when it sent one.
	Message string
	// Ticket identifies THIS ATTEMPT to the server, across every rejection it
	// takes. corrallm mints one when it turns away a request that carried no id
	// of its own, and hands it back on the 429; echoing it on the retry is what
	// lets the server tell a caller returning for the fourth time from a caller
	// arriving for the first — which is the difference between waiting your turn
	// and starving behind an endless supply of newcomers.
	//
	// The client echoes this automatically (see postWithRetry). Empty for a
	// server that does not issue tickets.
	Ticket string
}

type ChatOpts struct {
	ToolChoice string
	TraceID    string
	// RequestID identifies this ATTEMPT to a fair-share proxy, and is what lets
	// a retry keep the place in line the previous attempt earned.
	//
	// Distinct from TraceID, which says WHICH THREAD this belongs to and lives
	// for the life of a conversation. A request id names one attempt to get one
	// answer, and its whole purpose is to be reused across the retries of that
	// attempt — including retries this client cannot see.
	//
	// That is the case it exists for. postWithRetry already carries the ticket
	// across its own 429 retries, but a TURN-SCOPE retry (a host re-running the
	// whole turn after a mid-stream death — see TurnRetryBudget) starts a fresh
	// loop, and the caller arrives as a stranger having forfeited the waiting it
	// already did. Setting this from the ticket the previous attempt was given
	// (RetryEvent.BP.Ticket) makes the new call the same customer.
	//
	// PASS BACK THE SERVER'S TICKET, not an id of your own. corrallm signs the
	// ones it mints and grants queue age only to those; an arbitrary id still
	// groups the attempts in its log — which is worth having — but earns no
	// priority, because otherwise priority would be a header anyone can write.
	// Empty means "no id": the server mints one if it turns us away.
	RequestID      string
	Grammar        string
	ResponseFormat any

	// Temperature and Seed pin sampling for reproducibility. Both are pointers
	// so "unset" stays distinct from a deliberate 0 — temperature 0 is exactly
	// the value a caller most wants to send, and a plain float64 could never
	// express it without also forcing it on every caller that had no opinion.
	//
	// A measurement harness needs these: without them the server's own sampling
	// config decides, so a model launched with --temp 0.7 makes a pass/fail
	// probe a coin flip and single-shot runs disagree with themselves.
	// Providers that do not support seed ignore the field.
	Temperature *float64
	Seed        *int

	// Stop ends generation when any of these strings is produced. The provider
	// excludes the matched string from the returned text.
	//
	// Needed with Grammar: a completed grammar does NOT force EOS. Measured with
	// `root ::= call`, the model finished a valid call and then emitted the same
	// call again, repeatedly, to the token cap. Stopping on the block terminator
	// is what actually ends the turn.
	Stop []string

	// MaxTokens caps the completion. 0 = leave it to the server.
	//
	// This matters most with Grammar set. A grammar that permits repetition (a
	// heredoc body is `bodyline*`) never REQUIRES termination, so an unbounded
	// request can run to the context limit instead of finishing — observed as a
	// hang, not an error.
	MaxTokens int

	// Repetition controls. Pointers for the same reason as Temperature: 0 is a
	// meaningful value (it is "off"), and a plain float64 could not distinguish
	// "the caller wants it off" from "the caller had no opinion".
	//
	// These exist because a CUT is not a cure. RepetitionGuard stops a runaway
	// after the fact; it does nothing about the cause, and at temperature 0 the
	// cause is deterministic — measured on one survey page, the same generation
	// was cut three times in 34 seconds with byte-identical output, because a
	// greedy decoder in a repeating basin has no way out. Only sampling does.
	//
	// FrequencyPenalty and PresencePenalty are the OpenAI-standard pair
	// (penalize by count seen, and by seen-at-all). RepeatPenalty is
	// llama.cpp's own multiplicative penalty over a trailing window, ignored by
	// providers that do not implement it — 1.0 is off, ~1.1 is a light touch.
	FrequencyPenalty *float64
	PresencePenalty  *float64
	RepeatPenalty    *float64

	// Reasoning budget. See reasoning.go for the provider mapping — the two
	// dialects use different fields and NEITHER rejects the other's, so a
	// hand-written body cannot tell a working budget from a dropped one.
	//
	// Pointer for the same reason as the penalties: 0 is a meaningful value
	// (llama.cpp reads it as "end reasoning immediately") and -1 means
	// disabled, so a plain int could not express either without also forcing a
	// choice on callers that have none.
	ReasoningBudgetTokens *int

	// ReasoningBudgetMessage is injected at the cut. Only llama.cpp has this;
	// Anthropic ends thinking without saying so. Empty sends nothing.
	ReasoningBudgetMessage string

	// EnableThinking overrides the SERVER's default mode per request, via
	// chat_template_kwargs. Needed because the server flag is only a default:
	// Qwen3.8 ships thinking ON, a deployment may launch it with
	// --reasoning off, and a caller that wants thinking has to ask for it.
	//
	// Meaningless to a template that does not read the kwarg, and unused on the
	// Anthropic dialect, where a budget is itself the toggle.
	EnableThinking *bool

	// ReasoningEffort is "low", "medium" or "high" ("none": no thinking), for
	// an endpoint that takes an effort rather than a budget: the openrouter
	// and openai dialects send it (see reasoning.go); llama.cpp's and
	// Anthropic's use EnableThinking and ReasoningBudgetTokens.
	ReasoningEffort string
}

type Client struct {

	// RetryBudget caps the total wall-clock a single request spends retrying
	// 429/5xx before giving up (exponential backoff to retryMaxBackoff,
	// honoring retry_after, but bounded). 0 → defaultRetryBudget (5m). The
	// caller's ctx deadline still wins if shorter. Set it per Client for a
	// busy endpoint.
	//
	// NEGATIVE means unbounded: ride out backpressure until the ctx says stop.
	// Correct for a single-slot endpoint, where a wait is not a delay but
	// another attempt at the only slot, and a 5-minute default gives up in the
	// middle of an ordinary busy spell.
	//
	// Unbounded is safe because it only extends the 429 path. 5xx is ALSO capped
	// by an attempt count (retry5xxMaxAttempts), so a genuinely broken upstream
	// still stops; and a 500 reporting unparseable tool-call arguments is
	// returned immediately without retrying, since the model produced those bytes
	// deterministically from this context and every retry reproduces them.
	// Nothing can spin forever on an outcome that cannot change.
	RetryBudget time.Duration

	// Retry5xxAttempts caps how many times a 5xx is retried. 0 → default (5).
	//
	// The default is tuned for a chat turn, where five failures means the
	// upstream is genuinely broken and an operator should hear about it. It is
	// wrong for a long BATCH: transcribing a 33-page document is ~33 requests,
	// and a single upstream blip outlasting ~15s of backoff fails the whole
	// document. Measured on a real corpus, every document over 20 pages failed
	// this way and no document over 20 pages ever completed.
	//
	// Raising it is safe because RetryBudget still bounds the wall clock — the
	// attempt cap and the budget are two different guards, and for batch work the
	// budget is the one that should bind.
	Retry5xxAttempts int

	// TurnRetryBudget is the wall-clock budget for a CALLER's outer, turn-scope
	// retry (a caller that re-runs the whole turn after a mid-stream death —
	// see agentkit/agent or dun). 0 means unset: the caller falls back to this
	// client's request-scope RetryBudget. To DISABLE the caller's turn retry,
	// set DisableTurnRetry, because "0" here means "no opinion".
	TurnRetryBudget time.Duration

	// DisableTurnRetry tells a turn-scope caller not to retry at all: a
	// mid-stream death then fails the turn immediately.
	DisableTurnRetry bool

	// Repetition configures degenerate-loop detection on streamed output. The
	// zero value is ON with defaults; set Repetition.Off to disable. See
	// RepetitionGuard for why this is not opt-in.
	Repetition RepetitionGuard

	// MessagesAPI talks to the endpoint with the Claude Messages API whatever its
	// address; api.anthropic.com always does. See anthropic.go.
	MessagesAPI bool

	// Watch, when set, REPLACES the Repetition guard for a stream: it is called
	// once per stream with the messages that were sent, and what it returns
	// sees every delta. loopdetector.Install is the usual way to set it. Repetition.Off
	// has no effect on it.
	Watch func(history []Message) StreamWatcher

	// OnRetry, when set, is called once per retry decision (and once when the
	// loop recovers or gives up), so a caller can SHOW the wait instead of
	// sitting mute through it. See RetryEvent.
	//
	// Called from the goroutine issuing the request, synchronously, while the
	// caller is blocked in Chat/ChatStream/Embed — so it must not block for long
	// and must not re-enter the client.
	OnRetry func(RetryEvent)

	// User is sent as the OpenAI-standard "user" field on every chat request:
	// a stable id for the end user or conversation behind the call. A host that
	// gives each conversation its own Client sets it to the conversation's id,
	// so a proxy can route that conversation's calls to the slot that already
	// holds its prompt prefix. Empty means the field is omitted.
	User string

	// Header holds extra HTTP headers sent on every request this client makes
	// (chat, embeddings, transcription). The client's own headers —
	// Content-Type, Authorization, X-Trace-Id, X-Corrallm-Request-Id — are set
	// after these and win on a clash. nil sends none.
	Header http.Header

	// OnUsage, when set, is called once per completed chat response with its
	// final Usage — tokens, Cost, Limits and KeyBudget — so a host can total
	// spend across every call a client makes without reading each stream
	// itself. A stream that carried no usage chunk but whose headers reported
	// something still calls it, with the header facts alone.
	//
	// Called from the goroutine reading the response (for ChatStream, before
	// its channel closes), so it must not block for long and must not re-enter
	// the client.
	OnUsage func(Usage)
	// contains filtered or unexported fields
}

type CompletionTokensDetails struct {
	ReasoningTokens *int `json:"reasoning_tokens,omitempty"`
}

type ContentPart struct {
	Type     string    `json:"type"`
	Text     string    `json:"text,omitempty"`
	ImageURL *ImageURL `json:"image_url,omitempty"`
}

type Cost struct {
	// Amount in Currency. A reported 0 is a real zero (a cache hit, a free
	// model).
	Amount float64
	// Currency is CurrencyUSD, CurrencyCredits, or "" when the source names
	// none (usage.cost from an endpoint that is neither LiteLLM nor
	// OpenRouter).
	Currency string
	// Reported is true when the response carried a cost at all.
	Reported bool
}

type EchoResult struct {
	Name    string
	Want    string
	Got     string
	Lossy   bool
	NoCall  bool   // the endpoint returned no tool call at all
	BadJSON bool   // arguments did not parse
	Err     string // transport failure
}

type ImageURL struct {
	URL    string `json:"url"`
	Detail string `json:"detail,omitempty"`
}

type InputTooLargeError struct {
	Status int
	Body   string
}

type KeyBudget struct {
	// Spend is what the key has spent in the current budget period (or all
	// time when the budget does not reset).
	Spend float64
	// Limit is the budget. Meaningful only when HasLimit; false means no limit
	// is set or none was reported.
	Limit    float64
	HasLimit bool
	// Remaining is what is left. Meaningful only when HasRemaining.
	Remaining    float64
	HasRemaining bool
	// ResetAt is when the budget resets. Zero = not reported or never.
	ResetAt time.Time
	// Interval is the reset period as the gateway names it ("30d", "monthly").
	// Empty = not reported or never.
	Interval string
	// Currency of every amount above.
	Currency string
	// Source is one of the BudgetSource* constants.
	Source string
	// Reported is true when any of this was reported.
	Reported bool
}

type Message struct {
	Role    string `json:"role"`
	Content string `json:"content"`
	Name    string `json:"name,omitempty"`
	// Parts, when non-empty, replaces Content with a multimodal content array
	// (text + image parts). Send-only: responses are always plain text, so this
	// is never populated on a decoded reply. Tagged "-" — MarshalJSON emits it
	// under "content" itself.
	Parts []ContentPart `json:"-"`
	// ToolCalls carries an assistant message's requested tool calls, so a
	// reconstructed conversation replays a valid assistant(tool_calls) →
	// tool(tool_call_id) structure instead of orphan tool messages.
	ToolCalls []ToolCall `json:"tool_calls,omitempty"`
	// ToolCallID links a role="tool" result back to the assistant tool call
	// that produced it (OpenAI requires this correlation).
	ToolCallID string `json:"tool_call_id,omitempty"`
}

type NamedWindow struct {
	Name   string
	Window RateWindow
}

type PromptTokensDetails struct {
	CachedTokens int `json:"cached_tokens"`
}

type RateLimits struct {
	Requests RateWindow
	// Tokens is the combined token window (OpenAI, LiteLLM), or on Anthropic
	// the most restrictive token limit in effect.
	Tokens RateWindow
	// InputTokens and OutputTokens are Anthropic's split windows.
	InputTokens  RateWindow
	OutputTokens RateWindow
	// Reported is true when any window was.
	Reported bool
}

type RateWindow struct {
	// Limit is the window's size. 0 = not reported (a limit of zero is not a
	// window anyone is sent).
	Limit int64
	// Remaining is what is left in the window. Meaningful only when
	// HasRemaining: 0 left is the most important value this can carry.
	Remaining    int64
	HasRemaining bool
	// ResetAt is when the window is full again, absolute — a relative reset
	// ("6m0s") is added to the time the response arrived. Zero = not reported.
	ResetAt time.Time
	// Reported is true when any of the window's headers was present.
	Reported bool
}

type RepairOutcome int

type RepetitionGuard struct {
	// Off disables detection entirely.
	Off bool

	// MinPeriod is the shortest repeating block considered a loop (default 24).
	// Below this, ordinary formatting repeats freely.
	MinPeriod int
	// MaxPeriod is the longest block scanned for (default 1024). Cost is linear
	// in this, so it bounds the check rather than the phenomenon.
	MaxPeriod int
	// MinReps is how many consecutive identical copies are required (default 3).
	MinReps int
	// MinSpan is the minimum total bytes the repetition must cover (default
	// 512). This is what stops three copies of a short header from tripping it.
	MinSpan int
	// CheckEvery is how many newly-arrived bytes trigger a scan (default 256).
	// The scan is not run per delta: a delta is a token, and scanning per token
	// would multiply the cost by ~4 for no earlier detection than this.
	CheckEvery int
}

type RepetitionInfo struct {
	// Where the loop was detected: "content", "reasoning", or "tool:<name>" for
	// a call's arguments.
	Where string `json:"where"`
	// Kind names the detector when it was not RepetitionGuard, which leaves it
	// empty. See package loopdetector.
	Kind string `json:"kind,omitempty"`
	// Period is the length in bytes of the repeating block, 0 when the loop has
	// no one block (Kind is set); Reps is how many copies of it were seen.
	Period int `json:"period,omitempty"`
	Reps   int `json:"loops"`
	// Trailing is the byte count of redundant copies — (Reps-1)*Period — i.e.
	// how much can be trimmed from the end of the accumulated text while leaving
	// one intact copy of the block.
	Trailing int `json:"trailing,omitempty"`
	// Sample is one copy of the repeating block, capped for logging.
	Sample string `json:"sample,omitempty"`
	// First and Last are the earliest repeat in view and the one that tripped
	// this, capped: sentences, or for KindTool the call's name and arguments.
	// They show where the loop was entered and where it has got to, which one
	// sample cannot. Empty from RepetitionGuard.
	First string `json:"first,omitempty"`
	Last  string `json:"last,omitempty"`
}

type RetryEvent struct {
	Kind RetryKind
	// Attempt is the 1-based attempt this event describes: the one that just
	// failed, or — for RetryRecovered — the one that succeeded.
	Attempt int
	// Status is the HTTP status, 0 on the transport path (no response at all).
	Status int
	// Err is the transport error, nil on the status paths.
	Err error
	// Body is the first line of the server's own message, when it sent one. The
	// single most actionable field on a 5xx (see the "input too large" case in
	// postWithRetry) and empty far more often than not.
	Body string
	// Delay is how long the client will wait before the next attempt. 0 on
	// RetryRecovered / RetryGiveUp.
	Delay time.Duration
	// Elapsed is time spent in this call's retry loop so far.
	Elapsed time.Duration
	// Budget is the wall-clock ceiling for the whole loop (Client.RetryBudget).
	Budget time.Duration
	// Attempts5xx / Max5xx expose the SEPARATE attempt cap that bounds the 5xx
	// path, which is what actually ends most upstream outages — the budget rarely
	// gets a chance to. Both 0 outside the 5xx path.
	Attempts5xx, Max5xx int
	// BP carries the queue detail a fair-share proxy attached to a 429 — slots
	// busy, requests ahead, why. Non-nil only on the 429 path, and only worth
	// rendering when BP.Queued().
	BP *Backpressure
	// ServerAsked is true when Delay is what the SERVER instructed (Retry-After or
	// a backpressure body) rather than our own exponential schedule. Worth saying
	// out loud: it means the wait is a real estimate, not a guess.
	ServerAsked bool
	// Reason is a human-readable sentence: what failed and what happens next.
	Reason string
}

type RetryKind string

type ServerTiming struct {
	// QueuedMS is time the request spent waiting for a slot. NOT execution.
	QueuedMS int64
	// LoadMS is time spent bringing a backend up. NOT execution.
	LoadMS int64
	// UpstreamTTFBMS is the model's prompt-eval latency. This IS execution.
	UpstreamTTFBMS int64
	// Reported is true when the response carried the headers at all.
	Reported bool
}

type Session struct {
	ID       string
	Messages []Message
	// contains filtered or unexported fields
}

type StopReason string

type StreamChunk struct {
	Content string
	// Reasoning is one delta of the model's reasoning text, when the provider
	// streams it apart from Content (llama.cpp and DeepSeek send
	// delta.reasoning_content; vLLM and OpenRouter send delta.reasoning). Empty
	// on a provider that hides its reasoning or sends none — which is not
	// evidence that no reasoning happened. Never part of Content, so a consumer
	// that ignores it sees exactly what it saw before.
	Reasoning string
	ToolCall  *ToolCall
	Done      bool
	Error     string
	Usage     *Usage

	// PartialToolCalls carries whatever tool-call arguments had accumulated
	// when an Error chunk was produced. Providers stream tool arguments
	// incrementally, so a call cut off mid-argument (a context window filling
	// mid-write) is already buffered client-side — the caller can salvage the
	// work instead of making the model regenerate it.
	//
	// These are INCOMPLETE and deliberately NOT delivered as ToolCall chunks:
	// dispatching a truncated call would run it with half its arguments.
	// Only ever set alongside Error.
	PartialToolCalls []ToolCall

	// StopReason names a non-model reason the stream ended — set on the Done
	// chunk. Empty on a normal completion, so consumers that ignore it are
	// unaffected. StopReasonRepetition (this client cut a loop) and
	// StopReasonLength (the provider ran out of room) are the values.
	StopReason StopReason

	// TruncatedToolCalls names the calls whose arguments the cut left
	// unparseable, when StopReason is StopReasonLength. Empty when the cut
	// landed in prose — which is the distinction a consumer needs, because a
	// truncated CALL already gets an error result to answer it and a truncated
	// REPLY gets nothing at all.
	TruncatedToolCalls []string

	// Repetition describes the degenerate loop that was cut, when StopReason is
	// StopReasonRepetition. With no StopReason, a proxy found a loop and let the
	// stream pass: nothing was cut and the finding is for the caller to weigh. It carries what to log, what to tell the model, and
	// how many trailing bytes are redundant copies. Nil otherwise.
	Repetition *RepetitionInfo
}

type StreamWatcher interface {
	Watch(channel, delta string) *RepetitionInfo
}

type Timings struct {
	CacheN      int     `json:"cache_n"`
	PromptN     int     `json:"prompt_n"`
	PromptMS    float64 `json:"prompt_ms"`
	PredictedN  int     `json:"predicted_n"`
	PredictedMS float64 `json:"predicted_ms"`
	// DraftN and DraftNAccepted are the speculative-decoding (MTP) counts
	// llama.cpp adds when it drafts: tokens drafted and tokens the model kept.
	// Zero when the server does not draft.
	DraftN         int `json:"draft_n,omitempty"`
	DraftNAccepted int `json:"draft_n_accepted,omitempty"`
}

type ToolCall struct {
	ID       string `json:"id"`
	Type     string `json:"type"`
	Function struct {
		Name      string `json:"name"`
		Arguments string `json:"arguments"`
	} `json:"function"`
}

type ToolCallWatcher interface {
	WatchToolCall(name, args string) *RepetitionInfo
}

type ToolDef struct {
	Type     string `json:"type"`
	Function struct {
		Name        string `json:"name"`
		Description string `json:"description"`
		Parameters  any    `json:"parameters"`
	} `json:"function"`
}

type ToolSurfaceReport struct {
	Model string
	// Echo records one call-and-echo round trip per payload shape: the model is
	// asked to pass a known string through a tool argument, and the argument that
	// comes back is compared byte for byte.
	Echo []EchoResult
	// GrammarAccepted reports whether the endpoint accepts a GBNF grammar at all.
	GrammarAccepted bool
	// GrammarWithToolsAccepted reports whether a grammar may be combined with
	// tools. llama.cpp refuses ("Cannot specify grammar with tools"), which is
	// why a custom tool-call format parsed from content is the ONLY path where
	// generation can be constrained at the sampler.
	GrammarWithToolsAccepted bool
	GrammarError             string
}

type TruncatedToolCallError struct {
	Status int
	Body   string
}

type Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`

	// Anthropic's shape: flat, top-level.
	CacheReadInputTokens     int `json:"cache_read_input_tokens,omitempty"`
	CacheCreationInputTokens int `json:"cache_creation_input_tokens,omitempty"`

	// OpenAI's shape: NESTED. llama-server (and thus corrallm) speaks this
	// one, and only this one — it was previously unparsed, so every cached
	// token read as zero and prompt_tokens looked like real work forever.
	PromptTokensDetails *PromptTokensDetails `json:"prompt_tokens_details,omitempty"`

	// CompletionTokensDetails is OpenAI's nested completion breakdown, which is
	// where a provider that counts reasoning separately reports it. llama.cpp
	// (b11191, measured 2026-09-26) sends no such block: its completion_tokens
	// includes the reasoning and says nothing about the split. See
	// ReportedReasoningTokens.
	CompletionTokensDetails *CompletionTokensDetails `json:"completion_tokens_details,omitempty"`

	// LatencyMS is wall-clock for this one round-trip, measured client-side.
	// Not reported by any provider — but "how long did that call take" is half
	// of what anyone asks of a trace, and without it the token counts alone
	// cannot distinguish a slow cold load from a large generation.
	LatencyMS int64 `json:"latency_ms,omitempty"`

	// Server is what the PROXY said about its own overhead on this round-trip —
	// queueing and cold-load time it held the request without running anything.
	// Stamped client-side from response headers, like LatencyMS; zero-valued and
	// Reported=false against an endpoint that says nothing.
	//
	// LatencyMS alone cannot be attributed: it is one number covering the model
	// working and the proxy waiting. This is what separates them.
	Server ServerTiming `json:"server_timing,omitzero"`

	// Timings is what the MODEL SERVER said it spent computing this round, when
	// it says so: llama.cpp's non-standard top-level "timings" object, a sibling
	// of "usage" on the final stream event (corrallm relays it verbatim). Nil
	// against an endpoint that sends none.
	//
	// It is the other half Server cannot give. Server reports the proxy's own
	// admission queue; a request the proxy admitted can still wait inside
	// llama-server behind other requests on a single slot, and that wait is on
	// no header. LatencyMS minus Timings.ComputeMS is the whole of what this
	// round waited, wherever it waited.
	Timings *Timings `json:"timings,omitempty"`

	// Cost is the money this response cost, when the endpoint said: LiteLLM's
	// x-litellm-response-cost header, or usage.cost in the body (OpenRouter;
	// LiteLLM on a stream). Zero-valued and Reported=false otherwise. See
	// cost.go.
	Cost Cost `json:"response_cost,omitzero"`

	// Limits is the rate-limit snapshot the response headers carried (OpenAI,
	// Anthropic and LiteLLM shapes). Zero-valued and Reported=false when absent.
	Limits RateLimits `json:"rate_limits,omitzero"`

	// KeyBudget is the key's spend and budget as of this response, when the
	// gateway stamps it on every call (LiteLLM's x-litellm-key-spend and
	// x-litellm-key-max-budget). No reset time: for that, Client.KeyBudget.
	KeyBudget KeyBudget `json:"key_budget,omitzero"`
}