Documentation
¶
Overview ¶
Package proxy implements the FinOps circuit-breaker reverse proxy: the lean, Apache-2.0 core engine that fingerprints request bodies, rejects looped repeats, and enforces token-budget caps.
It depends only on the sibling core packages (circuitbreaker, pricing, logstream) and a small set of seams declared in this package. Every enterprise concern — multi-tenant hierarchy, SSO/session auth, webhook delivery, dashboard persistence — is pushed behind those seams and supplied by the enterprise module, never imported here.
Index ¶
- Constants
- Variables
- func DashboardHandler() http.Handler
- func HashAPIKey(raw string) string
- func MetricsHandler(m *Metrics, det circuitbreaker.Detector) http.Handler
- func Middleware(a Authenticator, next http.Handler) http.Handler
- func NewHandler(upstream *url.URL, det circuitbreaker.Detector, price pricing.PriceSource, ...) http.Handler
- func WithCaller(ctx context.Context, c Caller) context.Context
- type Authenticator
- type BudgetConfig
- type BudgetLedger
- type BudgetMonitor
- type BudgetSpender
- type Caller
- type CallerStore
- type CostThresholdReader
- type Emitter
- type Event
- type EventType
- type HandlerOption
- func WithAuth(a Authenticator) HandlerOption
- func WithBudget(cfg BudgetConfig) HandlerOption
- func WithBudgetLedger(l BudgetLedger) HandlerOption
- func WithBudgetSpender(s BudgetSpender) HandlerOption
- func WithCostThreshold(r CostThresholdReader) HandlerOption
- func WithEmitter(e Emitter) HandlerOption
- func WithIncidentRecorder(r IncidentRecorder) HandlerOption
- func WithLogStream(hub *logstream.Hub) HandlerOption
- func WithMetrics(m *Metrics) HandlerOption
- func WithNotifier(n Notifier) HandlerOption
- func WithRequestLogStore(s RequestLogStore) HandlerOption
- func WithStore(s Store) HandlerOption
- type IncidentInput
- type IncidentRecorder
- type LoopAlertGate
- type Metrics
- type MetricsSnapshot
- type Notifier
- type PayloadEvent
- type RequestLogStore
- type RequestRecord
- type SettingsRow
- type SettingsStore
- type Store
- type TokenPoint
- type TriggerEvent
Constants ¶
const APIKeyHeader = "X-FinOps-API-Key"
APIKeyHeader is the request header carrying the caller's API key. It is stripped before the request is forwarded upstream (it is proxy-internal).
const AgentIDHeader = "X-FinOps-Agent-Id"
AgentIDHeader is the request header carrying the caller's agent id. It scopes loop history per agent and is stripped before the request is forwarded upstream (it is proxy-internal).
const ProjectHeader = "X-FinOps-Project"
ProjectHeader is the request header carrying the caller's project name. It scopes cost accounting per project and is stripped before the request is forwarded upstream (it is proxy-internal). The empty string is the default bucket when the header is absent.
Variables ¶
var ErrInvalidKey = errors.New("proxy: invalid API key")
ErrInvalidKey is returned when a raw key is empty or its hash matches no caller. Every store failure surfaces as ErrInvalidKey so the middleware always answers 401 (fail-closed) and never leaks whether a caller exists.
var MaxBodyBytes int64 = 10 << 20 // 10 MiB
MaxBodyBytes caps the request body that is read and hashed by the proxy.
Requests whose declared Content-Length exceeds this cap are forwarded untouched (fail-open, the circuit breaker is skipped for them); the cap also backs http.MaxBytesReader for chunked or unknown-length bodies, so a body that overflows it during the read yields a 413 instead of a hash. Tests may lower this variable.
Functions ¶
func DashboardHandler ¶
DashboardHandler serves the embedded developer dashboard. The page drives itself entirely from /v1/metrics, so this handler has no other dependency.
func HashAPIKey ¶
HashAPIKey returns the lowercase hex SHA-256 of raw, the same fingerprint convention as the loop detector. Raw keys are never stored — only this hash.
func MetricsHandler ¶
func MetricsHandler(m *Metrics, det circuitbreaker.Detector) http.Handler
MetricsHandler serves GET /v1/metrics. It takes the detector so it can report the current blocked-agent count via the optional DetectorStatus capability, exactly as the EE control plane does (type-assert at startup, no core change).
func Middleware ¶
func Middleware(a Authenticator, next http.Handler) http.Handler
Middleware authenticates each request via a, strips the X-FinOps-API-Key header, and injects the Caller into the request context before calling next. A nil authenticator is pass-through (fail-open). An authentication failure answers 401 unauthorized and the request never reaches next, so no body is read, hashed, or forwarded for an unauthenticated request.
func NewHandler ¶
func NewHandler(upstream *url.URL, det circuitbreaker.Detector, price pricing.PriceSource, opts ...HandlerOption) http.Handler
NewHandler builds the FinOps circuit-breaker proxy handler for the given upstream. Every non-empty POST body up to MaxBodyBytes is hashed and checked against det before being forwarded to upstream; blocked repeats receive an HTTP 429 and are never forwarded. The DELETE /v1/agent/{id}/history route clears one agent's history via det.Reset.
When price and a request-log store are wired (WithStore), every proxied call is recorded with its computed cost, and every 429 carries the estimated saved cost. When either is absent, all cost recording is skipped and the proxy behaves exactly as it did without the pricing layer.
Optional HandlerOptions configure the alerting layer: WithNotifier enables loop_blocked and budget_level events, WithBudget enables daily-budget monitoring. Omitting them leaves alerting fully disabled.
Types ¶
type Authenticator ¶
Authenticator verifies a raw API key and returns the matching Caller. It is the core-shaped equivalent of the enterprise auth.Authenticator: hash the key, look it up, carry the Caller in the request context.
func NewAuthenticator ¶
func NewAuthenticator(store CallerStore) Authenticator
NewAuthenticator returns an Authenticator that resolves raw keys against store.
func Noop ¶
func Noop() Authenticator
Noop returns an Authenticator that never rejects a request and always yields the zero Caller — "auth disabled". It is the fail-open counterpart to NewAuthenticator.
type BudgetConfig ¶
type BudgetConfig struct {
DailyBudgetMicro int64 // 0 = daily budget disabled (the monitor is inert)
ThresholdPct []int // percentages of the daily budget that fire an event
}
BudgetConfig is the immutable configuration of the global daily budget monitor. Money is integer µUSD.
type BudgetLedger ¶
type BudgetLedger interface {
// SumActualCostSince returns the sum of actual cost (µUSD) for every
// request logged at or after since (Unix epoch seconds).
SumActualCostSince(ctx context.Context, since int64) (int64, error)
// SumActualCostForKeySince returns the sum of actual cost (µUSD) for one
// caller's requests logged at or after since (Unix epoch seconds). The empty
// keyID selects the anonymous/legacy bucket.
SumActualCostForKeySince(ctx context.Context, keyID string, since int64) (int64, error)
}
BudgetLedger is the read seam the handler consults for budget enforcement. SumActualCostSince backs the global daily budget monitor; SumActualCostForKeySince backs the per-caller monthly/daily gates. Both sums cover only non-loop-blocked (actual) spend, the source of truth for enforcement. Implementations MUST be safe for concurrent use.
type BudgetMonitor ¶
type BudgetMonitor struct {
// contains filtered or unexported fields
}
BudgetMonitor emits budget_level events when real daily spend crosses a configured percentage of the daily budget, at most once per threshold per UTC day. It is safe for concurrent use: Spend holds a mutex only for map lookups and integer comparisons, and the emit callback is invoked without any lock.
State is in-memory only — a restart forgets today's fired thresholds, so an already-crossed threshold fires again after the process restarts. That is acceptable for advisory alerts, which are not accounting records.
func NewBudgetMonitor ¶
func NewBudgetMonitor(cfg BudgetConfig) *BudgetMonitor
NewBudgetMonitor builds a BudgetMonitor from a BudgetConfig. Thresholds are sorted ascending and deduplicated; values <= 0 or > 100 are dropped. A zero daily budget or no remaining thresholds makes the monitor inert (Spend is a no-op).
func (*BudgetMonitor) Spend ¶
Spend evaluates the current daily spend against every threshold not yet fired today, emitting a budget_level event per newly crossed threshold. On a UTC day rollover the fired state resets. A threshold is recorded as fired only if emit returns true (so a dropped delivery re-fires on the next Spend).
type BudgetSpender ¶
BudgetSpender is the narrow seam through which the proxy's hot path evaluates the daily budget. Both *BudgetMonitor (the file-only monitor) and the enterprise runtime-reconfigurable dispatcher satisfy it, so the proxy never depends on a concrete type. Spend must be non-blocking and nil-safe.
type Caller ¶
type Caller struct {
ID string // caller id; always non-empty for authenticated requests
Name string // display name; "" for anonymous callers
// MonthlyBudgetMicro is the per-caller monthly spend cap in µUSD; 0 =
// unlimited (the monthly gate is skipped).
MonthlyBudgetMicro int64
// DailyBudgetMicro is the per-caller daily spend cap in µUSD; 0 = unlimited
// (the daily gate is skipped).
DailyBudgetMicro int64
}
Caller is the identified caller of the proxy. Money is integer micro-dollars (µUSD). It is the flat, enterprise-agnostic projection of whatever identity system the caller module layers on top (an EE tenant, an API key, a service account). A budget of 0 on a dimension means unlimited.
type CallerStore ¶
type CallerStore interface {
// CallerByKeyHash returns the Caller for the given key hash. It MUST return
// ErrInvalidKey when the hash is unknown; every other failure is returned
// unwrapped so the caller can decide how to surface it.
CallerByKeyHash(ctx context.Context, hash string) (Caller, error)
}
CallerStore is the narrow persistence seam through which the key authenticator resolves a raw API key to a Caller. *store/sqlite.DB and *store/memory.Store implement it; the enterprise module supplies its own (flattening its multi-tenant hierarchy into a flat Caller).
type CostThresholdReader ¶
type CostThresholdReader interface {
// CostAlertThresholdMicro returns the per-request cost alert threshold in
// integer µUSD. 0 disables per-request cost alerts.
CostAlertThresholdMicro() int64
}
CostThresholdReader is the narrow seam through which the proxy reads the runtime cost-alert threshold on the hot path. The enterprise settings service satisfies it; defining the interface here keeps package proxy free of a settings import.
type Emitter ¶
type Emitter interface {
Emit(PayloadEvent)
}
Emitter is the seam through which the handler publishes raw payloads to an async consumer (the EE semantic worker pool). The contract mirrors Notifier: Emit MUST return promptly (never block on the worker, the network, or a full queue), MUST be safe for concurrent use, and MUST NOT panic. Implementations buffer internally and drop on a full queue.
type Event ¶
type Event struct {
Type EventType `json:"type"`
Timestamp int64 `json:"timestamp"` // Unix epoch seconds (UTC), like RequestRecord.Timestamp
AgentID string `json:"agent_id,omitempty"`
Project string `json:"project,omitempty"`
Model string `json:"model,omitempty"`
// loop_blocked
SavedCostMicro int64 `json:"saved_cost_micro_usd,omitempty"`
Reason string `json:"reason,omitempty"`
RetryAfter int `json:"retry_after,omitempty"`
// budget_level
BudgetMicroUSD int64 `json:"budget_micro_usd,omitempty"`
SpentMicroUSD int64 `json:"spent_micro_usd,omitempty"`
ThresholdPct int `json:"threshold_pct,omitempty"`
// semantic_loop / intent_drift (EE)
Similarity float64 `json:"similarity,omitempty"` // cosine that tripped the loop
IntentScore float64 `json:"intent_score,omitempty"` // cosine to the intent anchor
// quota_level (EE)
QuotaKind string `json:"quota_kind,omitempty"` // "org" | "team" | "user"
QuotaID string `json:"quota_id,omitempty"`
}
Event is the single value the notification layer transports. µUSD fields carry integer micro-dollars; a USD string for display is produced only by the consumer. Zero-valued fields are omitted from the JSON encoding so payloads stay minimal.
type EventType ¶
type EventType string
EventType discriminates the two events the notification layer can carry.
const ( // EventLoopBlocked reports a request the circuit breaker blocked (429). // The carried SavedCostMicro is the estimated µUSD the proxy avoided // spending by not forwarding the request. EventLoopBlocked EventType = "loop_blocked" // EventBudgetLevel reports that real daily spend crossed a configured // threshold percentage of the daily budget. EventBudgetLevel EventType = "budget_level" // EventSemanticLoop reports that the EE semantic worker flagged a // semantically near-duplicate conversation (cosine drift above threshold). // Emitted by the EE worker, never the request path. EventSemanticLoop EventType = "semantic_loop" // EventIntentDrift reports that conversation semantics diverged from the // original system intent (cosine to the intent anchor below the floor). // Emitted by the EE worker. EventIntentDrift EventType = "intent_drift" // EventQuotaLevel reports that an org/team/user hierarchical quota reached a // configured percentage (e.g. 80% or 100%). Emitted by the EE quota monitor. EventQuotaLevel EventType = "quota_level" )
type HandlerOption ¶
type HandlerOption func(*handler)
HandlerOption customizes a handler at construction time. Options are applied in order after the handler struct is built and before the mux is created.
func WithAuth ¶
func WithAuth(a Authenticator) HandlerOption
WithAuth wires an authenticator into the handler. A nil authenticator (or omitting this option) disables authentication: the proxy behaves exactly as it did without the auth layer. When set, NewHandler wraps the whole mux in Middleware, so the proxy path and the DELETE /v1/agent/{id}/history route both require a valid key.
func WithBudget ¶
func WithBudget(cfg BudgetConfig) HandlerOption
WithBudget wires a daily budget monitor built from a BudgetConfig. NewBudgetMonitor returns an inert monitor for a zero budget or empty thresholds, so Spend no-ops either way.
func WithBudgetLedger ¶
func WithBudgetLedger(l BudgetLedger) HandlerOption
WithBudgetLedger wires budget enforcement independently of cost recording.
func WithBudgetSpender ¶
func WithBudgetSpender(s BudgetSpender) HandlerOption
WithBudgetSpender wires a BudgetSpender (a runtime-reconfigurable manager, or any test fake) into the handler's budget seam. It is the preferred wiring when the budget is reloadable at runtime; WithBudget remains for the file-only path and tests.
func WithCostThreshold ¶
func WithCostThreshold(r CostThresholdReader) HandlerOption
WithCostThreshold wires a CostThresholdReader so every completed request whose cost reaches the threshold emits a cost_alert live event. A nil reader (or omitting this option) disables per-request cost alerts. The reader must be safe for concurrent calls from the proxy's hot path.
func WithEmitter ¶
func WithEmitter(e Emitter) HandlerOption
WithEmitter wires an Emitter into the handler. A nil emitter (or omitting the option) disables emission: the request path is byte-for-byte unchanged.
func WithIncidentRecorder ¶
func WithIncidentRecorder(r IncidentRecorder) HandlerOption
WithIncidentRecorder wires durable incident recording. A nil recorder (the default) skips incident persistence; loop-blocked rows still land in the request log.
func WithLogStream ¶
func WithLogStream(hub *logstream.Hub) HandlerOption
WithLogStream wires a live logstream.Hub into the handler so interception events (request.complete, loop_blocked, budget_exceeded, budget_level, cost_alert) are published to the stream. A nil hub (or omitting this option) disables live logging; publishLog is a no-op.
func WithMetrics ¶
func WithMetrics(m *Metrics) HandlerOption
WithMetrics wires a Metrics registry into the handler. A nil registry (or omitting this option) disables collection.
func WithNotifier ¶
func WithNotifier(n Notifier) HandlerOption
WithNotifier wires a Notifier into the handler. A nil notifier (or omitting this option) disables all alerting: emit becomes a no-op.
func WithRequestLogStore ¶
func WithRequestLogStore(s RequestLogStore) HandlerOption
WithRequestLogStore wires cost recording independently of the budget ledger.
func WithStore ¶
func WithStore(s Store) HandlerOption
WithStore wires a combined Store into the handler: it becomes the request logger, the budget ledger, and (if the store also satisfies IncidentRecorder) the incident recorder. This is the primary wiring for the reference stores. Individual concerns can be wired or nil'd separately with WithRequestLogStore, WithBudgetLedger, and WithIncidentRecorder.
type IncidentInput ¶
type IncidentInput struct {
AgentID string
KeyID string
ProjectName string
Model string
Fingerprint string
CostMicro int64 // µUSD would-be cost of this blocked request
}
IncidentInput carries the fields a blocked request contributes to a durable incident record. Fingerprint is the lowercase SHA-256 hex of the request body.
type IncidentRecorder ¶
type IncidentRecorder interface {
RecordIncident(ctx context.Context, in IncidentInput) error
}
IncidentRecorder is the optional seam through which the proxy persists a durable loop episode on each block. A nil recorder (the default for the core standalone binary) skips incident recording entirely — loop-blocked rows are still present in request_logs with IsLoopBlocked set. The enterprise module supplies the implementation backed by its incidents table.
type LoopAlertGate ¶
type LoopAlertGate interface {
LoopAlertsEnabled() bool
}
LoopAlertGate is the narrow seam the proxy's 429 path checks before emitting a loop_blocked event (D9). A Notifier that implements the gate reports its live flag; one that does not (a noop, a test fake, the JSON stdout logger) defaults to enabled. Implementations MUST be safe for concurrent use and MUST NOT block.
type Metrics ¶
type Metrics struct {
// contains filtered or unexported fields
}
Metrics is the zero-dependency, in-memory observability registry behind GET /v1/metrics and the embedded /dashboard. Every counter is an atomic so the hot path records at the cost of a single atomic Add (no lock, no allocation, no goroutine); the only mutexes guard the two bounded history rings. A nil *Metrics makes every mark method a no-op, so a handler built without WithMetrics is byte-for-byte identical to one without observability.
func NewMetrics ¶
func NewMetrics() *Metrics
NewMetrics returns an empty registry stamped with the current time.
func (*Metrics) Snapshot ¶
func (m *Metrics) Snapshot() MetricsSnapshot
Snapshot returns a point-in-time read of every gauge. It is off the hot path (only invoked when /v1/metrics is polled), so the bounded slices it builds are acceptable.
type MetricsSnapshot ¶
type MetricsSnapshot struct {
ActiveRequests int64 `json:"active_requests"`
TotalRequests int64 `json:"total_requests"`
LoopBlocks int64 `json:"loop_blocks"`
BlockedAgents int `json:"blocked_agents"`
SavedCostMicroUSD int64 `json:"saved_cost_micro_usd"`
AvgLatencyMicro int64 `json:"avg_latency_micro"`
TokenVelocity float64 `json:"token_velocity"`
TokenSeries []TokenPoint `json:"token_series"`
RecentTriggers []TriggerEvent `json:"recent_triggers"`
UptimeSeconds int64 `json:"uptime_seconds"`
}
MetricsSnapshot is the JSON shape served at GET /v1/metrics.
type Notifier ¶
type Notifier interface {
// Notify delivers one event. It MUST return promptly (no blocking I/O on
// the caller), MUST be safe for concurrent use, and MUST NOT panic.
// Implementations with slow delivery (network/disk) MUST buffer internally
// and return immediately.
Notify(event Event)
// Close flushes any remaining events within a bounded timeout and then
// stops the worker. It is idempotent.
Close()
}
Notifier is the seam through which the proxy ties into alerting — never a concrete sink, so tests can inject a fake notifier and the handler can stay oblivious to the transport.
func NoopNotifier ¶
func NoopNotifier() Notifier
NoopNotifier returns a Notifier that discards every event. It is the "alerting disabled" default and a convenient test double.
type PayloadEvent ¶
type PayloadEvent struct {
AgentID string
KeyID string
Project string
Model string
Fingerprint string
Body []byte
Timestamp int64
}
PayloadEvent is the raw request payload handed to an async semantic consumer.
Body is the fully-read request body. The handler rewinds a separate copy for the upstream, so the receiver may retain Body; the receiver MUST NOT mutate it (the handler may still be reading from the same backing array during the rewind).
KeyID is the caller/tenant id ("" = anonymous/legacy bucket). Fingerprint is the lowercase SHA-256 hex the detector was handed. Timestamp is Unix epoch seconds (UTC).
type RequestLogStore ¶
type RequestLogStore interface {
LogRequest(ctx context.Context, r RequestRecord) error
}
RequestLogStore is the persistence seam the handler uses to record each proxied (or loop-blocked) request. Implementations MUST be safe for concurrent use and MUST NOT block the request path: the handler treats a LogRequest error as non-fatal.
type RequestRecord ¶
type RequestRecord struct {
AgentID string
KeyID string // caller id; "" = anonymous / legacy bucket
ProjectName string
Model string
PromptTokens int64
CompletionTokens int64
CostUSD int64 // µUSD; saved cost for a loop-blocked row, actual for a forwarded row
IsLoopBlocked bool
Timestamp int64
}
RequestRecord is the public form of one persisted proxy request. It is the enterprise-agnostic row the handler hands to a RequestLogStore; KeyID replaces the EE tenant_id column, so the core never models multi-tenancy. Money is integer micro-dollars (µUSD); Timestamp is Unix epoch seconds (UTC).
type SettingsRow ¶
type SettingsRow struct {
LoopThresholdCount int // first sighting always passes; the count-th repeat blocks
CostAlertThresholdMicro int64 // 0 = per-request cost alerts disabled
}
SettingsRow is the single-row runtime settings snapshot: the loop-detection repeat threshold and the per-request cost-alert threshold. Money is µUSD.
type SettingsStore ¶
type SettingsStore interface {
GetSettings(ctx context.Context) (SettingsRow, error)
UpdateSettings(ctx context.Context, s SettingsRow) error
}
SettingsStore is the persistence seam for the single runtime settings row. The handler does not consume it directly — it reads the live loop threshold through circuitbreaker.DetectorLimitSetter and the cost-alert threshold through CostThresholdReader — but the reference stores and the enterprise settings service both implement it.
type Store ¶
type Store interface {
RequestLogStore
BudgetLedger
SettingsStore
}
Store is the aggregate persistence seam the reference stores (memory, sqlite) implement: request logging, the budget ledger, and settings. The handler consumes it through WithStore; the narrower interfaces let a caller wire or nil each concern independently.
Incident recording is deliberately NOT part of Store: it is an enterprise concern surfaced only through the separate IncidentRecorder seam.
type TokenPoint ¶
TokenPoint is one second of the token-consumption sparkline.
type TriggerEvent ¶
type TriggerEvent struct {
Timestamp int64 `json:"timestamp"`
AgentID string `json:"agent_id,omitempty"`
Model string `json:"model,omitempty"`
Reason string `json:"reason"`
SavedMicro int64 `json:"saved_cost_micro_usd"`
}
TriggerEvent is one recent loop-blocked episode for the trigger-history panel.