proxy

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 10, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Overview

Package proxy implements the FinOps circuit-breaker reverse proxy: the lean, Apache-2.0 core engine that fingerprints request bodies, rejects looped repeats, and enforces token-budget caps.

It depends only on the sibling core packages (circuitbreaker, pricing, logstream) and a small set of seams declared in this package. Every enterprise concern — multi-tenant hierarchy, SSO/session auth, webhook delivery, dashboard persistence — is pushed behind those seams and supplied by the enterprise module, never imported here.

Index

Constants

View Source
const APIKeyHeader = "X-FinOps-API-Key"

APIKeyHeader is the request header carrying the caller's API key. It is stripped before the request is forwarded upstream (it is proxy-internal).

View Source
const AgentIDHeader = "X-FinOps-Agent-Id"

AgentIDHeader is the request header carrying the caller's agent id. It scopes loop history per agent and is stripped before the request is forwarded upstream (it is proxy-internal).

View Source
const ProjectHeader = "X-FinOps-Project"

ProjectHeader is the request header carrying the caller's project name. It scopes cost accounting per project and is stripped before the request is forwarded upstream (it is proxy-internal). The empty string is the default bucket when the header is absent.

Variables

View Source
var ErrInvalidKey = errors.New("proxy: invalid API key")

ErrInvalidKey is returned when a raw key is empty or its hash matches no caller. Every store failure surfaces as ErrInvalidKey so the middleware always answers 401 (fail-closed) and never leaks whether a caller exists.

View Source
var MaxBodyBytes int64 = 10 << 20 // 10 MiB

MaxBodyBytes caps the request body that is read and hashed by the proxy.

Requests whose declared Content-Length exceeds this cap are forwarded untouched (fail-open, the circuit breaker is skipped for them); the cap also backs http.MaxBytesReader for chunked or unknown-length bodies, so a body that overflows it during the read yields a 413 instead of a hash. Tests may lower this variable.

Functions

func DashboardHandler

func DashboardHandler() http.Handler

DashboardHandler serves the embedded developer dashboard. The page drives itself entirely from /v1/metrics, so this handler has no other dependency.

func HashAPIKey

func HashAPIKey(raw string) string

HashAPIKey returns the lowercase hex SHA-256 of raw, the same fingerprint convention as the loop detector. Raw keys are never stored — only this hash.

func MetricsHandler

func MetricsHandler(m *Metrics, det circuitbreaker.Detector) http.Handler

MetricsHandler serves GET /v1/metrics. It takes the detector so it can report the current blocked-agent count via the optional DetectorStatus capability, exactly as the EE control plane does (type-assert at startup, no core change).

func Middleware

func Middleware(a Authenticator, next http.Handler) http.Handler

Middleware authenticates each request via a, strips the X-FinOps-API-Key header, and injects the Caller into the request context before calling next. A nil authenticator is pass-through (fail-open). An authentication failure answers 401 unauthorized and the request never reaches next, so no body is read, hashed, or forwarded for an unauthenticated request.

func NewHandler

func NewHandler(upstream *url.URL, det circuitbreaker.Detector, price pricing.PriceSource, opts ...HandlerOption) http.Handler

NewHandler builds the FinOps circuit-breaker proxy handler for the given upstream. Every non-empty POST body up to MaxBodyBytes is hashed and checked against det before being forwarded to upstream; blocked repeats receive an HTTP 429 and are never forwarded. The DELETE /v1/agent/{id}/history route clears one agent's history via det.Reset.

When price and a request-log store are wired (WithStore), every proxied call is recorded with its computed cost, and every 429 carries the estimated saved cost. When either is absent, all cost recording is skipped and the proxy behaves exactly as it did without the pricing layer.

Optional HandlerOptions configure the alerting layer: WithNotifier enables loop_blocked and budget_level events, WithBudget enables daily-budget monitoring. Omitting them leaves alerting fully disabled.

func WithCaller

func WithCaller(ctx context.Context, c Caller) context.Context

WithCaller returns a copy of ctx carrying the authenticated caller c.

Types

type Authenticator

type Authenticator interface {
	Authenticate(rawKey string) (Caller, error)
}

Authenticator verifies a raw API key and returns the matching Caller. It is the core-shaped equivalent of the enterprise auth.Authenticator: hash the key, look it up, carry the Caller in the request context.

func NewAuthenticator

func NewAuthenticator(store CallerStore) Authenticator

NewAuthenticator returns an Authenticator that resolves raw keys against store.

func Noop

func Noop() Authenticator

Noop returns an Authenticator that never rejects a request and always yields the zero Caller — "auth disabled". It is the fail-open counterpart to NewAuthenticator.

type BudgetConfig

type BudgetConfig struct {
	DailyBudgetMicro int64 // 0 = daily budget disabled (the monitor is inert)
	ThresholdPct     []int // percentages of the daily budget that fire an event
}

BudgetConfig is the immutable configuration of the global daily budget monitor. Money is integer µUSD.

type BudgetLedger

type BudgetLedger interface {
	// SumActualCostSince returns the sum of actual cost (µUSD) for every
	// request logged at or after since (Unix epoch seconds).
	SumActualCostSince(ctx context.Context, since int64) (int64, error)

	// SumActualCostForKeySince returns the sum of actual cost (µUSD) for one
	// caller's requests logged at or after since (Unix epoch seconds). The empty
	// keyID selects the anonymous/legacy bucket.
	SumActualCostForKeySince(ctx context.Context, keyID string, since int64) (int64, error)
}

BudgetLedger is the read seam the handler consults for budget enforcement. SumActualCostSince backs the global daily budget monitor; SumActualCostForKeySince backs the per-caller monthly/daily gates. Both sums cover only non-loop-blocked (actual) spend, the source of truth for enforcement. Implementations MUST be safe for concurrent use.

type BudgetMonitor

type BudgetMonitor struct {
	// contains filtered or unexported fields
}

BudgetMonitor emits budget_level events when real daily spend crosses a configured percentage of the daily budget, at most once per threshold per UTC day. It is safe for concurrent use: Spend holds a mutex only for map lookups and integer comparisons, and the emit callback is invoked without any lock.

State is in-memory only — a restart forgets today's fired thresholds, so an already-crossed threshold fires again after the process restarts. That is acceptable for advisory alerts, which are not accounting records.

func NewBudgetMonitor

func NewBudgetMonitor(cfg BudgetConfig) *BudgetMonitor

NewBudgetMonitor builds a BudgetMonitor from a BudgetConfig. Thresholds are sorted ascending and deduplicated; values <= 0 or > 100 are dropped. A zero daily budget or no remaining thresholds makes the monitor inert (Spend is a no-op).

func (*BudgetMonitor) Spend

func (m *BudgetMonitor) Spend(spentMicro int64, now time.Time, emit func(Event) bool)

Spend evaluates the current daily spend against every threshold not yet fired today, emitting a budget_level event per newly crossed threshold. On a UTC day rollover the fired state resets. A threshold is recorded as fired only if emit returns true (so a dropped delivery re-fires on the next Spend).

type BudgetSpender

type BudgetSpender interface {
	Spend(spentMicro int64, now time.Time, emit func(Event) bool)
}

BudgetSpender is the narrow seam through which the proxy's hot path evaluates the daily budget. Both *BudgetMonitor (the file-only monitor) and the enterprise runtime-reconfigurable dispatcher satisfy it, so the proxy never depends on a concrete type. Spend must be non-blocking and nil-safe.

type Caller

type Caller struct {
	ID   string // caller id; always non-empty for authenticated requests
	Name string // display name; "" for anonymous callers

	// MonthlyBudgetMicro is the per-caller monthly spend cap in µUSD; 0 =
	// unlimited (the monthly gate is skipped).
	MonthlyBudgetMicro int64

	// DailyBudgetMicro is the per-caller daily spend cap in µUSD; 0 = unlimited
	// (the daily gate is skipped).
	DailyBudgetMicro int64
}

Caller is the identified caller of the proxy. Money is integer micro-dollars (µUSD). It is the flat, enterprise-agnostic projection of whatever identity system the caller module layers on top (an EE tenant, an API key, a service account). A budget of 0 on a dimension means unlimited.

func CallerFrom

func CallerFrom(ctx context.Context) (Caller, bool)

CallerFrom extracts the authenticated caller from ctx. The bool is false when no caller is present (auth disabled, or the request never passed through Middleware).

type CallerStore

type CallerStore interface {
	// CallerByKeyHash returns the Caller for the given key hash. It MUST return
	// ErrInvalidKey when the hash is unknown; every other failure is returned
	// unwrapped so the caller can decide how to surface it.
	CallerByKeyHash(ctx context.Context, hash string) (Caller, error)
}

CallerStore is the narrow persistence seam through which the key authenticator resolves a raw API key to a Caller. *store/sqlite.DB and *store/memory.Store implement it; the enterprise module supplies its own (flattening its multi-tenant hierarchy into a flat Caller).

type CostThresholdReader

type CostThresholdReader interface {
	// CostAlertThresholdMicro returns the per-request cost alert threshold in
	// integer µUSD. 0 disables per-request cost alerts.
	CostAlertThresholdMicro() int64
}

CostThresholdReader is the narrow seam through which the proxy reads the runtime cost-alert threshold on the hot path. The enterprise settings service satisfies it; defining the interface here keeps package proxy free of a settings import.

type Emitter

type Emitter interface {
	Emit(PayloadEvent)
}

Emitter is the seam through which the handler publishes raw payloads to an async consumer (the EE semantic worker pool). The contract mirrors Notifier: Emit MUST return promptly (never block on the worker, the network, or a full queue), MUST be safe for concurrent use, and MUST NOT panic. Implementations buffer internally and drop on a full queue.

type Event

type Event struct {
	Type      EventType `json:"type"`
	Timestamp int64     `json:"timestamp"` // Unix epoch seconds (UTC), like RequestRecord.Timestamp

	AgentID string `json:"agent_id,omitempty"`
	Project string `json:"project,omitempty"`
	Model   string `json:"model,omitempty"`

	// loop_blocked
	SavedCostMicro int64  `json:"saved_cost_micro_usd,omitempty"`
	Reason         string `json:"reason,omitempty"`
	RetryAfter     int    `json:"retry_after,omitempty"`

	// budget_level
	BudgetMicroUSD int64 `json:"budget_micro_usd,omitempty"`
	SpentMicroUSD  int64 `json:"spent_micro_usd,omitempty"`
	ThresholdPct   int   `json:"threshold_pct,omitempty"`

	// semantic_loop / intent_drift (EE)
	Similarity  float64 `json:"similarity,omitempty"`   // cosine that tripped the loop
	IntentScore float64 `json:"intent_score,omitempty"` // cosine to the intent anchor

	// quota_level (EE)
	QuotaKind string `json:"quota_kind,omitempty"` // "org" | "team" | "user"
	QuotaID   string `json:"quota_id,omitempty"`
}

Event is the single value the notification layer transports. µUSD fields carry integer micro-dollars; a USD string for display is produced only by the consumer. Zero-valued fields are omitted from the JSON encoding so payloads stay minimal.

type EventType

type EventType string

EventType discriminates the two events the notification layer can carry.

const (
	// EventLoopBlocked reports a request the circuit breaker blocked (429).
	// The carried SavedCostMicro is the estimated µUSD the proxy avoided
	// spending by not forwarding the request.
	EventLoopBlocked EventType = "loop_blocked"

	// EventBudgetLevel reports that real daily spend crossed a configured
	// threshold percentage of the daily budget.
	EventBudgetLevel EventType = "budget_level"

	// EventSemanticLoop reports that the EE semantic worker flagged a
	// semantically near-duplicate conversation (cosine drift above threshold).
	// Emitted by the EE worker, never the request path.
	EventSemanticLoop EventType = "semantic_loop"

	// EventIntentDrift reports that conversation semantics diverged from the
	// original system intent (cosine to the intent anchor below the floor).
	// Emitted by the EE worker.
	EventIntentDrift EventType = "intent_drift"

	// EventQuotaLevel reports that an org/team/user hierarchical quota reached a
	// configured percentage (e.g. 80% or 100%). Emitted by the EE quota monitor.
	EventQuotaLevel EventType = "quota_level"
)

type HandlerOption

type HandlerOption func(*handler)

HandlerOption customizes a handler at construction time. Options are applied in order after the handler struct is built and before the mux is created.

func WithAuth

func WithAuth(a Authenticator) HandlerOption

WithAuth wires an authenticator into the handler. A nil authenticator (or omitting this option) disables authentication: the proxy behaves exactly as it did without the auth layer. When set, NewHandler wraps the whole mux in Middleware, so the proxy path and the DELETE /v1/agent/{id}/history route both require a valid key.

func WithBudget

func WithBudget(cfg BudgetConfig) HandlerOption

WithBudget wires a daily budget monitor built from a BudgetConfig. NewBudgetMonitor returns an inert monitor for a zero budget or empty thresholds, so Spend no-ops either way.

func WithBudgetLedger

func WithBudgetLedger(l BudgetLedger) HandlerOption

WithBudgetLedger wires budget enforcement independently of cost recording.

func WithBudgetSpender

func WithBudgetSpender(s BudgetSpender) HandlerOption

WithBudgetSpender wires a BudgetSpender (a runtime-reconfigurable manager, or any test fake) into the handler's budget seam. It is the preferred wiring when the budget is reloadable at runtime; WithBudget remains for the file-only path and tests.

func WithCostThreshold

func WithCostThreshold(r CostThresholdReader) HandlerOption

WithCostThreshold wires a CostThresholdReader so every completed request whose cost reaches the threshold emits a cost_alert live event. A nil reader (or omitting this option) disables per-request cost alerts. The reader must be safe for concurrent calls from the proxy's hot path.

func WithEmitter

func WithEmitter(e Emitter) HandlerOption

WithEmitter wires an Emitter into the handler. A nil emitter (or omitting the option) disables emission: the request path is byte-for-byte unchanged.

func WithIncidentRecorder

func WithIncidentRecorder(r IncidentRecorder) HandlerOption

WithIncidentRecorder wires durable incident recording. A nil recorder (the default) skips incident persistence; loop-blocked rows still land in the request log.

func WithLogStream

func WithLogStream(hub *logstream.Hub) HandlerOption

WithLogStream wires a live logstream.Hub into the handler so interception events (request.complete, loop_blocked, budget_exceeded, budget_level, cost_alert) are published to the stream. A nil hub (or omitting this option) disables live logging; publishLog is a no-op.

func WithMetrics

func WithMetrics(m *Metrics) HandlerOption

WithMetrics wires a Metrics registry into the handler. A nil registry (or omitting this option) disables collection.

func WithNotifier

func WithNotifier(n Notifier) HandlerOption

WithNotifier wires a Notifier into the handler. A nil notifier (or omitting this option) disables all alerting: emit becomes a no-op.

func WithRequestLogStore

func WithRequestLogStore(s RequestLogStore) HandlerOption

WithRequestLogStore wires cost recording independently of the budget ledger.

func WithStore

func WithStore(s Store) HandlerOption

WithStore wires a combined Store into the handler: it becomes the request logger, the budget ledger, and (if the store also satisfies IncidentRecorder) the incident recorder. This is the primary wiring for the reference stores. Individual concerns can be wired or nil'd separately with WithRequestLogStore, WithBudgetLedger, and WithIncidentRecorder.

type IncidentInput

type IncidentInput struct {
	AgentID     string
	KeyID       string
	ProjectName string
	Model       string
	Fingerprint string
	CostMicro   int64 // µUSD would-be cost of this blocked request
}

IncidentInput carries the fields a blocked request contributes to a durable incident record. Fingerprint is the lowercase SHA-256 hex of the request body.

type IncidentRecorder

type IncidentRecorder interface {
	RecordIncident(ctx context.Context, in IncidentInput) error
}

IncidentRecorder is the optional seam through which the proxy persists a durable loop episode on each block. A nil recorder (the default for the core standalone binary) skips incident recording entirely — loop-blocked rows are still present in request_logs with IsLoopBlocked set. The enterprise module supplies the implementation backed by its incidents table.

type LoopAlertGate

type LoopAlertGate interface {
	LoopAlertsEnabled() bool
}

LoopAlertGate is the narrow seam the proxy's 429 path checks before emitting a loop_blocked event (D9). A Notifier that implements the gate reports its live flag; one that does not (a noop, a test fake, the JSON stdout logger) defaults to enabled. Implementations MUST be safe for concurrent use and MUST NOT block.

type Metrics

type Metrics struct {
	// contains filtered or unexported fields
}

Metrics is the zero-dependency, in-memory observability registry behind GET /v1/metrics and the embedded /dashboard. Every counter is an atomic so the hot path records at the cost of a single atomic Add (no lock, no allocation, no goroutine); the only mutexes guard the two bounded history rings. A nil *Metrics makes every mark method a no-op, so a handler built without WithMetrics is byte-for-byte identical to one without observability.

func NewMetrics

func NewMetrics() *Metrics

NewMetrics returns an empty registry stamped with the current time.

func (*Metrics) Snapshot

func (m *Metrics) Snapshot() MetricsSnapshot

Snapshot returns a point-in-time read of every gauge. It is off the hot path (only invoked when /v1/metrics is polled), so the bounded slices it builds are acceptable.

type MetricsSnapshot

type MetricsSnapshot struct {
	ActiveRequests    int64          `json:"active_requests"`
	TotalRequests     int64          `json:"total_requests"`
	LoopBlocks        int64          `json:"loop_blocks"`
	BlockedAgents     int            `json:"blocked_agents"`
	SavedCostMicroUSD int64          `json:"saved_cost_micro_usd"`
	AvgLatencyMicro   int64          `json:"avg_latency_micro"`
	TokenVelocity     float64        `json:"token_velocity"`
	TokenSeries       []TokenPoint   `json:"token_series"`
	RecentTriggers    []TriggerEvent `json:"recent_triggers"`
	UptimeSeconds     int64          `json:"uptime_seconds"`
}

MetricsSnapshot is the JSON shape served at GET /v1/metrics.

type Notifier

type Notifier interface {
	// Notify delivers one event. It MUST return promptly (no blocking I/O on
	// the caller), MUST be safe for concurrent use, and MUST NOT panic.
	// Implementations with slow delivery (network/disk) MUST buffer internally
	// and return immediately.
	Notify(event Event)
	// Close flushes any remaining events within a bounded timeout and then
	// stops the worker. It is idempotent.
	Close()
}

Notifier is the seam through which the proxy ties into alerting — never a concrete sink, so tests can inject a fake notifier and the handler can stay oblivious to the transport.

func NoopNotifier

func NoopNotifier() Notifier

NoopNotifier returns a Notifier that discards every event. It is the "alerting disabled" default and a convenient test double.

type PayloadEvent

type PayloadEvent struct {
	AgentID     string
	KeyID       string
	Project     string
	Model       string
	Fingerprint string
	Body        []byte
	Timestamp   int64
}

PayloadEvent is the raw request payload handed to an async semantic consumer.

Body is the fully-read request body. The handler rewinds a separate copy for the upstream, so the receiver may retain Body; the receiver MUST NOT mutate it (the handler may still be reading from the same backing array during the rewind).

KeyID is the caller/tenant id ("" = anonymous/legacy bucket). Fingerprint is the lowercase SHA-256 hex the detector was handed. Timestamp is Unix epoch seconds (UTC).

type RequestLogStore

type RequestLogStore interface {
	LogRequest(ctx context.Context, r RequestRecord) error
}

RequestLogStore is the persistence seam the handler uses to record each proxied (or loop-blocked) request. Implementations MUST be safe for concurrent use and MUST NOT block the request path: the handler treats a LogRequest error as non-fatal.

type RequestRecord

type RequestRecord struct {
	AgentID          string
	KeyID            string // caller id; "" = anonymous / legacy bucket
	ProjectName      string
	Model            string
	PromptTokens     int64
	CompletionTokens int64
	CostUSD          int64 // µUSD; saved cost for a loop-blocked row, actual for a forwarded row
	IsLoopBlocked    bool
	Timestamp        int64
}

RequestRecord is the public form of one persisted proxy request. It is the enterprise-agnostic row the handler hands to a RequestLogStore; KeyID replaces the EE tenant_id column, so the core never models multi-tenancy. Money is integer micro-dollars (µUSD); Timestamp is Unix epoch seconds (UTC).

type SettingsRow

type SettingsRow struct {
	LoopThresholdCount      int   // first sighting always passes; the count-th repeat blocks
	CostAlertThresholdMicro int64 // 0 = per-request cost alerts disabled
}

SettingsRow is the single-row runtime settings snapshot: the loop-detection repeat threshold and the per-request cost-alert threshold. Money is µUSD.

type SettingsStore

type SettingsStore interface {
	GetSettings(ctx context.Context) (SettingsRow, error)
	UpdateSettings(ctx context.Context, s SettingsRow) error
}

SettingsStore is the persistence seam for the single runtime settings row. The handler does not consume it directly — it reads the live loop threshold through circuitbreaker.DetectorLimitSetter and the cost-alert threshold through CostThresholdReader — but the reference stores and the enterprise settings service both implement it.

type Store

type Store interface {
	RequestLogStore
	BudgetLedger
	SettingsStore
}

Store is the aggregate persistence seam the reference stores (memory, sqlite) implement: request logging, the budget ledger, and settings. The handler consumes it through WithStore; the narrower interfaces let a caller wire or nil each concern independently.

Incident recording is deliberately NOT part of Store: it is an enterprise concern surfaced only through the separate IncidentRecorder seam.

type TokenPoint

type TokenPoint struct {
	Second int64 `json:"second"`
	Tokens int64 `json:"tokens"`
}

TokenPoint is one second of the token-consumption sparkline.

type TriggerEvent

type TriggerEvent struct {
	Timestamp  int64  `json:"timestamp"`
	AgentID    string `json:"agent_id,omitempty"`
	Model      string `json:"model,omitempty"`
	Reason     string `json:"reason"`
	SavedMicro int64  `json:"saved_cost_micro_usd"`
}

TriggerEvent is one recent loop-blocked episode for the trigger-history panel.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL