proxy

package
v0.0.0-...-48df7e4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 12, 2025 License: MIT Imports: 29 Imported by: 0

README

Proxy Package

This package implements the core proxy functionality:

  • Request forwarding to OpenAI API
  • Authentication and authorization
  • Rate limiting and request validation
  • Response transformation and metadata extraction
  • Streaming support and error handling

Documentation

Overview

Package proxy provides the transparent proxy functionality for the LLM API.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CacheKeyFromRequest

func CacheKeyFromRequest(r *http.Request) string

func CacheKeyFromRequestWithVary

func CacheKeyFromRequestWithVary(r *http.Request, vary string) string

CacheKeyFromRequestWithVary generates a cache key using the specified Vary header from the upstream response. This enables per-response driven cache key generation.

Behavior:

  • When vary is empty or "*", this function returns a key that ignores header values (treated as "no vary" for key generation purposes).
  • Header names parsed from vary are normalized to canonical MIME header names.

func Chain

func Chain(h http.Handler, middleware ...Middleware) http.Handler

Chain applies a series of middleware to a handler

Types

type APIConfig

type APIConfig struct {
	// APIs contains configurations for different API providers
	APIs map[string]*APIProviderConfig `yaml:"apis"`
	// DefaultAPI is the default API provider to use if not specified
	DefaultAPI string `yaml:"default_api"`
}

APIConfig represents the top-level configuration for API proxying

func LoadAPIConfigFromFile

func LoadAPIConfigFromFile(filePath string) (*APIConfig, error)

LoadAPIConfigFromFile loads API configuration from a YAML file

func (*APIConfig) GetProxyConfigForAPI

func (c *APIConfig) GetProxyConfigForAPI(apiName string) (*ProxyConfig, error)

GetProxyConfigForAPI returns a ProxyConfig for the specified API provider

type APIProviderConfig

type APIProviderConfig struct {
	// BaseURL is the target API base URL
	BaseURL string `yaml:"base_url"`
	// AllowedEndpoints is a list of endpoint prefixes that are allowed to be accessed
	AllowedEndpoints []string `yaml:"allowed_endpoints"`
	// AllowedMethods is a list of HTTP methods that are allowed
	AllowedMethods []string `yaml:"allowed_methods"`
	// Timeouts for various operations
	Timeouts TimeoutConfig `yaml:"timeouts"`
	// Connection settings
	Connection ConnectionConfig `yaml:"connection"`
	// ParamWhitelist is a map of parameter names to allowed values
	ParamWhitelist  map[string][]string `yaml:"param_whitelist"`
	AllowedOrigins  []string            `yaml:"allowed_origins"`
	RequiredHeaders []string            `yaml:"required_headers"`
}

APIProviderConfig represents the configuration for a specific API provider

type AuditLogger

type AuditLogger interface {
	// Log records an audit event
	Log(event *audit.Event) error
}

AuditLogger defines the interface for audit event logging

type CacheMetricType

type CacheMetricType int

CacheMetricType represents the kind of cache metric to increment.

const (
	CacheMetricHit CacheMetricType = iota
	CacheMetricMiss
	CacheMetricBypass
	CacheMetricStore
)

type ConnectionConfig

type ConnectionConfig struct {
	// MaxIdleConns is the maximum number of idle connections
	MaxIdleConns int `yaml:"max_idle_conns"`
	// MaxIdleConnsPerHost is the maximum number of idle connections per host
	MaxIdleConnsPerHost int `yaml:"max_idle_conns_per_host"`
}

ConnectionConfig contains connection settings for the proxy

type ErrorResponse

type ErrorResponse struct {
	Error       string `json:"error"`
	Description string `json:"description,omitempty"`
	Code        string `json:"code,omitempty"`
}

ErrorResponse is the standard format for error responses

type Middleware

type Middleware func(http.Handler) http.Handler

Middleware defines a function that wraps an http.Handler

func CircuitBreakerMiddleware

func CircuitBreakerMiddleware(failureThreshold int, cooldown time.Duration, isTransient func(status int) bool) Middleware

CircuitBreakerMiddleware returns a middleware that opens the circuit after N consecutive failures. While open, it returns 503 immediately. After a cooldown, it closes and allows requests again.

func ProjectActiveGuardMiddleware

func ProjectActiveGuardMiddleware(enforceActive bool, checker ProjectActiveChecker, auditLogger AuditLogger) Middleware

ProjectActiveGuardMiddleware creates middleware that enforces project active status If enforceActive is false, the middleware passes through all requests without checking If enforceActive is true, inactive projects receive a 403 Forbidden response

type Project

type Project struct {
	ID            string     `json:"id"`
	Name          string     `json:"name"`
	OpenAIAPIKey  string     `json:"openai_api_key"`
	IsActive      bool       `json:"is_active"`
	DeactivatedAt *time.Time `json:"deactivated_at,omitempty"`
	CreatedAt     time.Time  `json:"created_at"`
	UpdatedAt     time.Time  `json:"updated_at"`
}

Project represents a project for the management API and proxy (copied from database/models.go)

type ProjectActiveChecker

type ProjectActiveChecker interface {
	// GetProjectActive checks if a project is active
	GetProjectActive(ctx context.Context, projectID string) (bool, error)
}

ProjectActiveChecker defines the interface for checking project active status

type ProjectStore

type ProjectStore interface {
	// GetAPIKeyForProject retrieves the API key for a project
	GetAPIKeyForProject(ctx context.Context, projectID string) (string, error)
	// GetProjectActive checks if a project is active
	GetProjectActive(ctx context.Context, projectID string) (bool, error)
	// Management API CRUD
	ListProjects(ctx context.Context) ([]Project, error)
	CreateProject(ctx context.Context, project Project) error
	GetProjectByID(ctx context.Context, projectID string) (Project, error)
	UpdateProject(ctx context.Context, project Project) error
	DeleteProject(ctx context.Context, projectID string) error
}

ProjectStore defines the interface for retrieving and managing project information (extended for management API)

type Proxy

type Proxy interface {
	// Handler returns an http.Handler for the proxy
	Handler() http.Handler

	// Shutdown gracefully shuts down the proxy
	Shutdown(ctx context.Context) error
}

Proxy defines the interface for a transparent HTTP proxy

type ProxyConfig

type ProxyConfig struct {
	// TargetBaseURL is the base URL of the API to proxy to
	TargetBaseURL string

	// AllowedEndpoints is a whitelist of endpoints that can be accessed
	AllowedEndpoints []string

	// AllowedMethods is a whitelist of HTTP methods that can be used
	AllowedMethods []string

	// RequestTimeout is the maximum duration for a complete request
	RequestTimeout time.Duration

	// ResponseHeaderTimeout is the time to wait for response headers
	ResponseHeaderTimeout time.Duration

	// FlushInterval is how often to flush streaming responses
	FlushInterval time.Duration

	// MaxIdleConns is the maximum number of idle connections
	MaxIdleConns int

	// MaxIdleConnsPerHost is the maximum number of idle connections per host
	MaxIdleConnsPerHost int

	// IdleConnTimeout is how long to keep idle connections alive
	IdleConnTimeout time.Duration

	// LogLevel controls the verbosity of logging
	LogLevel string
	// LogFormat controls the log output format (json or console)
	LogFormat string
	// LogFile specifies a file path for logs (stdout if empty)
	LogFile string

	// SetXForwardedFor determines whether to set the X-Forwarded-For header
	SetXForwardedFor bool

	// ParamWhitelist is a map of parameter names to allowed values
	ParamWhitelist map[string][]string

	// AllowedOrigins is a list of allowed CORS origins for this provider
	AllowedOrigins []string

	// RequiredHeaders is a list of required request headers (case-insensitive)
	RequiredHeaders []string

	// Project active guard configuration
	EnforceProjectActive bool // Whether to enforce project active status

	// --- HTTP cache (global, opt-in; set programmatically, not via YAML) ---
	// HTTPCacheEnabled toggles the proxy cache for GET/HEAD based on HTTP semantics
	HTTPCacheEnabled bool
	// HTTPCacheDefaultTTL is used only when upstream allows caching but omits explicit TTL
	HTTPCacheDefaultTTL time.Duration
	// HTTPCacheMaxObjectBytes is a guardrail for maximum cacheable response size
	HTTPCacheMaxObjectBytes int64

	// RedisCacheURL enables Redis-backed cache when non-empty (e.g., redis://localhost:6379/0)
	RedisCacheURL string
	// RedisCacheKeyPrefix allows namespacing cache keys (default: llmproxy:cache:)
	RedisCacheKeyPrefix string
}

ProxyConfig contains configuration for the proxy

func (*ProxyConfig) Validate

func (c *ProxyConfig) Validate() error

Validate checks that the ProxyConfig is valid and returns an error if not.

type ProxyMetrics

type ProxyMetrics struct {
	RequestCount      int64
	ErrorCount        int64
	TotalResponseTime time.Duration
	// Cache metrics (provider-agnostic counters)
	CacheHits   int64 // Cache hits (responses served from cache)
	CacheMisses int64 // Cache misses (responses fetched from upstream)
	CacheBypass int64 // Cache bypassed (e.g., due to authorization)
	CacheStores int64 // Cache stores (responses stored in cache)
	// contains filtered or unexported fields
}

ProxyMetrics tracks proxy usage statistics

type TimeoutConfig

type TimeoutConfig struct {
	// Request is the overall request timeout
	Request time.Duration `yaml:"request"`
	// ResponseHeader is the timeout for receiving response headers
	ResponseHeader time.Duration `yaml:"response_header"`
	// IdleConnection is the timeout for idle connections
	IdleConnection time.Duration `yaml:"idle_connection"`
	// FlushInterval controls how often to flush streaming responses
	FlushInterval time.Duration `yaml:"flush_interval"`
}

TimeoutConfig contains timeout settings for the proxy

type TokenValidator

type TokenValidator interface {
	// ValidateToken validates a token and returns the associated project ID
	ValidateToken(ctx context.Context, token string) (string, error)

	// ValidateTokenWithTracking validates a token, increments its usage, and returns the project ID
	ValidateTokenWithTracking(ctx context.Context, token string) (string, error)
}

TokenValidator defines the interface for token validation

type TransparentProxy

type TransparentProxy struct {
	// contains filtered or unexported fields
}

TransparentProxy implements the Proxy interface for transparent proxying

func NewTransparentProxy

func NewTransparentProxy(config ProxyConfig, validator TokenValidator, store ProjectStore) (*TransparentProxy, error)

NewTransparentProxy creates a new proxy instance with an internally configured logger based on the provided ProxyConfig.

func NewTransparentProxyWithAudit

func NewTransparentProxyWithAudit(config ProxyConfig, validator TokenValidator, store ProjectStore, logger *zap.Logger, auditLogger AuditLogger, obsCfg middleware.ObservabilityConfig) (*TransparentProxy, error)

NewTransparentProxyWithAudit creates a proxy with audit logging capabilities.

func NewTransparentProxyWithLogger

func NewTransparentProxyWithLogger(config ProxyConfig, validator TokenValidator, store ProjectStore, logger *zap.Logger) (*TransparentProxy, error)

NewTransparentProxyWithLogger allows providing a custom logger. If logger is nil a new one is created based on the ProxyConfig.

func NewTransparentProxyWithLoggerAndObservability

func NewTransparentProxyWithLoggerAndObservability(config ProxyConfig, validator TokenValidator, store ProjectStore, logger *zap.Logger, obsCfg middleware.ObservabilityConfig) (*TransparentProxy, error)

NewTransparentProxyWithLoggerAndObservability creates a proxy with observability middleware using an existing logger.

func NewTransparentProxyWithObservability

func NewTransparentProxyWithObservability(config ProxyConfig, validator TokenValidator, store ProjectStore, obsCfg middleware.ObservabilityConfig) (*TransparentProxy, error)

NewTransparentProxyWithObservability creates a new proxy with observability middleware.

func (*TransparentProxy) Cache

func (p *TransparentProxy) Cache() httpCache

Cache returns the HTTP cache instance for management operations. Returns nil if caching is disabled.

func (*TransparentProxy) Handler

func (p *TransparentProxy) Handler() http.Handler

Handler returns the HTTP handler for the proxy

func (*TransparentProxy) LoggingMiddleware

func (p *TransparentProxy) LoggingMiddleware() Middleware

LoggingMiddleware logs request details

func (*TransparentProxy) Metrics

func (p *TransparentProxy) Metrics() *ProxyMetrics

Metrics returns a pointer to the current proxy metrics.

func (*TransparentProxy) MetricsMiddleware

func (p *TransparentProxy) MetricsMiddleware() Middleware

MetricsMiddleware collects metrics about requests

func (*TransparentProxy) SetMetrics

func (p *TransparentProxy) SetMetrics(m *ProxyMetrics)

SetMetrics overwrites the current metrics (primarily for testing).

func (*TransparentProxy) Shutdown

func (p *TransparentProxy) Shutdown(ctx context.Context) error

Shutdown gracefully shuts down the proxy

func (*TransparentProxy) TimeoutMiddleware

func (p *TransparentProxy) TimeoutMiddleware(timeout time.Duration) Middleware

TimeoutMiddleware adds a timeout to requests

func (*TransparentProxy) ValidateRequestMiddleware

func (p *TransparentProxy) ValidateRequestMiddleware() Middleware

ValidateRequestMiddleware validates the incoming request against allowed endpoints and methods

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL