backend

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 5, 2026 License: MIT Imports: 17 Imported by: 0

Documentation

Overview

Package backend provides pluggable interfaces for OmniProxy storage and caching.

Package backend provides pluggable interfaces for OmniProxy storage and caching.

OmniProxy supports three deployment modes with different backend implementations:

  • Laptop Mode: File-based traffic storage, in-memory cert cache
  • Team Mode: SQLite database, in-memory cert cache
  • Production Mode: Kafka traffic pipeline, Redis cert cache, PostgreSQL config

The same OmniProxy binary works in all modes - just configure which backends to use.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AsyncConfig

type AsyncConfig struct {
	// QueueSize is the buffer size for pending records (default: 10000).
	QueueSize int

	// BatchSize is the number of records to batch before writing (default: 100).
	BatchSize int

	// FlushPeriod is how often to flush partial batches (default: 100ms).
	FlushPeriod time.Duration

	// Workers is the number of concurrent workers (default: 2).
	Workers int

	// Metrics for observability (optional).
	Metrics Metrics
}

AsyncConfig configures the async wrapper.

func DefaultAsyncConfig

func DefaultAsyncConfig() *AsyncConfig

DefaultAsyncConfig returns default async configuration.

type AsyncTrafficStore

type AsyncTrafficStore interface {
	TrafficStore

	// QueueDepth returns the current number of records waiting to be stored.
	QueueDepth() int

	// Flush blocks until all queued records are stored.
	Flush(ctx context.Context) error
}

AsyncTrafficStore wraps a TrafficStore with async buffered writes. This is used in production mode to prevent blocking the proxy on storage.

type AsyncTrafficStoreWrapper

type AsyncTrafficStoreWrapper struct {
	// contains filtered or unexported fields
}

AsyncTrafficStoreWrapper wraps a TrafficStore with async buffered writes. This prevents the proxy from blocking on storage operations.

func NewAsyncTrafficStore

func NewAsyncTrafficStore(store TrafficStore, cfg *AsyncConfig) *AsyncTrafficStoreWrapper

NewAsyncTrafficStore wraps a TrafficStore with async buffered writes.

func (*AsyncTrafficStoreWrapper) Close

func (w *AsyncTrafficStoreWrapper) Close() error

Close stops all workers and flushes remaining records.

func (*AsyncTrafficStoreWrapper) Flush

Flush blocks until all queued records are stored.

func (*AsyncTrafficStoreWrapper) Handle

func (w *AsyncTrafficStoreWrapper) Handle(rec *capture.Record)

Handle implements capture.Handler for use with the capturer.

func (*AsyncTrafficStoreWrapper) QueueDepth

func (w *AsyncTrafficStoreWrapper) QueueDepth() int

QueueDepth returns the current number of records waiting to be stored.

func (*AsyncTrafficStoreWrapper) Store

Store queues a record for async storage. This method is non-blocking unless the queue is full.

func (*AsyncTrafficStoreWrapper) StoreBatch

func (w *AsyncTrafficStoreWrapper) StoreBatch(ctx context.Context, recs []*capture.Record) error

StoreBatch queues multiple records for async storage.

type CertCache

type CertCache interface {
	// Get retrieves a certificate for the given hostname.
	// Returns nil, false if not found or expired.
	Get(host string) (*tls.Certificate, bool)

	// Set stores a certificate for the given hostname.
	Set(host string, cert *tls.Certificate)

	// Delete removes a certificate from the cache.
	Delete(host string)

	// Clear removes all certificates from the cache.
	Clear()

	// Close releases any resources held by the cache.
	Close() error
}

CertCache is the interface for caching generated TLS certificates. Implementations must be safe for concurrent use.

type ConfigStore

type ConfigStore interface {
	// GetProxyConfig retrieves configuration for a proxy instance.
	GetProxyConfig(ctx context.Context, proxyID string) (*ProxyConfig, error)

	// SaveProxyConfig stores configuration for a proxy instance.
	SaveProxyConfig(ctx context.Context, cfg *ProxyConfig) error

	// ListProxyConfigs returns all proxy configurations for an org.
	ListProxyConfigs(ctx context.Context, orgID string) ([]*ProxyConfig, error)

	// DeleteProxyConfig removes a proxy configuration.
	DeleteProxyConfig(ctx context.Context, proxyID string) error

	// Close releases any resources held by the store.
	Close() error
}

ConfigStore is the interface for storing proxy configuration. Used in team and production modes for persistent configuration.

type DBConfig

type DBConfig struct {
	// Type is the database type (sqlite, postgres).
	Type DBType

	// DriverName is the driver name for sql.Open().
	DriverName string

	// DSN is the data source name for sql.Open().
	DSN string

	// Path is the file path (for SQLite only).
	Path string

	// Host is the database host (for PostgreSQL only).
	Host string

	// Port is the database port (for PostgreSQL only).
	Port string

	// Database is the database name (for PostgreSQL only).
	Database string

	// User is the database user (for PostgreSQL only).
	User string

	// Password is the database password (for PostgreSQL only).
	Password string

	// SSLMode is the SSL mode (for PostgreSQL only).
	SSLMode string
}

DBConfig holds parsed database configuration.

func ParseDatabaseURL

func ParseDatabaseURL(rawURL string) (*DBConfig, error)

ParseDatabaseURL parses a database URL into a DBConfig.

Supported formats:

  • sqlite://path/to/file.db
  • sqlite:///absolute/path/to/file.db
  • sqlite::memory: (in-memory database)
  • postgres://user:password@host:port/database?sslmode=disable
  • postgresql://user:password@host:port/database

func (*DBConfig) String

func (c *DBConfig) String() string

String returns a safe string representation (without password).

type DBType

type DBType string

DBType represents a database type.

const (
	DBTypeSQLite   DBType = "sqlite"
	DBTypePostgres DBType = "postgres"
)

type DatabaseTrafficStore

type DatabaseTrafficStore struct {
	// contains filtered or unexported fields
}

DatabaseTrafficStore stores traffic records in a database using Ent. Supports SQLite (laptop/team mode) and PostgreSQL (production mode).

func NewDatabaseTrafficStore

func NewDatabaseTrafficStore(ctx context.Context, cfg *DatabaseTrafficStoreConfig) (*DatabaseTrafficStore, error)

NewDatabaseTrafficStore creates a new database-backed traffic store.

func (*DatabaseTrafficStore) Client

func (s *DatabaseTrafficStore) Client() *ent.Client

Client returns the underlying Ent client for advanced queries.

func (*DatabaseTrafficStore) Close

func (s *DatabaseTrafficStore) Close() error

Close closes the database connection.

func (*DatabaseTrafficStore) Count

func (s *DatabaseTrafficStore) Count(ctx context.Context, filter *TrafficFilter) (int64, error)

Count returns the number of records matching the filter.

func (*DatabaseTrafficStore) GetByID

GetByID returns full traffic details for a single record.

func (*DatabaseTrafficStore) ProxyID

func (s *DatabaseTrafficStore) ProxyID() int

ProxyID returns the proxy ID used by this store.

func (*DatabaseTrafficStore) Query

Query returns traffic records matching the filter.

func (*DatabaseTrafficStore) Stats

Stats returns aggregate statistics for traffic matching the filter.

func (*DatabaseTrafficStore) Store

Store saves a single traffic record to the database.

func (*DatabaseTrafficStore) StoreBatch

func (s *DatabaseTrafficStore) StoreBatch(ctx context.Context, recs []*capture.Record) error

StoreBatch saves multiple traffic records efficiently using bulk insert.

type DatabaseTrafficStoreConfig

type DatabaseTrafficStoreConfig struct {
	// DatabaseURL is the database connection URL.
	// Formats:
	//   - sqlite://path/to/file.db
	//   - postgres://user:password@host:port/database
	DatabaseURL string

	// ProxyID is the ID of the proxy to associate traffic with.
	// If 0, a default proxy will be created/used.
	ProxyID int

	// ProxyName is the name for the default proxy (if ProxyID is 0).
	ProxyName string

	// Metrics for observability (optional).
	Metrics Metrics

	// Debug enables SQL query logging.
	Debug bool
}

DatabaseTrafficStoreConfig configures the database traffic store.

type DiscardTrafficStore

type DiscardTrafficStore struct{}

DiscardTrafficStore is a TrafficStore that discards all records. Useful for testing or when traffic capture is disabled.

func (DiscardTrafficStore) Close

func (DiscardTrafficStore) Close() error

func (DiscardTrafficStore) Store

func (DiscardTrafficStore) StoreBatch

func (DiscardTrafficStore) StoreBatch(ctx context.Context, recs []*capture.Record) error

type FileTrafficStore

type FileTrafficStore struct {
	// contains filtered or unexported fields
}

FileTrafficStore writes traffic records to a file in NDJSON format. This is the simplest backend, suitable for laptop mode.

func NewFileTrafficStore

func NewFileTrafficStore(cfg *FileTrafficStoreConfig) (*FileTrafficStore, error)

NewFileTrafficStore creates a new file-based traffic store.

func (*FileTrafficStore) Close

func (s *FileTrafficStore) Close() error

Close closes the file if one was opened.

func (*FileTrafficStore) Handle

func (s *FileTrafficStore) Handle(rec *capture.Record)

Handle implements capture.Handler for use with the capturer.

func (*FileTrafficStore) Store

func (s *FileTrafficStore) Store(ctx context.Context, rec *capture.Record) error

Store writes a single traffic record to the file.

func (*FileTrafficStore) StoreBatch

func (s *FileTrafficStore) StoreBatch(ctx context.Context, recs []*capture.Record) error

StoreBatch writes multiple traffic records to the file.

type FileTrafficStoreConfig

type FileTrafficStoreConfig struct {
	// Path is the file path to write to. Empty means stdout.
	Path string

	// Writer is an alternative to Path for custom output destinations.
	// If set, Path is ignored.
	Writer io.Writer

	// Format specifies the output format (default: ndjson).
	Format Format

	// Metrics for observability (optional).
	Metrics Metrics
}

FileTrafficStoreConfig configures a FileTrafficStore.

type Format

type Format string

Format specifies the output format for file-based storage.

const (
	FormatNDJSON Format = "ndjson" // Newline-delimited JSON (default)
	FormatJSON   Format = "json"   // Pretty-printed JSON
)

type LRUCertCache

type LRUCertCache struct {
	// contains filtered or unexported fields
}

LRUCertCache is a memory cache with LRU eviction. Useful when memory is constrained.

func NewLRUCertCache

func NewLRUCertCache(cfg *LRUCertCacheConfig) *LRUCertCache

NewLRUCertCache creates a new LRU certificate cache.

func (*LRUCertCache) Clear

func (c *LRUCertCache) Clear()

Clear removes all certificates from the cache.

func (*LRUCertCache) Close

func (c *LRUCertCache) Close() error

Close releases resources.

func (*LRUCertCache) Delete

func (c *LRUCertCache) Delete(host string)

Delete removes a certificate from the cache.

func (*LRUCertCache) Get

func (c *LRUCertCache) Get(host string) (*tls.Certificate, bool)

Get retrieves a certificate for the given hostname.

func (*LRUCertCache) Set

func (c *LRUCertCache) Set(host string, cert *tls.Certificate)

Set stores a certificate for the given hostname.

func (*LRUCertCache) Size

func (c *LRUCertCache) Size() int

Size returns the number of certificates in the cache.

type LRUCertCacheConfig

type LRUCertCacheConfig struct {
	// Capacity is the maximum number of certificates to cache.
	Capacity int

	// Metrics for observability (optional).
	Metrics Metrics
}

LRUCertCacheConfig configures an LRUCertCache.

type MemoryCertCache

type MemoryCertCache struct {
	// contains filtered or unexported fields
}

MemoryCertCache is an in-memory certificate cache. Suitable for laptop and team modes where all traffic goes through one process.

func NewMemoryCertCache

func NewMemoryCertCache(cfg *MemoryCertCacheConfig) *MemoryCertCache

NewMemoryCertCache creates a new in-memory certificate cache.

func (*MemoryCertCache) Clear

func (c *MemoryCertCache) Clear()

Clear removes all certificates from the cache.

func (*MemoryCertCache) Close

func (c *MemoryCertCache) Close() error

Close stops the cleanup goroutine and releases resources.

func (*MemoryCertCache) Delete

func (c *MemoryCertCache) Delete(host string)

Delete removes a certificate from the cache.

func (*MemoryCertCache) Get

func (c *MemoryCertCache) Get(host string) (*tls.Certificate, bool)

Get retrieves a certificate for the given hostname.

func (*MemoryCertCache) Set

func (c *MemoryCertCache) Set(host string, cert *tls.Certificate)

Set stores a certificate for the given hostname.

func (*MemoryCertCache) Size

func (c *MemoryCertCache) Size() int

Size returns the number of certificates in the cache.

type MemoryCertCacheConfig

type MemoryCertCacheConfig struct {
	// TTL is how long certificates are cached (default: 1 hour).
	TTL time.Duration

	// CleanupInterval is how often expired entries are removed (default: 5 minutes).
	CleanupInterval time.Duration

	// Metrics for observability (optional).
	Metrics Metrics
}

MemoryCertCacheConfig configures a MemoryCertCache.

type Metrics

type Metrics interface {
	// IncStoreSuccess increments successful store counter.
	IncStoreSuccess()

	// IncStoreError increments store error counter.
	IncStoreError()

	// ObserveStoreDuration records store operation duration.
	ObserveStoreDuration(d time.Duration)

	// IncCacheHit increments cache hit counter.
	IncCacheHit()

	// IncCacheMiss increments cache miss counter.
	IncCacheMiss()

	// SetQueueDepth sets the current queue depth gauge.
	SetQueueDepth(n int)
}

Metrics provides observability for backend operations.

type NoopMetrics

type NoopMetrics struct{}

NoopMetrics is a Metrics implementation that does nothing.

func (NoopMetrics) IncCacheHit

func (NoopMetrics) IncCacheHit()

func (NoopMetrics) IncCacheMiss

func (NoopMetrics) IncCacheMiss()

func (NoopMetrics) IncStoreError

func (NoopMetrics) IncStoreError()

func (NoopMetrics) IncStoreSuccess

func (NoopMetrics) IncStoreSuccess()

func (NoopMetrics) ObserveStoreDuration

func (NoopMetrics) ObserveStoreDuration(time.Duration)

func (NoopMetrics) SetQueueDepth

func (NoopMetrics) SetQueueDepth(int)

type ProxyConfig

type ProxyConfig struct {
	ID           string
	OrgID        string
	Name         string
	Slug         string
	Mode         string // "forward", "reverse", "transparent"
	Port         int
	Host         string
	MITMEnabled  bool
	SkipHosts    []string
	IncludeHosts []string
	ExcludeHosts []string
	IncludePaths []string
	ExcludePaths []string
	Upstream     string
	SkipBinary   bool
	Active       bool
	CreatedAt    time.Time
	UpdatedAt    time.Time
}

ProxyConfig represents stored proxy configuration.

type SamplingConfig

type SamplingConfig struct {
	// SampleRate is the fraction of requests to capture (0.0 to 1.0).
	SampleRate float64

	// AlwaysCapture is a list of hosts to always capture (overrides SampleRate).
	AlwaysCapture []string

	// NeverCapture is a list of hosts to never capture.
	NeverCapture []string
}

SamplingConfig configures traffic sampling.

type SamplingTrafficStore

type SamplingTrafficStore struct {
	// contains filtered or unexported fields
}

SamplingTrafficStore wraps a TrafficStore with sampling support. Used for high-volume deployments where capturing every request is not feasible.

func NewSamplingTrafficStore

func NewSamplingTrafficStore(store TrafficStore, cfg *SamplingConfig) *SamplingTrafficStore

NewSamplingTrafficStore wraps a TrafficStore with sampling.

func (*SamplingTrafficStore) Close

func (s *SamplingTrafficStore) Close() error

Close closes the underlying store.

func (*SamplingTrafficStore) Store

Store samples and potentially stores a record.

func (*SamplingTrafficStore) StoreBatch

func (s *SamplingTrafficStore) StoreBatch(ctx context.Context, recs []*capture.Record) error

StoreBatch samples and stores multiple records.

type TrafficDetail

type TrafficDetail struct {
	TrafficRecord

	// Request details
	Scheme             string              `json:"scheme"`
	Query              string              `json:"query,omitempty"`
	RequestHeaders     map[string][]string `json:"request_headers,omitempty"`
	RequestBody        string              `json:"request_body,omitempty"`
	RequestBodySize    int64               `json:"request_body_size"`
	RequestIsBinary    bool                `json:"request_is_binary"`
	RequestContentType string              `json:"request_content_type,omitempty"`

	// Response details
	StatusText          string              `json:"status_text,omitempty"`
	ResponseHeaders     map[string][]string `json:"response_headers,omitempty"`
	ResponseBody        string              `json:"response_body,omitempty"`
	ResponseBodySize    int64               `json:"response_body_size"`
	ResponseIsBinary    bool                `json:"response_is_binary"`
	ResponseContentType string              `json:"response_content_type,omitempty"`

	// Additional timing
	TTFBMs *float64 `json:"ttfb_ms,omitempty"`

	// Metadata
	ClientIP string   `json:"client_ip,omitempty"`
	Tags     []string `json:"tags,omitempty"`
}

TrafficDetail represents full traffic details for a single record (detail view).

type TrafficFilter

type TrafficFilter struct {
	// Time range
	StartTime time.Time
	EndTime   time.Time

	// Request filters
	Hosts   []string // Filter by host (supports wildcards)
	Paths   []string // Filter by path (supports wildcards)
	Methods []string // Filter by HTTP method

	// Response filters
	StatusCodes []int // Filter by status code
	MinStatus   int   // Minimum status code (e.g., 400 for errors)
	MaxStatus   int   // Maximum status code

	// Pagination
	Limit  int
	Offset int

	// Sorting
	OrderBy string // Field to sort by
	Desc    bool   // Sort descending
}

TrafficFilter specifies criteria for querying traffic records.

type TrafficQuerier

type TrafficQuerier interface {
	// Query returns traffic records matching the filter.
	Query(ctx context.Context, filter *TrafficFilter) ([]*TrafficRecord, error)

	// GetByID returns full traffic details for a single record.
	GetByID(ctx context.Context, id string) (*TrafficDetail, error)

	// Count returns the number of records matching the filter.
	Count(ctx context.Context, filter *TrafficFilter) (int64, error)

	// Stats returns aggregate statistics.
	Stats(ctx context.Context, filter *TrafficFilter) (*TrafficStats, error)
}

TrafficQuerier is an optional interface for querying stored traffic. Not all TrafficStore implementations support querying (e.g., Kafka producer).

type TrafficRecord

type TrafficRecord struct {
	ID        string
	Method    string
	URL       string
	Host      string
	Path      string
	Status    int
	Duration  time.Duration
	StartTime time.Time
	Error     string
}

TrafficRecord represents a stored traffic record for querying (list view).

type TrafficStats

type TrafficStats struct {
	TotalRequests    int64
	TotalErrors      int64
	AvgDurationMs    float64
	P50DurationMs    float64
	P95DurationMs    float64
	P99DurationMs    float64
	UniqueHosts      int64
	RequestsByMethod map[string]int64
	RequestsByStatus map[int]int64
}

TrafficStats contains aggregate traffic statistics.

type TrafficStore

type TrafficStore interface {
	// Store saves a single traffic record.
	Store(ctx context.Context, rec *capture.Record) error

	// StoreBatch saves multiple traffic records efficiently.
	StoreBatch(ctx context.Context, recs []*capture.Record) error

	// Close releases any resources held by the store.
	Close() error
}

TrafficStore is the interface for storing captured HTTP traffic. Implementations must be safe for concurrent use.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL