omniproxy

module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 27, 2026 License: MIT

README ΒΆ

OmniProxy

Go CI Go Lint Go SAST Docs Visualization License

Universal HTTP/HTTPS proxy with MITM support for traffic capture and analysis.

Features

  • πŸ”€ Forward Proxy - HTTP proxy for routing traffic
  • πŸ•΅οΈ MITM Proxy - HTTPS interception with automatic certificate generation
  • πŸ” Reverse Proxy - Server-side proxy with Let's Encrypt/ACME support
  • πŸ”’ Mutual TLS (mTLS) - Client certificate authentication, plus presenting client certificates to mTLS backends
  • πŸ“₯ Traffic Capture - Export to NDJSON, HAR, or IR formats
  • πŸ—„οΈ Database Storage - SQLite (team mode) or PostgreSQL (production mode)
  • πŸ”Ž Request Filtering - Include/exclude by host, path, or method
  • πŸ–ΌοΈ Binary Detection - Automatically skip binary content (images, videos, etc.)
  • πŸ”— Proxy Chaining - Forward through upstream proxy
  • πŸ“Š Observability - Prometheus metrics, distributed tracing, and structured logs via OpenTelemetry
  • πŸ–₯️ System Proxy Configuration - Automatic setup for macOS, Windows, and Linux
  • πŸ“ Config File Support - YAML configuration files
  • πŸ”‘ Pure Go CA - No OpenSSL dependency, uses Go's crypto libraries
  • 🐳 Docker Support - Multi-stage Dockerfile with health checks

Deployment Modes

OmniProxy supports three deployment modes with the same binary:

Mode Storage Use Case Command
Laptop File (NDJSON) Individual developer omniproxy serve --output traffic.ndjson
Team SQLite Small team, shared server omniproxy serve --db sqlite://traffic.db
Production PostgreSQL Enterprise, multi-tenant omniproxy serve --db postgres://...

Installation

From Source
go install github.com/grokify/omniproxy/cmd/omniproxy@latest
Docker
# Build
docker build -t omniproxy .

# Run
docker run -p 8080:8080 -p 9090:9090 omniproxy serve --host 0.0.0.0

# Or use docker-compose
docker-compose up

Quick Start

1. Generate and Install CA Certificate
# Generate CA (first time only)
omniproxy ca generate

# Install CA into system trust store (may require sudo)
sudo omniproxy ca install
2. Start the Proxy
# Laptop mode - file output
omniproxy serve --output traffic.ndjson

# Team mode - SQLite database
omniproxy serve --db sqlite://traffic.db --metrics-port 9090

# Production mode - PostgreSQL
omniproxy serve --db postgres://user:pass@localhost/omniproxy --metrics-port 9090
3. Configure System to Use Proxy
# Automatically configure system proxy
omniproxy system set --port 8080

# Check current proxy status
omniproxy system status

# Disable system proxy
omniproxy system unset

Usage

Serve Command

Start the proxy server:

omniproxy serve [flags]

Basic Flags:
  -p, --port int           Port to listen on (default 8080)
      --host string        Host to bind to (default "127.0.0.1")
  -v, --verbose            Enable verbose logging
      --mitm               Enable HTTPS MITM (default true)

Output Flags:
  -o, --output string      Output file for captured traffic
  -f, --format string      Output format: ndjson, json, har, ir (default "ndjson")
      --filter-header strings  Additional headers to filter
      --skip-binary        Skip capturing binary content (default true)

Database Flags:
      --db string          Database URL (sqlite://path or postgres://...)

Observability Flags:
      --metrics-port int   Port for metrics/health endpoints (0 = disabled)
      --metrics            Enable Prometheus metrics

Async Flags:
      --async-queue int    Async traffic queue size (default 10000)
      --async-batch int    Async batch size (default 100)
      --async-workers int  Number of async workers (default 2)

Filtering Flags:
      --include-host strings   Only capture requests to these hosts (wildcards supported)
      --exclude-host strings   Exclude requests to these hosts
      --include-path strings   Only capture requests matching these paths
      --exclude-path strings   Exclude requests matching these paths
      --include-method strings Only capture these HTTP methods
      --exclude-method strings Exclude these HTTP methods

Proxy Flags:
      --skip-host strings  Hosts to skip MITM for (cert pinning)
      --upstream string    Upstream proxy URL (e.g., http://proxy:8080)
Database URLs

OmniProxy supports the following database URL formats:

# SQLite
sqlite://traffic.db              # Relative path
sqlite:///var/lib/omniproxy/data.db  # Absolute path
sqlite::memory:                  # In-memory (testing)
sqlite://data.db?cache=shared    # With options

# PostgreSQL
postgres://localhost/omniproxy
postgres://user:pass@localhost:5432/omniproxy
postgres://user:pass@localhost/omniproxy?sslmode=disable
postgresql://...                 # Alternative scheme
Examples
# Capture only API traffic
omniproxy serve --include-host "api.example.com" --include-host "*.api.example.org"

# Exclude static assets
omniproxy serve --exclude-path "*.js" --exclude-path "*.css" --exclude-path "*.png"

# Capture only GET and POST requests
omniproxy serve --include-method GET --include-method POST

# Export to HAR format for browser DevTools
omniproxy serve --output traffic.har --format har

# Chain through corporate proxy
omniproxy serve --upstream http://corporate-proxy:8080

# Team mode with metrics
omniproxy serve --db sqlite://traffic.db --metrics-port 9090

# Production mode with all options
omniproxy serve \
  --db postgres://user:pass@localhost/omniproxy \
  --metrics-port 9090 \
  --async-queue 50000 \
  --async-batch 500 \
  --async-workers 4

# Use with config file
omniproxy serve --config ~/.omniproxy/config.yaml
CA Commands

Manage CA certificates:

# Generate new CA
omniproxy ca generate [--cert path] [--key path] [--org name] [--cn name]

# Install CA to system trust store
omniproxy ca install [--cert path]

# Remove CA from system trust store
omniproxy ca uninstall

# Show CA information
omniproxy ca info
System Commands

Manage system proxy configuration:

# Enable system proxy
omniproxy system set [--host host] [--port port]

# Disable system proxy
omniproxy system unset

# Show current status
omniproxy system status
Reverse Command

Start a reverse proxy with automatic TLS via Let's Encrypt:

omniproxy reverse [flags]

Backend Flags:
  -b, --backend strings    Backend mapping: host=target or /path=target (required)

Local Development Flags:
      --static string      Serve static files from this directory as fallback
      --cors-origin string Add CORS headers to path-based backend responses (e.g., "*")

ACME Flags:
      --acme-email string  Email for Let's Encrypt registration
      --acme-cache string  Directory to cache certificates (default "~/.omniproxy/acme")
      --acme-staging       Use Let's Encrypt staging environment (for testing)

Server Flags:
      --http-port int      HTTP port (default 80)
      --https-port int     HTTPS port (default 443)
      --redirect-http      Redirect HTTP to HTTPS (default true)
  -v, --verbose            Enable verbose logging

Client Certificate (mTLS) Flags:
      --client-ca strings  PEM file of CAs used to verify client certificates; repeatable
      --client-auth string Client certificate mode: none, request, require-any,
                           verify-if-given, require-and-verify
                           (default: require-and-verify when --client-ca is set)

Backend mTLS Flags:
      --backend-cert string PEM client certificate to present to HTTPS backends requiring mTLS
      --backend-key string  PEM private key for --backend-cert (defaults to the cert file)
      --backend-ca string   PEM CA bundle to verify backend server certificates
      --backend-insecure    Skip backend certificate verification (UNSAFE; dev only)

Output Flags:
  -o, --output string      Output file for captured traffic
  -f, --format string      Output format: ndjson, json, har, ir (default "ndjson")
Reverse Proxy Examples
# Basic reverse proxy with Let's Encrypt
sudo omniproxy reverse \
  --backend "api.example.com=http://localhost:3000" \
  --acme-email admin@example.com

# Multiple backends
sudo omniproxy reverse \
  --backend "api.example.com=http://localhost:3000" \
  --backend "web.example.com=http://localhost:8080" \
  --acme-email admin@example.com

# Use staging environment for testing
sudo omniproxy reverse \
  --backend "api.example.com=http://localhost:3000" \
  --acme-staging

# Custom ports (for local testing)
omniproxy reverse \
  --http-port 8080 \
  --https-port 8443 \
  --backend "localhost:8443=http://localhost:3000"

# With traffic capture
sudo omniproxy reverse \
  --backend "api.example.com=http://localhost:3000" \
  --output traffic.ndjson

# Local development: path-based proxy + static files (HTTP-only, no ACME)
omniproxy reverse \
  --http-port 8080 \
  --backend "/api=https://upstream.example.com" \
  --static ./public \
  --cors-origin "*"

# Mutual TLS: require every client to present a certificate signed by your CA
sudo omniproxy reverse \
  --backend "api.example.com=http://localhost:3000" \
  --acme-email admin@example.com \
  --client-ca /etc/omniproxy/client-ca.pem

Note: Running on ports 80 and 443 typically requires root/sudo. When --acme-email is omitted, the server runs HTTP-only (no TLS) for local development.

Local Development Mode

For frontend development, OmniProxy can proxy API requests by path and serve static files as a fallback β€” all without TLS configuration:

omniproxy reverse \
  --http-port 3000 \
  --backend "/api=https://api.example.com" \
  --static ./dist \
  --cors-origin "*"
Feature Description
Path-based routing --backend "/api=..." routes by URL path instead of hostname; the path prefix is stripped before forwarding
Static file fallback --static ./dist serves files when no backend matches (SPA support)
CORS headers --cors-origin "*" adds Access-Control-Allow-* headers to proxied responses and handles OPTIONS preflight
HTTP-only mode Omitting --acme-email skips TLS entirely β€” no certificates needed for localhost

Path-based backends take priority over host-based backends; the longest matching prefix wins.

Mutual TLS (mTLS)

Passing --client-ca turns on client certificate authentication, defaulting to require-and-verify β€” the standard mTLS posture, where a connection is refused unless the client presents a certificate signed by one of the given CAs. Pass --client-auth to choose a different mode:

Mode Behavior
none No client certificate is requested (default when no --client-ca is given)
request Requests a certificate but neither requires nor verifies it
require-any Requires a certificate but verifies it against nothing
verify-if-given Verifies a certificate against the CAs, but admits clients offering none
require-and-verify Requires a certificate and verifies it against the CAs

Once a client certificate has been verified, its identity is forwarded to the backend:

Header Value
X-Client-Cert-CN Subject common name
X-Client-Cert-Subject Full subject distinguished name
X-Client-Cert-Issuer Issuing CA distinguished name
X-Client-Cert-Serial Certificate serial number

These headers are always stripped from inbound requests before being set, so a client cannot impersonate an identity by sending them directly. They are populated only from a certificate that TLS actually verified against the CA pool β€” under request and require-any, which verify nothing, no identity is forwarded.

Security notes
  • --redirect-http must stay enabled. With --redirect-http=false the plain HTTP port serves the proxy directly, and requests arriving there complete no TLS handshake β€” they would reach backends with no certificate checked. Enabling client authentication alongside --redirect-http=false is refused at startup rather than silently exposing that path.
  • Verification is authentication, not authorization. Any certificate signed by one of the configured CAs is accepted for every backend. Use a CA dedicated to this proxy; do not point --client-ca at a public or organization-wide CA, which would admit any certificate it ever issued. Per-identity access rules are not yet supported.
  • No revocation checking. Revoked-but-unexpired certificates are still accepted, as neither CRL nor OCSP is consulted. Keep client certificate lifetimes short.

Test a configured endpoint with cURL:

curl --cert client.pem --key client-key.pem https://api.example.com/

Note: the server certificate still comes from ACME/Let's Encrypt, so mTLS currently requires a publicly resolvable domain. Static server certificates are not yet supported.

Backend mTLS (presenting a certificate to the upstream)

The flags above authenticate incoming clients. The --backend-* flags do the reverse: they present a client certificate to an HTTPS backend that itself enforces mutual TLS, so the proxy can front services deployed in a service mesh or zero-trust network while exposing plain HTTP/HTTPS to callers.

# Front an mTLS-protected backend, verifying its server certificate against a private CA
sudo omniproxy reverse \
  --backend "app.example.com=https://internal-svc:8443" \
  --acme-email admin@example.com \
  --backend-cert /etc/certs/client.pem \
  --backend-ca /etc/certs/internal-ca.pem

# Local development against a self-signed backend
omniproxy reverse \
  --http-port 8080 --https-port 8443 \
  --backend "localhost:8443=https://localhost:9443" \
  --backend-cert /etc/certs/client.pem \
  --backend-insecure

Per-backend configuration is available in the config file, which is the only way to present a different certificate or CA to each backend β€” the CLI flags apply uniformly to all of them:

reverse:
  httpPort: 80
  httpsPort: 443
  backends:
    - host: app.example.com
      target: https://internal-svc:8443
      tls:
        certFile: /etc/certs/client.pem
        keyFile: /etc/certs/client-key.pem   # optional; defaults to certFile
        caFile: /etc/certs/internal-ca.pem
        insecureSkipVerify: false
    - host: other.example.com
      target: https://other-svc:8443
      tls:
        certFile: /etc/certs/other-client.pem   # a distinct certificate per backend

Run with --config:

sudo omniproxy reverse --config /etc/omniproxy/config.yaml

Backend TLS options only apply to https:// targets; setting them on a plain http:// target is rejected at startup rather than silently ignored. --backend-insecure disables verification of the backend's server certificate and is intended only for self-signed development backends β€” never use it in production.

Config Commands

Manage configuration files:

# Create default config file
omniproxy config init

# Show example configuration
omniproxy config show

Observability

When running with --metrics-port, OmniProxy exposes:

Endpoint Description
/metrics Prometheus metrics
/healthz Liveness probe (always returns 200 when running)
/readyz Readiness probe (returns 200 when ready to accept traffic)
Prometheus Metrics
Metric Type Description
omniproxy_requests_total Counter Total HTTP requests processed
omniproxy_request_duration_seconds Histogram Request duration in seconds
omniproxy_active_requests Gauge Currently active requests
omniproxy_certs_generated_total Counter TLS certificates generated
omniproxy_cert_cache_hits_total Counter Certificate cache hits
omniproxy_cert_cache_misses_total Counter Certificate cache misses
omniproxy_traffic_stored_total Counter Traffic records stored
omniproxy_traffic_store_errors_total Counter Traffic store errors
omniproxy_traffic_queue_depth Gauge Async queue depth

Configuration File

OmniProxy supports YAML configuration files. Pass one to serve or reverse with --config; generate a starting point with omniproxy config init.

Precedence: the config file is the base, and an explicitly-set command-line flag overrides it. A flag left at its default falls back to the config value. Without --config, behavior is flag-only and unchanged. For the reverse proxy, backends come from the config file unless --backend is given, since only the config file can express per-backend TLS.

server:
  host: 127.0.0.1
  port: 8080
  verbose: false

mitm:
  enabled: true
  skipHosts:
    - "*.pinned-app.com"

reverse:
  httpPort: 80
  httpsPort: 443
  redirectHTTP: true
  acmeEmail: admin@example.com
  staticDir: ""             # Serve static files as fallback (local dev)
  # Frontend client authentication (verify caller certificates):
  clientCAPaths:
    - /etc/omniproxy/client-ca.pem
  clientAuth: require-and-verify
  backends:
    - host: app.example.com
      target: https://internal-svc:8443
      # Backend mTLS (present a certificate to the upstream):
      tls:
        certFile: /etc/certs/client.pem
        caFile: /etc/certs/internal-ca.pem
    # Path-based backend (local dev):
    - pathPrefix: /api
      target: https://api.example.com
      corsOrigin: "*"       # Add CORS headers to responses

capture:
  output: traffic.ndjson
  format: ndjson
  includeHeaders: true
  includeBody: true
  maxBodySize: 1048576
  skipBinary: true
  filterHeaders:
    - authorization
    - cookie
    - set-cookie

filter:
  includeHosts:
    - "api.example.com"
    - "*.example.org"
  excludePaths:
    - "*.js"
    - "*.css"
    - "*.png"

observability:
  metrics: false
  metricsPort: 0        # 0 disables the metrics/health server

storage:
  databaseURL: ""       # sqlite://path or postgres://...; empty = write to capture output
  async:
    queueSize: 10000
    batchSize: 100
    workers: 2

upstream: ""

Generate a fully populated example with omniproxy config init (or print one with omniproxy config show). The observability and storage sections apply to serve; the reverse section applies to reverse; server, mitm, capture, and filter are shared.

Output Formats

NDJSON (default)

Newline-delimited JSON, one record per line:

{"request":{"method":"GET","url":"https://api.example.com/users","host":"api.example.com","path":"/users","scheme":"https"},"response":{"status":200},"startTime":"2024-12-30T10:00:00Z","durationMs":45.5}
HAR

HTTP Archive format, compatible with browser DevTools. Use Ctrl+C to gracefully shutdown and write the HAR file.

IR

APISpecRift Intermediate Representation format, for generating OpenAPI specs.

Docker

Dockerfile

The included multi-stage Dockerfile creates a minimal image:

# Build
docker build -t omniproxy .

# Run with file output
docker run -p 8080:8080 -v $(pwd)/data:/data omniproxy serve \
  --host 0.0.0.0 \
  --output /data/traffic.ndjson

# Run with SQLite
docker run -p 8080:8080 -p 9090:9090 -v $(pwd)/data:/data omniproxy serve \
  --host 0.0.0.0 \
  --db sqlite:///data/traffic.db \
  --metrics-port 9090
Docker Compose
version: '3.8'
services:
  omniproxy:
    build: .
    ports:
      - "8080:8080"
      - "9090:9090"
    volumes:
      - ./data:/data
    command:
      - serve
      - --host=0.0.0.0
      - --db=sqlite:///data/traffic.db
      - --metrics-port=9090
    healthcheck:
      test: ["CMD", "wget", "--spider", "http://localhost:9090/healthz"]
      interval: 30s
      timeout: 5s
      retries: 3

Integration with APISpecRift

OmniProxy is designed to work with APISpecRift for generating OpenAPI specifications from captured traffic:

# Capture traffic
omniproxy serve --output traffic.ndjson --format ir

# Generate OpenAPI spec
apispecrift generate --input traffic.ndjson --output openapi.yaml

Platform Support

Platform Proxy Config CA Install
macOS networksetup Keychain
Windows Registry certutil
Linux GNOME/KDE/env ca-certificates

Security Considerations

  • The CA private key is stored in ~/.omniproxy/ca/ with restricted permissions (0600)
  • Sensitive headers (Authorization, Cookie, etc.) are filtered by default
  • Use --skip-host for applications with certificate pinning
  • Use filtering to capture only relevant traffic
  • Database passwords in PostgreSQL URLs are hidden in logs

License

MIT

Directories ΒΆ

Path Synopsis
cmd
omniproxy command
OmniProxy is a universal HTTP/HTTPS proxy with MITM support for traffic capture.
OmniProxy is a universal HTTP/HTTPS proxy with MITM support for traffic capture.
pkg
backend
Package backend provides pluggable interfaces for OmniProxy storage and caching.
Package backend provides pluggable interfaces for OmniProxy storage and caching.
ca
Package ca provides certificate authority management for MITM proxy.
Package ca provides certificate authority management for MITM proxy.
capture
Package capture provides traffic capture functionality for the proxy.
Package capture provides traffic capture functionality for the proxy.
config
Package config provides configuration file support for OmniProxy.
Package config provides configuration file support for OmniProxy.
contentdetect
Package contentdetect provides content type detection using multiple methods: Content-Type headers, magic bytes (file signatures), and byte analysis heuristics.
Package contentdetect provides content type detection using multiple methods: Content-Type headers, magic bytes (file signatures), and byte analysis heuristics.
daemon
Package daemon provides daemon lifecycle management for omniproxyd.
Package daemon provides daemon lifecycle management for omniproxyd.
observability
Package observability provides OpenTelemetry instrumentation for OmniProxy.
Package observability provides OpenTelemetry instrumentation for OmniProxy.
proxy
Package proxy provides HTTP/HTTPS proxy functionality using goproxy.
Package proxy provides HTTP/HTTPS proxy functionality using goproxy.
reverseproxy
Package reverseproxy provides reverse proxy functionality with automatic TLS via ACME.
Package reverseproxy provides reverse proxy functionality with automatic TLS via ACME.
system
Package system provides OS-level proxy configuration.
Package system provides OS-level proxy configuration.
ui
auth
Package auth provides authentication and session management.
Package auth provides authentication and session management.
ent
store
Package store provides database initialization and multi-tenant support.
Package store provides database initialization and multi-tenant support.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL