README
¶
Prometheus Backpressure Proxy
Adaptive Protection for Your Backend Services
🛡️ Dynamically shield your services from traffic overload using smart, metrics-driven congestion control.
See Netflix concurrency-limits for the inspiration.
Key Features
- 📊 Adaptive Traffic Management: Automatically adjusts request concurrency based on real-time Prometheus metrics
- 🔀 Smart Scaling: Uses Additive Increase/Multiplicative Decrease (AIMD) algorithm
- 🚦 Configurable Limits: Set min and max concurrent request thresholds
- 🔍 Multi-Signal Monitoring: Track system health across multiple metrics simultaneously
Quick Example
config := proxymw.BackpressureConfig{
EnableBackpressure: true,
BackpressureQueries: []BackpressureQuery{
{
Query: `sum(rate(http_server_errors_total[5m]))`,
// Start to throttle when error rate reaches 50%
WarningThreshold: 0.5,
// Hard throttling up to 100 CongestionWindowMax when error rate is >80%
EmergencyThreshold: 0.8,
}
},
CongestionWindowMin: 10,
CongestionWindowMax: 100,
}
How It Works
- 🔭 Continuously monitor system metrics
- 📈 Dynamically adjust request throughput
- 🛑 Automatically throttle when system stress detected
When to Use
- Protecting microservices from sudden traffic spikes
- Preventing cascading failures
- Maintaining system stability under unpredictable load
Quick Start
- Configure backpressure queries as Prometheus metrics
- Define min/max request windows
- Choose the server-side http proxy or client-side roundtripper
- Import the starter Grafana dashboard
- Let the proxy handle the rest!
Development
Installation
Build the docker-compose stack
make all
docker compose down
docker-compose up --build
- Generate fake traffic with
./scripts/traffic_generator.py - View metrics in the local Grafana instance
Lint and Test
make test
make lintfix
Upgrade dependencies
make deps
Contributing
Documentation
¶
Overview ¶
Package throttleproxy provides an adaptive backpressure proxy mechanism for dynamically managing traffic and protecting backend services using Prometheus metrics.
Usage Example:
config := proxymw.BackpressureConfig{
EnableBackpressure: true,
BackpressureQueries: []BackpressureQuery{
{
Query: `sum(rate(http_server_errors_total[5m]))`,
Name: "http_error_rate"
WarningThreshold: 0.5,
EmergencyThreshold: 0.8,
}
},
CongestionWindowMin: 10,
CongestionWindowMax: 100,
}
Use Cases:
- Protecting microservices from traffic spikes
- Preventing cascading failures
- Maintaining system stability under unpredictable load
The package supports both server-side HTTP proxy and client-side RoundTripper implementations, providing flexible integration options.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
examples
module
|
|
|
roundtripper
command
|
|
|
Package proxymw holds interfaces and configuration to safeguard backend services from dynamic load
|
Package proxymw holds interfaces and configuration to safeguard backend services from dynamic load |
|
Package proxyutil handles parsing logic for proxymw configs
|
Package proxyutil handles parsing logic for proxymw configs |
|
proxyhttp
Package proxyhttp generates an http mux handler to servce requests using *httputil.ReverseProxy
|
Package proxyhttp generates an http mux handler to servce requests using *httputil.ReverseProxy |
Click to show internal directories.
Click to hide internal directories.