throttle

package
v1.16.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 4, 2025 License: MIT Imports: 16 Imported by: 0

README

HTTP middleware for API throttling

The package provides two types of server-side throttling for HTTP services:

  1. In-flight throttling limits the total number of currently served (in-flight) HTTP requests.
  2. Rate-limit throttling limits the rate of HTTP requests using the leaky bucket or the sliding window algorithms (it's configurable, the leaky bucket is used by default).

Throttling is implemented as a typical Go HTTP middleware and can be configured from the code or from the JSON/YAML configuration file.

Please see testable example to understand how to configure and use the middleware.

Throttling configuration

The throttling configuration is usually stored in the JSON or YAML file. The configuration consists of the following parts:

  1. rateLimitZones. Each zone has a rate limit, burst limit, and other parameters.
  2. inFlightLimitZones. Each zone has an in-flight limit, backlog limit, and other parameters.
  3. rules. Each rule contains a list of routes and rate/in-flight limiting zones that should be applied to these routes.
Global throttling example

Global throttling assesses all traffic coming into an API from all sources and ensures that the overall rate/concurrency limit is not exceeded. Overwhelming an endpoint with traffic is an easy and efficient way to carry out a denial-of-service attack. By using a global rate/concurrency limit, you can ensure that all incoming requests are within a specific limit.

API-level throttling
rateLimitZones:
  rl_total:
    rateLimit: 5000/s
    burstLimit: 10000
    responseStatusCode: 503
    responseRetryAfter: 5s

inFlightLimitZones:
  ifl_total:
    inFlightLimit: 5000
    backlogLimit: 10000
    backlogTimeout: 30s
    responseStatusCode: 503
    responseRetryAfter: 1m

rules:
  - routes:
      - path: "/"
    rateLimits:
      - zone: rl_total
    inFlightLimits:
      - zone: ifl_total

With this configuration, all HTTP requests will be limited by rate (no more than 5000 requests per second, rateLimit). Excessive requests within the burst limit (10000 here, burstLimit) will be served immediately regardless of the specified rate, requests above the burst limit will be rejected with the 503 error (responseStatusCode), and "Retry-After: 5" HTTP header (responseRetryAfter).

Additionally, there is a concurrency limit. If 5000 requests (inFlightLimit) are being processed right now, new incoming requests will be backlogged (suspended). If there are more than 10000 such backlogged requests (backlogLimit), the rest will be rejected immediately. The request can be in backlogged status for no more than 30 seconds (backlogTimeout) and then it will be rejected. Response for rejected request contains 503 (responseStatusCode) HTTP error code and "Retry-After: 60" HTTP header (responseRetryAfter).

backlogLimit and backlogTimeout can be specified for rate-limiting zones too.

Per-client throttling example

Per-client throttling is focused on controlling traffic from individual sources and making sure that API clients are staying within their prescribed limits. Per-client throttling allows avoiding cases when one client exhausts the application resources of the entire backend service (for example, it uses all connections from the DB pool), and all other clients have to wait for their release.

To implement per-client throttling, the package uses the concept of "identity". If the client is identified by a unique key, the package can throttle requests per this key. MiddlewareOpts struct has a GetKeyIdentity callback that should return the key for the current request. It may be a user ID, JWT "sub" claim, or any other unique identifier. If any rate/in-flight limiting zone's key.type field is set to identity, the GetKeyIdentity callback must be implemented.

Example of per-client throttling configuration:

rateLimitZones:
  rl_identity:
    rateLimit: 50/s
    burstLimit: 100
    responseStatusCode: 429
    responseRetryAfter: auto
    key:
      type: identity
    maxKeys: 50000

inFlightLimitZones:
  ifl_identity:
    inFlightLimit: 64
    backlogLimit: 128
    backlogTimeout: 30s
    responseStatusCode: 429
    key:
      type: identity
    maxKeys: 50000

  ifl_identity_expensive_op:
    inFlightLimit: 4
    backlogLimit: 8
    backlogTimeout: 30s
    responseStatusCode: 429
    key:
      type: identity
    maxKeys: 50000
    excludedKeys:
      - "150853ab-322c-455d-9793-8d71bf6973d9" # Exclude root admin.

rules:
  - routes:
      - path: "/"
    rateLimits:
      - zone: rl_identity
    inFlightLimits:
      - zone: ifl_identity
    alias: per_identity
  - routes:
      - path: "= /api/v1/do_expensive_op_1"
        methods: POST
      - path: "= /api/v1/do_expensive_op_2"
        methods: POST
    inFlightLimits:
      - zone: ifl_identity_expensive_op
    alias: per_identity_expensive_ops

All throttling counters are stored inside an in-memory LRU cache (maxKeys determines its size).

For the rate-limiting zone, responseRetryAfter may be specified as "auto". In this case, the time when a client may retry the request will be calculated automatically.

Each throttling rule may contain an unlimited number of rate/in-flight limiting zones. All rule zones will be applied to all specified routes. The route is described as path + list of HTTP methods. To select a route, exactly the same algorithm is used as to select a location in Nginx (http://nginx.org/en/docs/http/ngx_http_core_module.html#location). Also, the route may have an alias that will be used in the Prometheus metrics label (see example below).

Sliding window rate-limiting
rateLimitZones:
  rl_identity:
    alg: sliding_window
    rateLimit: 15/m
    responseStatusCode: 429
    responseRetryAfter: auto
    key:
      type: identity
    maxKeys: 50000

rules:
  - routes:
      - path: "/"
    rateLimits:
      - zone: rl_identity
    alias: per_identity

In this example sliding window algorithm will be used for rate-limiting (alg parameter has "token_bucket" value by default). It means, only 15 requests are allowed per minute. They could be sent even simultaneously, but all exceeding requests that are received in the same minute will be rejected.

Example of throttling of all requests with a "bad" User-Agent HTTP header
rateLimitZones:
  rl_bad_user_agents:
    rateLimit: 500/s
    burstLimit: 1000
    responseStatusCode: 503
    responseRetryAfter: 15s
    key:
      type: header
      headerName: "User-Agent"
      noBypassEmpty: true
    includedKeys:
      - ""
      - "Go-http-client/1.1"
      - "python-requests/*"
      - "Python-urllib/*"
    maxKeys: 1000

rules:
  - routes:
      - path: "/"
    rateLimits:
      - zone: rl_bad_user_agents
Throttle requests by remote address example
rateLimitZones:
  rl_by_remote_addr:
    rateLimit: 100/s
    burstLimit: 1000
    responseStatusCode: 503
    responseRetryAfter: auto
    key:
      type: remote_addr
    maxKeys: 10000

rules:
  - routes:
      - path: "/"
    rateLimits:
      - zone: rl_by_remote_addr

Prometheus metrics

The package collects several metrics in the Prometheus format:

  • rate_limit_rejects_total. Type: counter; Labels: dry_run, rule.
  • in_flight_limit_rejects_total. Type: counter; Labels: dry_run, rule, backlogged.

Tags

Tags are useful when different rules of the same configuration should be used by different middlewares. For example, suppose you want to have two different throttling rules:

  1. A rule for all requests.
  2. A rule for all identity-aware (authorized) requests.
# ...
rules:
  - routes:
    - path: "/hello"
      methods: GET
    rateLimits:
      - zone: rl_zone1
    tags: all_reqs

  - routes:
    - path: "/feedback"
      methods: POST
    inFlightLimits:
      - zone: ifl_zone1
    tags: all_reqs

  - routes:
    - path: /api/1/users
      methods: PUT
    rateLimits:
      - zone: rl_zone2
    tags: require_auth_reqs
# ...

In your code, you will have two middlewares that will be executed at different steps of the HTTP request serving process. Each middleware should only apply its own throttling rule.

allMw := MiddlewareWithOpts(cfg, "my-app-domain", throttleMetrics, MiddlewareOpts{Tags: []string{"all_reqs"}})
requireAuthMw := MiddlewareWithOpts(cfg, "my-app-domain", throttleMetrics, MiddlewareOpts{Tags: []string{"require_auth_reqs"}})

Dry-run mode

Before configuring real-life throttling, usually, it's a good idea to try the dry-run mode. It doesn't affect the processing requests flow, however, all excessive requests are still counted and logged. Dry-run mode allows you to better understand how your API is used and determine the right throttling parameters.

The dry-run mode can be enabled using the dryRun configuration parameter. Example:

rateLimitZones:
  rl_identity:
    rateLimit: 50/s
    burstLimit: 100
    responseStatusCode: 429
    responseRetryAfter: auto
    key:
      type: identity
    maxKeys: 50000
    dryRun: true

inFlightLimitZones:
  ifl_identity:
    inFlightLimit: 64
    backlogLimit: 128
    backlogTimeout: 30s
    responseStatusCode: 429
    key:
      type: identity
    maxKeys: 50000
    dryRun: true

rules:
  - routes:
      - path: "/"
    rateLimits:
      - zone: rl_identity
    inFlightLimits:
      - zone: ifl_identity
    alias: per_identity

If specified limits are exceeded, the corresponding messages will be logged.

For rate-limiting:

{"msg": "too many requests, serving will be continued because of dry run mode", "rate_limit_key": "ee9a0dd8-7396-5478-8b83-ab7402d6746b"}

For in-flight limiting:

{"msg": "too many in-flight requests, serving will be continued because of dry run mode", "in_flight_limit_key": "3c00e780-5721-59f8-acad-f0bf719777d4"}

License

Copyright © 2024 Acronis International GmbH.

Licensed under MIT License.

Documentation

Overview

Package throttle provides configurable middleware for throttling HTTP requests on the server side.

Example 
package main

import (
	"bytes"
	"fmt"
	stdlog "log"
	"net/http"
	"net/http/httptest"
	"regexp"
	"strconv"
	"time"

	"github.com/acronis/go-appkit/config"
	"github.com/acronis/go-appkit/httpserver/middleware/throttle"
)

const apiErrDomain = "MyService"

func main() {
	configReader := bytes.NewReader([]byte(`
rateLimitZones:
  rl_zone1:
    rateLimit: 1/s
    burstLimit: 0
    responseStatusCode: 503
    responseRetryAfter: auto
    dryRun: false

  rl_zone2:
    rateLimit: 5/m
    burstLimit: 0
    responseStatusCode: 429
    responseRetryAfter: auto
    key:
      type: "identity"
    dryRun: false

inFlightLimitZones:
  ifl_zone1:
    inFlightLimit: 1
    backlogLimit: 0
    backlogTimeout: 15s
    responseStatusCode: 503
    dryRun: false

rules:
  - routes:
    - path: "/hello-world"
      methods: GET
    excludedRoutes:
      - path: "/healthz"
    rateLimits:
      - zone: rl_zone1
    tags: all_reqs

  - routes:
    - path: "= /long-work"
      methods: POST
    inFlightLimits:
      - zone: ifl_zone1
    tags: all_reqs

  - routes:
    - path: ~^/api/2/tenants/([\w\-]{36})/?$
      methods: PUT
    rateLimits:
      - zone: rl_zone2
    tags: authenticated_reqs
`))
	configLoader := config.NewLoader(config.NewViperAdapter())
	cfg := &throttle.Config{}
	if err := configLoader.LoadFromReader(configReader, config.DataTypeYAML, cfg); err != nil {
		stdlog.Fatal(err)
		return
	}

	const longWorkDelay = time.Second

	srv, err := makeExampleTestServer(cfg, longWorkDelay)
	if err != nil {
		stdlog.Fatal(err)
		return
	}
	defer srv.Close()

	// Rate limiting.
	// 1st request finished successfully.
	resp1, _ := http.Get(srv.URL + "/hello-world")
	_ = resp1.Body.Close()
	fmt.Println("[1] GET /hello-world " + strconv.Itoa(resp1.StatusCode))
	// 2nd request is throttled.
	resp2, _ := http.Get(srv.URL + "/hello-world")
	_ = resp2.Body.Close()
	fmt.Println("[2] GET /hello-world " + strconv.Itoa(resp2.StatusCode))

	// In-flight limiting.
	// 3rd request finished successfully.
	resp3code := make(chan int)
	go func() {
		resp3, _ := http.Post(srv.URL+"/long-work", "", nil)
		_ = resp3.Body.Close()
		resp3code <- resp3.StatusCode
	}()
	time.Sleep(longWorkDelay / 2)
	// 4th request is throttled.
	resp4, _ := http.Post(srv.URL+"/long-work", "", nil)
	_ = resp4.Body.Close()
	fmt.Println("[3] POST /long-work " + strconv.Itoa(<-resp3code))
	fmt.Println("[4] POST /long-work " + strconv.Itoa(resp4.StatusCode))

	// Unmatched (unspecified) routes are not limited.
	resp5code := make(chan int)
	go func() {
		resp5, _ := http.Post(srv.URL+"/long-work-without-limits", "", nil)
		_ = resp5.Body.Close()
		resp5code <- resp5.StatusCode
	}()
	time.Sleep(longWorkDelay / 2)
	resp6, _ := http.Post(srv.URL+"/long-work", "", nil)
	_ = resp6.Body.Close()
	fmt.Println("[5] POST /long-work-without-limits " + strconv.Itoa(<-resp5code))
	fmt.Println("[6] POST /long-work-without-limits " + strconv.Itoa(resp6.StatusCode))

	// Throttle authenticated requests by username from basic auth.
	const tenantPath = "/api/2/tenants/446507ba-2f9b-4347-adbc-63581383ba25"
	doReqWithBasicAuth := func(username string) *http.Response {
		req, _ := http.NewRequest(http.MethodPut, srv.URL+tenantPath, http.NoBody)
		req.SetBasicAuth(username, username+"-password")
		resp, _ := http.DefaultClient.Do(req)
		return resp
	}
	// 7th request is not throttled.
	resp7 := doReqWithBasicAuth("ba27afb7-ad60-4077-956e-366e77358b92")
	_ = resp7.Body.Close()
	fmt.Printf("[7] PUT %s %d\n", tenantPath, resp7.StatusCode)
	// 8th request is throttled (the same username as in the previous request, and it's rate-limited).
	resp8 := doReqWithBasicAuth("ba27afb7-ad60-4077-956e-366e77358b92")
	_ = resp8.Body.Close()
	fmt.Printf("[8] PUT %s %d\n", tenantPath, resp8.StatusCode)
	// 9th request is not throttled (the different username is used).
	resp9 := doReqWithBasicAuth("97d8d1e6-948d-4c41-91d6-495dcc8c7b1a")
	_ = resp9.Body.Close()
	fmt.Printf("[9] PUT %s %d\n", tenantPath, resp9.StatusCode)

}

func makeExampleTestServer(cfg *throttle.Config, longWorkDelay time.Duration) (*httptest.Server, error) {
	promMetrics := throttle.NewPrometheusMetrics()
	promMetrics.MustRegister()
	defer promMetrics.Unregister()

	// Configure middleware that should do global throttling ("all_reqs" tag says about that).
	globalThrottleMiddleware, err := throttle.MiddlewareWithOpts(cfg, apiErrDomain, promMetrics, throttle.MiddlewareOpts{
		Tags: []string{"all_reqs"}})
	if err != nil {
		return nil, fmt.Errorf("create global throttling middleware: %w", err)
	}

	// Configure middleware that should do per-client throttling based on the username from basic auth ("authenticated_reqs" tag says about that).
	clientThrottleMiddleware, err := throttle.MiddlewareWithOpts(cfg, apiErrDomain, promMetrics, throttle.MiddlewareOpts{
		Tags: []string{"authenticated_reqs"},
		GetKeyIdentity: func(r *http.Request) (key string, bypass bool, err error) {
			username, _, ok := r.BasicAuth()
			if !ok {
				return "", true, fmt.Errorf("no basic auth")
			}
			return username, false, nil
		},
	})
	if err != nil {
		return nil, fmt.Errorf("create client throttling middleware: %w", err)
	}

	restoreTenantPathRegExp := regexp.MustCompile(`^/api/2/tenants/([\w-]{36})/?$`)
	return httptest.NewServer(globalThrottleMiddleware(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
		switch r.URL.Path {
		case "/long-work":
			if r.Method != http.MethodPost {
				rw.WriteHeader(http.StatusMethodNotAllowed)
				return
			}
			time.Sleep(longWorkDelay) // Emulate long work.
			rw.WriteHeader(http.StatusOK)
			return

		case "/hello-world":
			if r.Method != http.MethodGet {
				rw.WriteHeader(http.StatusMethodNotAllowed)
				return
			}
			rw.WriteHeader(http.StatusOK)
			_, _ = rw.Write([]byte("Hello world!"))
			return

		case "/long-work-without-limits":
			if r.Method != http.MethodPost {
				rw.WriteHeader(http.StatusMethodNotAllowed)
				return
			}
			time.Sleep(longWorkDelay) // Emulate long work.
			rw.WriteHeader(http.StatusOK)
			return
		}

		if restoreTenantPathRegExp.MatchString(r.URL.Path) {
			if r.Method != http.MethodPut {
				rw.WriteHeader(http.StatusMethodNotAllowed)
				return
			}
			clientThrottleMiddleware(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
				rw.WriteHeader(http.StatusNoContent)
			})).ServeHTTP(rw, r)
			return
		}

		rw.WriteHeader(http.StatusNotFound)
	}))), nil
}

Output:

[1] GET /hello-world 200
[2] GET /hello-world 503
[3] POST /long-work 200
[4] POST /long-work 503
[5] POST /long-work-without-limits 200
[6] POST /long-work-without-limits 200
[7] PUT /api/2/tenants/446507ba-2f9b-4347-adbc-63581383ba25 204
[8] PUT /api/2/tenants/446507ba-2f9b-4347-adbc-63581383ba25 429
[9] PUT /api/2/tenants/446507ba-2f9b-4347-adbc-63581383ba25 204

Index

Examples

Constants

View Source 
const (
	RateLimitAlgLeakyBucket   = "leaky_bucket"
	RateLimitAlgSlidingWindow = "sliding_window"
)

Rate-limiting algorithms.

View Source 
const RuleLogFieldName = "throttle_rule"

RuleLogFieldName is a logged field that contains the name of the throttling rule.

Variables

This section is empty.

Functions

func InFlightLimitMiddleware added in v1.16.0 

func InFlightLimitMiddleware(
	cfg *InFlightLimitZoneConfig, errDomain string, ruleName string, mc MetricsCollector,
) (func(next http.Handler) http.Handler, error)

InFlightLimitMiddleware is a middleware that performs in-flight limiting based on the passed configuration.

func InFlightLimitMiddlewareWithOpts added in v1.16.0 

func InFlightLimitMiddlewareWithOpts(
	cfg *InFlightLimitZoneConfig, errDomain string, ruleName string, mc MetricsCollector, opts InFlightLimitMiddlewareOpts,
) (func(next http.Handler) http.Handler, error)

InFlightLimitMiddlewareWithOpts is a more configurable version of InFlightLimitMiddleware.

func MapstructureDecodeHook added in v1.11.0 

func MapstructureDecodeHook() mapstructure.DecodeHookFunc

MapstructureDecodeHook returns a DecodeHookFunc for mapstructure to handle custom types.

func Middleware 

func Middleware(cfg *Config, errDomain string, mc MetricsCollector) (func(next http.Handler) http.Handler, error)

Middleware is a middleware that throttles incoming HTTP requests based on the passed configuration.

func MiddlewareWithOpts 

func MiddlewareWithOpts(
	cfg *Config, errDomain string, mc MetricsCollector, opts MiddlewareOpts,
) (func(next http.Handler) http.Handler, error)

MiddlewareWithOpts is a more configurable version of Middleware.

func RateLimitMiddleware added in v1.16.0 

func RateLimitMiddleware(
	cfg *RateLimitZoneConfig, errDomain string, ruleName string, mc MetricsCollector,
) (func(next http.Handler) http.Handler, error)

RateLimitMiddleware is a middleware that performs rate limiting based on the passed configuration.

func RateLimitMiddlewareWithOpts added in v1.16.0 

func RateLimitMiddlewareWithOpts(
	cfg *RateLimitZoneConfig, errDomain string, ruleName string, mc MetricsCollector, opts RateLimitMiddlewareOpts,
) (func(next http.Handler) http.Handler, error)

RateLimitMiddlewareWithOpts is a more configurable version of RateLimitMiddleware.

Types

type Config 

type Config struct {
	// RateLimitZones contains rate limiting zones.
	// Key is a zone's name, and value is a zone's configuration.
	RateLimitZones map[string]RateLimitZoneConfig `mapstructure:"rateLimitZones" yaml:"rateLimitZones" json:"rateLimitZones"`

	// InFlightLimitZones contains in-flight limiting zones.
	// Key is a zone's name, and value is a zone's configuration.
	InFlightLimitZones map[string]InFlightLimitZoneConfig `mapstructure:"inFlightLimitZones" yaml:"inFlightLimitZones" json:"inFlightLimitZones"` // nolint: lll

	// Rules contains list of so-called throttling rules.
	// Basically, throttling rule represents a route (or multiple routes),
	// and rate/in-flight limiting zones based on which all matched HTTP requests will be throttled.
	Rules []RuleConfig `mapstructure:"rules" yaml:"rules" json:"rules"`
	// contains filtered or unexported fields
}

Config represents a configuration for throttling of HTTP requests on the server side. Configuration can be loaded in different formats (YAML, JSON) using config.Loader, viper, or with json.Unmarshal/yaml.Unmarshal functions directly.

func NewConfig 

func NewConfig(options ...ConfigOption) *Config

NewConfig creates a new instance of the Config.

func NewConfigWithKeyPrefix 

func NewConfigWithKeyPrefix(keyPrefix string) *Config

NewConfigWithKeyPrefix creates a new instance of the Config with a key prefix. This prefix will be used by config.Loader. Deprecated: use NewConfig with WithKeyPrefix instead.

func (*Config) KeyPrefix 

func (c *Config) KeyPrefix() string

KeyPrefix returns a key prefix with which all configuration parameters should be presented. Implements config.KeyPrefixProvider interface.

func (*Config) Set 

func (c *Config) Set(dp config.DataProvider) error

Set sets throttling configuration values from config.DataProvider. Implements config.Config interface.

func (*Config) SetProviderDefaults 

func (c *Config) SetProviderDefaults(_ config.DataProvider)

SetProviderDefaults sets default configuration values for logger in config.DataProvider. Implements config.Config interface.

func (*Config) Validate 

func (c *Config) Validate() error

Validate validates configuration.

type ConfigOption added in v1.11.0 

type ConfigOption func(*configOptions)

ConfigOption is a type for functional options for the Config.

func WithKeyPrefix added in v1.11.0 

func WithKeyPrefix(keyPrefix string) ConfigOption

WithKeyPrefix returns a ConfigOption that sets a key prefix for parsing configuration parameters. This prefix will be used by config.Loader.

type InFlightLimitMiddlewareOpts added in v1.16.0 

type InFlightLimitMiddlewareOpts struct {
	// GetKeyIdentity is a function that returns identity string representation.
	// The returned string is used as a key for zone when key.type is "identity".
	GetKeyIdentity func(r *http.Request) (key string, bypass bool, err error)

	// InFlightLimitOnReject is a callback that is called for rejecting HTTP request when the in-flight limit is exceeded.
	InFlightLimitOnReject middleware.InFlightLimitOnRejectFunc

	// RateLimitOnRejectInDryRun is a callback that is called for rejecting HTTP request in the dry-run mode
	// when the in-flight limit is exceeded.
	InFlightLimitOnRejectInDryRun middleware.InFlightLimitOnRejectFunc

	// RateLimitOnError is a callback that is called in case of any error that may occur during the in-flight limiting.
	InFlightLimitOnError middleware.InFlightLimitOnErrorFunc
}

type InFlightLimitZoneConfig 

type InFlightLimitZoneConfig struct {
	ZoneConfig         `mapstructure:",squash" yaml:",inline"`
	InFlightLimit      int                 `mapstructure:"inFlightLimit" yaml:"inFlightLimit" json:"inFlightLimit"`
	BacklogLimit       int                 `mapstructure:"backlogLimit" yaml:"backlogLimit" json:"backlogLimit"`
	BacklogTimeout     config.TimeDuration `mapstructure:"backlogTimeout" yaml:"backlogTimeout" json:"backlogTimeout"`
	ResponseRetryAfter config.TimeDuration `mapstructure:"responseRetryAfter" yaml:"responseRetryAfter" json:"responseRetryAfter"`
}

InFlightLimitZoneConfig represents zone configuration for in-flight limiting.

func (*InFlightLimitZoneConfig) Validate 

func (c *InFlightLimitZoneConfig) Validate() error

Validate validates zone configuration for in-flight limiting.

type MetricsCollector 

type MetricsCollector interface {
	// IncInFlightLimitRejects increments the counter of rejected requests due to in-flight limit exceeded.
	IncInFlightLimitRejects(ruleName string, dryRun bool, backlogged bool)

	// IncRateLimitRejects increments the counter of rejected requests due to rate limit exceeded.
	IncRateLimitRejects(ruleName string, dryRun bool)
}

MetricsCollector represents a collector of metrics for rate/in-flight limiting rejects.

type MiddlewareOpts 

type MiddlewareOpts struct {
	// GetKeyIdentity is a function that returns identity string representation.
	// The returned string is used as a key for zone when key.type is "identity".
	GetKeyIdentity func(r *http.Request) (key string, bypass bool, err error)

	// RateLimitOnReject is a callback called for rejecting HTTP request when the rate limit is exceeded.
	RateLimitOnReject middleware.RateLimitOnRejectFunc

	// RateLimitOnRejectInDryRun is a callback called for rejecting HTTP request in the dry-run mode
	// when the rate limit is exceeded.
	RateLimitOnRejectInDryRun middleware.RateLimitOnRejectFunc

	// RateLimitOnError is a callback called in case of any error that may occur during the rate limiting.
	RateLimitOnError middleware.RateLimitOnErrorFunc

	// InFlightLimitOnReject is a callback called for rejecting HTTP request when the in-flight limit is exceeded.
	InFlightLimitOnReject middleware.InFlightLimitOnRejectFunc

	// RateLimitOnRejectInDryRun is a callback called for rejecting HTTP request in the dry-run mode
	// when the in-flight limit is exceeded.
	InFlightLimitOnRejectInDryRun middleware.InFlightLimitOnRejectFunc

	// RateLimitOnError is a callback called in case of any error that may occur during the in-flight limiting.
	InFlightLimitOnError middleware.InFlightLimitOnErrorFunc

	// Tags is a list of tags for filtering throttling rules from the config. If it's empty, all rules can be applied.
	Tags []string

	// BuildHandlerAtInit determines where the final handler will be constructed.
	// If true, it will be done at the initialization step (i.e., in the constructor),
	// false (default) - right in the ServeHTTP() method (gorilla/mux case).
	BuildHandlerAtInit bool
}

MiddlewareOpts represents an options for Middleware.

func (MiddlewareOpts) InFlightLimitOpts 

func (opts MiddlewareOpts) InFlightLimitOpts() InFlightLimitMiddlewareOpts

InFlightLimitOpts returns options for constructing in-flight limiting middleware.

func (MiddlewareOpts) RateLimitOpts 

func (opts MiddlewareOpts) RateLimitOpts() RateLimitMiddlewareOpts

RateLimitOpts returns options for constructing rate limiting middleware.

type PrometheusMetrics added in v1.3.0 

type PrometheusMetrics struct {
	InFlightLimitRejects *prometheus.CounterVec
	RateLimitRejects     *prometheus.CounterVec
}

PrometheusMetrics represents a collector of Prometheus metrics for rate/in-flight limiting rejects.

func NewPrometheusMetrics added in v1.3.0 

func NewPrometheusMetrics() *PrometheusMetrics

NewPrometheusMetrics creates a new instance of PrometheusMetrics.

func NewPrometheusMetricsWithOpts added in v1.3.0 

func NewPrometheusMetricsWithOpts(opts PrometheusMetricsOpts) *PrometheusMetrics

NewPrometheusMetricsWithOpts creates a new instance of PrometheusMetrics with the provided options.

func (*PrometheusMetrics) IncInFlightLimitRejects added in v1.3.0 

func (pm *PrometheusMetrics) IncInFlightLimitRejects(ruleName string, dryRun bool, backlogged bool)

IncInFlightLimitRejects increments the counter of rejected requests due to in-flight limit exceeded.

func (*PrometheusMetrics) IncRateLimitRejects added in v1.3.0 

func (pm *PrometheusMetrics) IncRateLimitRejects(ruleName string, dryRun bool)

IncRateLimitRejects increments the counter of rejected requests due to rate limit exceeded.

func (*PrometheusMetrics) MustCurryWith added in v1.3.0 

func (pm *PrometheusMetrics) MustCurryWith(labels prometheus.Labels) *PrometheusMetrics

MustCurryWith curries the metrics collector with the provided labels.

func (*PrometheusMetrics) MustRegister added in v1.3.0 

func (pm *PrometheusMetrics) MustRegister()

MustRegister does registration of metrics collector in Prometheus and panics if any error occurs.

func (*PrometheusMetrics) Unregister added in v1.3.0 

func (pm *PrometheusMetrics) Unregister()

Unregister cancels registration of metrics collector in Prometheus.

type PrometheusMetricsOpts added in v1.3.0 

type PrometheusMetricsOpts struct {
	// Namespace is a namespace for metrics. It will be prepended to all metric names.
	Namespace string

	// ConstLabels is a set of labels that will be applied to all metrics.
	ConstLabels prometheus.Labels

	// CurriedLabelNames is a list of label names that will be curried with the provided labels.
	// See PrometheusMetrics.MustCurryWith method for more details.
	// Keep in mind that if this list is not empty,
	// PrometheusMetrics.MustCurryWith method must be called further with the same labels.
	// Otherwise, the collector will panic.
	CurriedLabelNames []string
}

PrometheusMetricsOpts represents options for PrometheusMetrics.

type RateLimitMiddlewareOpts added in v1.16.0 

type RateLimitMiddlewareOpts struct {
	// GetKeyIdentity is a function that returns identity string representation.
	// The returned string is used as a key for zone when key.type is "identity".
	GetKeyIdentity func(r *http.Request) (key string, bypass bool, err error)

	// RateLimitOnReject is a callback that is called for rejecting HTTP request when the rate limit is exceeded.
	RateLimitOnReject middleware.RateLimitOnRejectFunc

	// RateLimitOnRejectInDryRun is a callback that is called for rejecting HTTP request in the dry-run mode
	// when the rate limit is exceeded.
	RateLimitOnRejectInDryRun middleware.RateLimitOnRejectFunc

	// RateLimitOnError is a callback that is called in case of any error that may occur during the rate limiting.
	RateLimitOnError middleware.RateLimitOnErrorFunc
}

RateLimitMiddlewareOpts represents an options for RateLimitMiddleware.

type RateLimitRetryAfterValue 

type RateLimitRetryAfterValue struct {
	IsAuto   bool
	Duration time.Duration
}

RateLimitRetryAfterValue represents structured retry-after value for rate limiting.

func (RateLimitRetryAfterValue) String added in v1.11.0 

func (ra RateLimitRetryAfterValue) String() string

String returns a string representation of the retry-after value. Implements fmt.Stringer interface.

func (*RateLimitRetryAfterValue) UnmarshalJSON added in v1.11.0 

func (ra *RateLimitRetryAfterValue) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.Unmarshaler interface. Implements the json.Unmarshaler interface.

func (*RateLimitRetryAfterValue) UnmarshalText 

func (ra *RateLimitRetryAfterValue) UnmarshalText(text []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface. Implements the encoding.TextUnmarshaler interface which is used by mapstructure.TextUnmarshallerHookFunc.

func (*RateLimitRetryAfterValue) UnmarshalYAML added in v1.11.0 

func (ra *RateLimitRetryAfterValue) UnmarshalYAML(value *yaml.Node) error

UnmarshalYAML implements the yaml.Unmarshaler interface. Implements the yaml.Unmarshaler interface.

type RateLimitValue 

type RateLimitValue struct {
	Count    int
	Duration time.Duration
}

RateLimitValue represents value for rate limiting.

func (RateLimitValue) String added in v1.11.0 

func (rl RateLimitValue) String() string

String returns a string representation of the rate limit value. Implements fmt.Stringer interface.

func (*RateLimitValue) UnmarshalJSON added in v1.11.0 

func (rl *RateLimitValue) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.Unmarshaler interface. Implements the json.Unmarshaler interface.

func (*RateLimitValue) UnmarshalText 

func (rl *RateLimitValue) UnmarshalText(text []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface. Implements the encoding.TextUnmarshaler interface which is used by mapstructure.TextUnmarshallerHookFunc.

func (*RateLimitValue) UnmarshalYAML added in v1.11.0 

func (rl *RateLimitValue) UnmarshalYAML(value *yaml.Node) error

UnmarshalYAML implements the yaml.Unmarshaler interface. Implements the yaml.Unmarshaler interface.

type RateLimitZoneConfig 

type RateLimitZoneConfig struct {
	ZoneConfig         `mapstructure:",squash" yaml:",inline"`
	Alg                string                   `mapstructure:"alg" yaml:"alg" json:"alg"`
	RateLimit          RateLimitValue           `mapstructure:"rateLimit" yaml:"rateLimit" json:"rateLimit"`
	BurstLimit         int                      `mapstructure:"burstLimit" yaml:"burstLimit" json:"burstLimit"`
	BacklogLimit       int                      `mapstructure:"backlogLimit" yaml:"backlogLimit" json:"backlogLimit"`
	BacklogTimeout     config.TimeDuration      `mapstructure:"backlogTimeout" yaml:"backlogTimeout" json:"backlogTimeout"`
	ResponseRetryAfter RateLimitRetryAfterValue `mapstructure:"responseRetryAfter" yaml:"responseRetryAfter" json:"responseRetryAfter"`
}

RateLimitZoneConfig represents zone configuration for rate limiting.

func (*RateLimitZoneConfig) Validate 

func (c *RateLimitZoneConfig) Validate() error

Validate validates zone configuration for rate limiting.

type RuleConfig 

type RuleConfig struct {
	// Alias is an alternative name for the rule. It will be used as a label in metrics.
	Alias string `mapstructure:"alias" yaml:"alias" json:"alias"`

	// Routes contains a list of routes (HTTP verb + URL path) for which the rule will be applied.
	Routes []restapi.RouteConfig `mapstructure:"routes" yaml:"routes" json:"routes"`

	// ExcludedRoutes contains list of routes (HTTP verb + URL path) to be excluded from throttling limitations.
	// The following service endpoints fit should typically be added to this list:
	// - healthcheck endpoint serving as readiness probe
	// - status endpoint serving as liveness probe
	ExcludedRoutes []restapi.RouteConfig `mapstructure:"excludedRoutes" yaml:"excludedRoutes" json:"excludedRoutes"`

	// Tags is useful when the different rules of the same config should be used by different middlewares.
	// As example let's suppose we would like to have 2 different throttling rules:
	// 1) for absolutely all requests;
	// 2) for all identity-aware (authorized) requests.
	// In the code, we will have 2 middlewares that will be executed on the different steps of the HTTP request serving,
	// and each one should do only its own throttling.
	// We can achieve this using different tags for rules and passing needed tag in the MiddlewareOpts.
	Tags TagsList `mapstructure:"tags" yaml:"tags" json:"tags"`

	// RateLimits contains a list of the rate limiting zones that are used in the rule.
	RateLimits []RuleRateLimit `mapstructure:"rateLimits" yaml:"rateLimits" json:"rateLimits"`

	// InFlightLimits contains a list of the in-flight limiting zones that are used in the rule.
	InFlightLimits []RuleInFlightLimit `mapstructure:"inFlightLimits" yaml:"inFlightLimits" json:"inFlightLimits"`
}

RuleConfig represents configuration for throttling rule.

func (*RuleConfig) Name 

func (c *RuleConfig) Name() string

Name returns throttling rule name.

func (*RuleConfig) Validate 

func (c *RuleConfig) Validate(
	rateLimitZones map[string]RateLimitZoneConfig, inFlightLimitZones map[string]InFlightLimitZoneConfig,
) error

Validate validates throttling rule configuration.

type RuleInFlightLimit 

type RuleInFlightLimit struct {
	Zone string `mapstructure:"zone" yaml:"zone"`
}

RuleInFlightLimit represents rule's in-flight limiting parameters.

type RuleRateLimit 

type RuleRateLimit struct {
	Zone string `mapstructure:"zone" yaml:"zone"`
}

RuleRateLimit represents rule's rate limiting parameters.

type TagsList added in v1.11.0 

type TagsList []string

TagsList represents a list of tags.

func (*TagsList) UnmarshalJSON added in v1.11.0 

func (tl *TagsList) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.Unmarshaler interface.

func (*TagsList) UnmarshalText added in v1.11.0 

func (tl *TagsList) UnmarshalText(text []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface.

func (*TagsList) UnmarshalYAML added in v1.11.0 

func (tl *TagsList) UnmarshalYAML(value *yaml.Node) error

UnmarshalYAML implements the yaml.Unmarshaler interface.

type ZoneConfig 

type ZoneConfig struct {
	Key                ZoneKeyConfig `mapstructure:"key" yaml:"key" json:"key"`
	MaxKeys            int           `mapstructure:"maxKeys" yaml:"maxKeys" json:"maxKeys"`
	ResponseStatusCode int           `mapstructure:"responseStatusCode" yaml:"responseStatusCode" json:"responseStatusCode"`
	DryRun             bool          `mapstructure:"dryRun" yaml:"dryRun" json:"dryRun"`
	IncludedKeys       []string      `mapstructure:"includedKeys" yaml:"includedKeys" json:"includedKeys"`
	ExcludedKeys       []string      `mapstructure:"excludedKeys" yaml:"excludedKeys" json:"excludedKeys"`
}

ZoneConfig represents a basic zone configuration.

func (*ZoneConfig) Validate 

func (c *ZoneConfig) Validate() error

Validate validates zone configuration.

type ZoneKeyConfig 

type ZoneKeyConfig struct {
	// Type determines type of key that will be used for throttling.
	Type ZoneKeyType `mapstructure:"type" yaml:"type" json:"type"`

	// HeaderName is a name of the HTTP request header which value will be used as a key.
	// Matters only when Type is a "header".
	HeaderName string `mapstructure:"headerName" yaml:"headerName" json:"headerName"`

	// NoBypassEmpty specifies whether throttling will be used if the value obtained by the key is empty.
	NoBypassEmpty bool `mapstructure:"noBypassEmpty" yaml:"noBypassEmpty" json:"noBypassEmpty"`
}

ZoneKeyConfig represents a configuration of zone's key.

func (*ZoneKeyConfig) Validate 

func (c *ZoneKeyConfig) Validate() error

Validate validates keys zone configuration.

type ZoneKeyType 

type ZoneKeyType string

ZoneKeyType is a type of keys zone. 

const (
	ZoneKeyTypeNoKey      ZoneKeyType = ""
	ZoneKeyTypeIdentity   ZoneKeyType = "identity"
	ZoneKeyTypeHTTPHeader ZoneKeyType = "header"
	ZoneKeyTypeRemoteAddr ZoneKeyType = "remote_addr"
)

Zone key types.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.