retry

package module
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 15, 2026 License: MIT Imports: 11 Imported by: 0

README

go-httpretry

Trivy Security Scan Testing Go Report Card codecov Go Reference

A flexible HTTP client with automatic retry logic using exponential backoff, built with the Functional Options Pattern.

Table of Contents

Features

  • Automatic Retries: Retries failed requests with configurable exponential backoff
  • Smart Retry Logic: Default retries on network errors, 5xx server errors, and 429 (Too Many Requests)
  • Preset Configurations: Ready-to-use presets for common scenarios (realtime, background, rate-limited, microservice, webhook, critical, fast-fail, etc.)
  • Structured Error Types: Rich error information with RetryError for programmatic error inspection
  • Convenience Methods: Simple HTTP methods (Get, Post, Put, Patch, Delete, Head) with optional request configuration
  • Request Options: Flexible request configuration with WithBody(), WithJSON(), WithHeader(), and WithHeaders()
  • Jitter Support: Optional random jitter to prevent thundering herd problem
  • Retry-After Header: Respects HTTP Retry-After header for rate limiting (RFC 2616)
  • Observability: Built-in support for metrics collection, distributed tracing, and structured logging (uses standard library log/slog by default, interface-driven for custom implementations)
  • Flexible Configuration: Use functional options to customize retry behavior
  • Context Support: Respects context cancellation and timeouts
  • Custom Retry Logic: Pluggable retry checker for custom retry conditions
  • Resource Safe: Automatically closes response bodies before retries to prevent leaks
  • Zero Dependencies: Uses only Go standard library

Installation

Install the package using go get:

go get github.com/appleboy/go-httpretry

Then import it in your Go code:

import "github.com/appleboy/go-httpretry"

Quick Start

Basic Usage (Default Settings)
package main

import (
    "context"
    "log"

    "github.com/appleboy/go-httpretry"
)

func main() {
    // Create a retry client with defaults:
    // - 3 max retries
    // - 1 second initial delay
    // - 10 second max delay
    // - 2.0x exponential multiplier
    // - Jitter enabled (±25% randomization)
    // - Retry-After header respected (HTTP standard compliant)
    // - Structured logging to stderr using log/slog (INFO level)
    client, err := retry.NewClient()
    if err != nil {
        log.Fatal(err)
    }

    // Simple GET request
    // Retry operations will be automatically logged to stderr:
    // 2024/02/14 10:00:00 WARN request failed, will retry method=GET attempt=1 reason=5xx
    // 2024/02/14 10:00:00 INFO retrying request method=GET attempt=2 delay=1s
    resp, err := client.Get(context.Background(), "https://api.example.com/data")
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()
}
Using Convenience Methods
// GET request
resp, err := client.Get(ctx, "https://api.example.com/users")

// POST request with JSON body (automatic marshaling)
type User struct {
    Name  string `json:"name"`
    Email string `json:"email"`
}
user := User{Name: "John", Email: "john@example.com"}
resp, err := client.Post(ctx, "https://api.example.com/users",
    retry.WithJSON(user))

// POST request with raw JSON body
jsonData := bytes.NewReader([]byte(`{"name":"John"}`))
resp, err := client.Post(ctx, "https://api.example.com/users",
    retry.WithBody("application/json", jsonData))

// PUT request with custom headers
resp, err := client.Put(ctx, "https://api.example.com/users/123",
    retry.WithJSON(user),
    retry.WithHeader("Authorization", "Bearer token"))

// DELETE request
resp, err := client.Delete(ctx, "https://api.example.com/users/123")
JSON Requests Made Easy

The WithJSON() helper automatically marshals your data to JSON:

type CreateUserRequest struct {
    Name  string `json:"name"`
    Email string `json:"email"`
    Age   int    `json:"age"`
}

user := CreateUserRequest{
    Name:  "John Doe",
    Email: "john@example.com",
    Age:   30,
}

// Automatically marshals to JSON and sets Content-Type header
resp, err := client.Post(ctx, "https://api.example.com/users",
    retry.WithJSON(user))
Custom Configuration
client, err := retry.NewClient(
    retry.WithMaxRetries(5),                           // Retry up to 5 times
    retry.WithInitialRetryDelay(500*time.Millisecond), // Start with 500ms delay
    retry.WithMaxRetryDelay(30*time.Second),           // Cap delay at 30s
    retry.WithRetryDelayMultiple(3.0),                 // Triple delay each time
)
if err != nil {
    log.Fatal(err)
}
Using Preset Configurations

The library provides optimized presets for common scenarios:

// Realtime client - Fast response times for user-facing requests
client, err := retry.NewRealtimeClient()

// Background client - Reliable background task processing
client, err := retry.NewBackgroundClient()

// Rate-limited client - Respects API rate limits
client, err := retry.NewRateLimitedClient()

// Microservice client - Internal service communication
client, err := retry.NewMicroserviceClient()

// Critical client - Mission-critical operations (payments, etc.)
client, err := retry.NewCriticalClient()

// Fast-fail client - Health checks and service discovery
client, err := retry.NewFastFailClient()

All presets can be customized by passing additional options:

// Start with realtime preset but use more retries
client, err := retry.NewRealtimeClient(
    retry.WithMaxRetries(5), // Override preset default
)

Documentation

For detailed documentation, please refer to:

  • Preset Configurations - Pre-configured clients for common scenarios (realtime, background, rate-limited, microservice, webhook, critical, fast-fail, etc.)
  • Configuration Options - All available configuration options including retry behavior, HTTP client settings, custom TLS, and request options
  • Error Handling - Structured error handling with RetryError and response inspection
  • Observability - Metrics collection, distributed tracing, and structured logging (OpenTelemetry, Prometheus, slog integration patterns)
  • Examples - Detailed usage examples for various scenarios
Key Topics
Exponential Backoff

Retries use exponential backoff to avoid overwhelming the server:

  1. First retry: Wait initialRetryDelay (default: 1s)
  2. Second retry: Wait initialRetryDelay * multiplier (default: 2s)
  3. Third retry: Wait initialRetryDelay * multiplier² (default: 4s)
  4. Subsequent retries: Continue multiplying until maxRetryDelay is reached
Default Retry Behavior

The DefaultRetryableChecker retries in the following cases:

  • Network errors: Connection refused, timeouts, DNS errors, etc.
  • 5xx Server Errors: 500, 502, 503, 504, etc.
  • 429 Too Many Requests: Rate limiting errors

It does NOT retry:

  • 4xx Client Errors (except 429): 400, 401, 403, 404, etc.
  • 2xx Success: 200, 201, 204, etc.
  • 3xx Redirects: 301, 302, 307, etc.
Context Support

The client respects context cancellation and timeouts. There are two ways to pass context:

Option 1: Use request's context (recommended)

// Overall timeout for the entire operation (including retries)
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

req, _ := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.example.com/data", nil)
resp, err := client.Do(req)
if err != nil {
    // May be context.DeadlineExceeded
    log.Printf("Request failed: %v", err)
}

Option 2: Use DoWithContext for explicit context

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

req, _ := http.NewRequest(http.MethodGet, "https://api.example.com/data", nil)
resp, err := client.DoWithContext(ctx, req)
if err != nil {
    log.Printf("Request failed: %v", err)
}
Complete Working Examples

For complete, runnable examples, see:

Each example can be run independently:

cd _example/basic && go run main.go
cd _example/advanced && go run main.go
cd _example/convenience_methods && go run main.go
cd _example/request_options && go run main.go
cd _example/large_file_upload && go run main.go
⚠️ Important: Large File Uploads

Do NOT use WithBody() or WithJSON() for files larger than 10MB. These functions buffer the entire body in memory to support retries.

For large files, use the Do() method with a custom GetBody function:

// ✅ CORRECT: Upload large files with retry support
file, _ := os.Open("large-file.dat")
req, _ := http.NewRequestWithContext(ctx, "POST", url, file)

// CRITICAL: Set GetBody to reopen the file for each retry
req.GetBody = func() (io.ReadCloser, error) {
    return os.Open("large-file.dat")
}

resp, err := client.Do(req)

Size Guidelines:

  • <1MB: Safe to use WithBody() or WithJSON()
  • ⚠️ 1-10MB: Use with caution, monitor memory usage
  • >10MB: Use Do() with GetBody (see large_file_upload example)

For complete patterns and best practices, see the large_file_upload example with detailed explanations.

Testing

Run the test suite:

go test -v ./...

With coverage:

go test -v -cover ./...

Or use the Makefile:

make test
make lint

Design Principles

  • Functional Options Pattern: Provides clean, flexible API for both client configuration and request options
  • Sensible Defaults: Works out of the box for most use cases
  • Convenience Methods: Simple HTTP methods (Get, Post, Put, Patch, Delete, Head) with optional configuration through RequestOption functions
  • Separation of Concerns: HTTP client configuration (including TLS) is the user's responsibility; retry logic is ours
  • Single Responsibility: Focus exclusively on retry behavior, not HTTP client building
  • Context-Aware: Respects cancellation and timeouts
  • Resource Safe: Prevents response body leaks by closing them before retries
  • Request Cloning: Clones requests for each retry to handle consumed request bodies
  • Zero Dependencies: Uses only standard library

License

This project is licensed under the MIT License - see the LICENSE file for details.

Copyright (c) 2026 Bo-Yi Wu

Author

Support this project:

Donate

Documentation

Overview

Example (Basic)

Example_basic demonstrates basic usage with default configuration

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Create a retry client with default settings
	// (3 retries, 1s initial delay, 10s max delay, 2.0 multiplier)
	client, err := retry.NewClient()
	if err != nil {
		log.Fatal(err)
	}

	// Create a request
	ctx := context.Background()
	req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.example.com/data", nil)
	if err != nil {
		log.Fatal(err)
	}

	// Execute with automatic retries
	resp, err := client.Do(req)
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	if resp.StatusCode == http.StatusOK {
		fmt.Println("Request succeeded")
	}
}
Example (CustomConfiguration)

Example_customConfiguration demonstrates custom retry configuration

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"time"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Create a retry client with custom settings
	client, err := retry.NewClient(
		retry.WithMaxRetries(5),                           // Retry up to 5 times
		retry.WithInitialRetryDelay(500*time.Millisecond), // Start with 500ms delay
		retry.WithMaxRetryDelay(30*time.Second),           // Cap delay at 30s
		retry.WithRetryDelayMultiple(3.0),                 // Triple delay each time
	)
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	req, err := http.NewRequestWithContext(
		ctx,
		http.MethodPost,
		"https://api.example.com/submit",
		nil,
	)
	if err != nil {
		log.Fatal(err)
	}
	resp, err := client.Do(req)
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Printf("Status: %d\n", resp.StatusCode)
}
Example (CustomHTTPClient)

Example_customHTTPClient demonstrates using a custom http.Client

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"time"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Create a custom http.Client with specific settings
	httpClient := &http.Client{
		Timeout: 10 * time.Second,
		Transport: &http.Transport{
			MaxIdleConns:        100,
			MaxIdleConnsPerHost: 10,
			IdleConnTimeout:     90 * time.Second,
		},
	}

	// Use the custom client with retry logic
	client, err := retry.NewClient(
		retry.WithHTTPClient(httpClient),
		retry.WithMaxRetries(3),
		retry.WithInitialRetryDelay(1*time.Second),
	)
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.example.com/data", nil)
	if err != nil {
		log.Fatal(err)
	}

	resp, err := client.Do(req)
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Printf("Response received: %d\n", resp.StatusCode)
}
Example (CustomRetryChecker)

Example_customRetryChecker demonstrates custom retry logic

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"time"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Custom checker that also retries on 403 Forbidden
	customChecker := func(err error, resp *http.Response) bool {
		if err != nil {
			return true // Retry on network errors
		}
		if resp == nil {
			return false
		}

		// Retry on 5xx, 429, and also 403
		statusCode := resp.StatusCode
		return statusCode >= 500 ||
			statusCode == http.StatusTooManyRequests ||
			statusCode == http.StatusForbidden
	}

	client, err := retry.NewClient(
		retry.WithRetryableChecker(customChecker),
		retry.WithMaxRetries(3),
		retry.WithInitialRetryDelay(1*time.Second),
	)
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.example.com/data", nil)
	if err != nil {
		log.Fatal(err)
	}

	resp, err := client.Do(req)
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Printf("Final status: %d\n", resp.StatusCode)
}
Example (CustomTLSConfiguration)

Example_customTLSConfiguration demonstrates how to configure TLS when using go-httpretry with services that require custom certificates.

This example shows the recommended approach: configure your http.Client with the desired TLS settings, then pass it to the retry client.

package main

import (
	"context"
	"crypto/tls"
	"crypto/x509"
	"fmt"
	"log"
	"net/http"
	"os"
	"time"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Step 1: Load custom certificate
	certPool, err := x509.SystemCertPool()
	if err != nil {
		// Fall back to empty pool if system pool unavailable
		certPool = x509.NewCertPool()
	}

	certPEM, err := os.ReadFile("/path/to/internal-ca.pem")
	if err != nil {
		log.Fatal(err)
	}

	if !certPool.AppendCertsFromPEM(certPEM) {
		log.Fatal("failed to append certificate")
	}

	// Step 2: Create TLS configuration
	tlsConfig := &tls.Config{
		RootCAs:    certPool,
		MinVersion: tls.VersionTLS12, // Enforce TLS 1.2+
	}

	// Step 3: Create HTTP client with TLS config
	httpClient := &http.Client{
		Transport: &http.Transport{
			TLSClientConfig: tlsConfig,
		},
		Timeout: 30 * time.Second,
	}

	// Step 4: Create retry client with pre-configured HTTP client
	client, err := retry.NewClient(
		retry.WithHTTPClient(httpClient),
		retry.WithMaxRetries(3),
		retry.WithJitter(true),
	)
	if err != nil {
		log.Fatal(err)
	}

	// Step 5: Use the retry client normally
	ctx := context.Background()
	req, _ := http.NewRequestWithContext(
		ctx,
		http.MethodGet,
		"https://internal.company.com/api",
		nil,
	)

	resp, err := client.Do(req)
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Printf("Status: %d\n", resp.StatusCode)
}
Example (DoWithContext)

Example_doWithContext demonstrates using DoWithContext for explicit context control. Use DoWithContext when you need to pass a different context than the one in the request, or when you want explicit control over the context used for retries.

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"time"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	client, err := retry.NewClient(
		retry.WithMaxRetries(3),
		retry.WithInitialRetryDelay(1*time.Second),
	)
	if err != nil {
		log.Fatal(err)
	}

	// Create context with timeout
	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()

	// Create request with background context first
	req, err := http.NewRequestWithContext(
		context.Background(),
		http.MethodGet,
		"https://api.example.com/data",
		nil,
	)
	if err != nil {
		cancel()
		log.Fatal(err) //nolint:gocritic // cancel() is called before Fatal
	}

	// Use DoWithContext to override the request's context with our timeout context
	resp, err := client.DoWithContext(ctx, req)
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Printf("Request failed: %v", err)
		return
	}
	defer resp.Body.Close()

	fmt.Printf("Status: %d\n", resp.StatusCode)
}
Example (InsecureTLS)

Example_insecureTLS demonstrates how to skip TLS verification for testing. WARNING: Only use this in development/testing environments!

package main

import (
	"context"
	"crypto/tls"
	"fmt"
	"log"
	"net/http"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Create HTTP client with insecure TLS config
	httpClient := &http.Client{
		Transport: &http.Transport{
			TLSClientConfig: &tls.Config{
				InsecureSkipVerify: true, // #nosec G402
			},
		},
	}

	client, err := retry.NewClient(
		retry.WithHTTPClient(httpClient),
		retry.WithMaxRetries(2),
	)
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	req, _ := http.NewRequestWithContext(
		ctx,
		http.MethodGet,
		"https://self-signed.badssl.com/",
		nil,
	)

	resp, err := client.Do(req)
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Printf("Connected (insecure): %d\n", resp.StatusCode)
}
Example (LargeFileUpload)

Example_largeFileUpload demonstrates the correct way to upload large files with retry support. This is important because WithBody() buffers the entire body in memory, which is not suitable for large files.

package main

import (
	"context"
	"fmt"
	"io"
	"log"
	"net/http"
	"os"
	"time"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	client, err := retry.NewClient(
		retry.WithMaxRetries(3),
		retry.WithInitialRetryDelay(1*time.Second),
	)
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	filePath := "/path/to/large-file.dat"

	// ❌ WRONG: Do NOT use WithBody for large files (buffers entire file in memory)
	// file, _ := os.ReadFile(filePath)  // Loads entire file into memory!
	// resp, _ := client.Post(ctx, "https://api.example.com/upload",
	//     retry.WithBody("application/octet-stream", bytes.NewReader(file)))

	// ✅ CORRECT: Use Do() with GetBody for large files (memory efficient)

	// Open the file
	file, err := os.Open(filePath)
	if err != nil {
		log.Fatal(err)
	}

	// Get file info for Content-Length
	stat, err := file.Stat()
	if err != nil {
		file.Close()
		log.Fatal(err)
	}

	// Create request with file as body
	req, err := http.NewRequestWithContext(ctx, http.MethodPost,
		"https://api.example.com/upload", file)
	if err != nil {
		file.Close()
		log.Fatal(err)
	}

	req.Header.Set("Content-Type", "application/octet-stream")
	req.ContentLength = stat.Size()

	// CRITICAL: Set GetBody to reopen the file for each retry attempt
	// This enables retry support without buffering the entire file in memory
	req.GetBody = func() (io.ReadCloser, error) {
		return os.Open(filePath)
	}

	// Execute with automatic retry support
	resp, err := client.Do(req)
	file.Close() // Close the file after making the request
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Printf("Upload completed: %d\n", resp.StatusCode)
}
Example (NoRetries)

Example_noRetries demonstrates disabling retries

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Set maxRetries to 0 to disable retries
	client, err := retry.NewClient(
		retry.WithMaxRetries(0),
	)
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.example.com/data", nil)
	if err != nil {
		log.Fatal(err)
	}

	resp, err := client.Do(req)
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Println("Request executed once (no retries)")
}
Example (PresetAggressiveClient)

Example_presetAggressiveClient demonstrates using the aggressive preset for scenarios with frequent transient failures

package main

import (
	"context"
	"fmt"
	"log"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Use the aggressive preset for unreliable networks
	// (10 retries, 100ms initial delay, 5s max delay)
	client, err := retry.NewAggressiveClient()
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	resp, err := client.Get(ctx, "https://unreliable-api.example.com/data")
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Println("Data retrieved after aggressive retries")
}
Example (PresetBackgroundClient)

Example_presetBackgroundClient demonstrates using the background preset for non-time-sensitive operations like batch jobs

package main

import (
	"context"
	"fmt"
	"log"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Use the background preset for background tasks
	// (10 retries, 5s initial delay, 60s max delay, 3.0x multiplier, 30s per-attempt timeout)
	client, err := retry.NewBackgroundClient()
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	resp, err := client.Post(ctx, "https://api.example.com/batch/sync")
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Println("Batch sync completed")
}
Example (PresetConservativeClient)

Example_presetConservativeClient demonstrates using the conservative preset for operations where you want to be cautious about retry storms

package main

import (
	"context"
	"fmt"
	"log"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Use the conservative preset to avoid retry storms
	// (2 retries, 5s initial delay)
	client, err := retry.NewConservativeClient()
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	resp, err := client.Post(ctx, "https://api.example.com/expensive-operation")
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Println("Expensive operation completed")
}
Example (PresetMicroserviceClient)

Example_presetMicroserviceClient demonstrates using the microservice preset for internal service-to-service communication

package main

import (
	"context"
	"fmt"
	"log"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Use the microservice preset for internal calls
	// (3 retries, 50ms initial delay, 500ms max delay, 2s per-attempt timeout, jitter enabled)
	client, err := retry.NewMicroserviceClient()
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	resp, err := client.Get(ctx, "http://user-service:8080/users/123")
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Println("User data retrieved from internal service")
}
Example (PresetRateLimitedClient)

Example_presetRateLimitedClient demonstrates using the rate-limited preset for APIs with strict rate limits and Retry-After headers

package main

import (
	"context"
	"fmt"
	"log"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Use the rate-limited preset for third-party APIs
	// (5 retries, 2s initial delay, respects Retry-After header, jitter enabled)
	client, err := retry.NewRateLimitedClient()
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	resp, err := client.Get(ctx, "https://api.github.com/users/appleboy")
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Printf("GitHub API responded: %d\n", resp.StatusCode)
}
Example (PresetRealtimeClient)

Example_presetRealtimeClient demonstrates using the realtime preset for user-facing requests that require fast response times

package main

import (
	"context"
	"fmt"
	"log"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Use the realtime preset for user-facing operations
	// (2 retries, 100ms initial delay, 1s max delay, 3s per-attempt timeout)
	client, err := retry.NewRealtimeClient()
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	resp, err := client.Get(ctx, "https://api.example.com/search?q=hello")
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Println("Search results retrieved")
}
Example (PresetWithCustomOverride)

Example_presetWithCustomOverride demonstrates overriding preset defaults

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Start with a preset and override specific settings
	client, err := retry.NewRealtimeClient(
		retry.WithMaxRetries(5),                          // Override: more retries than default (2)
		retry.WithInitialRetryDelay(50*time.Millisecond), // Override: faster initial retry
	)
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	resp, err := client.Get(ctx, "https://api.example.com/data")
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Println("Request completed with custom realtime config")
}
Example (StandardInterface)

Example_standardInterface demonstrates that retry.Client is compatible with http.Client interface. You can use retry.Client anywhere that accepts the standard Do(req) signature.

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"time"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	// Create retry client
	retryClient, err := retry.NewClient(
		retry.WithMaxRetries(3),
		retry.WithInitialRetryDelay(1*time.Second),
	)
	if err != nil {
		log.Fatal(err)
	}

	// Function that accepts anything with Do(*http.Request) signature
	executeRequest := func(doer interface {
		Do(*http.Request) (*http.Response, error)
	}, url string,
	) error {
		req, err := http.NewRequestWithContext(context.Background(), http.MethodGet, url, nil)
		if err != nil {
			return err
		}

		resp, err := doer.Do(req)
		if err != nil {
			if resp != nil && resp.Body != nil {
				resp.Body.Close()
			}
			return err
		}
		defer resp.Body.Close()

		fmt.Printf("Status: %d\n", resp.StatusCode)
		return nil
	}

	// Works with retry.Client
	if err := executeRequest(retryClient, "https://api.example.com/data"); err != nil {
		log.Printf("Request failed: %v", err)
	}

	// Also works with standard http.Client
	if err := executeRequest(http.DefaultClient, "https://api.example.com/data"); err != nil {
		log.Printf("Request failed: %v", err)
	}
}
Example (WithTimeout)

Example_withTimeout demonstrates using context timeout

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"time"

	retry "github.com/appleboy/go-httpretry"
)

func main() {
	client, err := retry.NewClient(
		retry.WithMaxRetries(10),
		retry.WithInitialRetryDelay(2*time.Second),
	)
	if err != nil {
		log.Fatal(err)
	}

	// Set overall timeout for the operation (including retries)
	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()

	req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.example.com/data", nil)
	if err != nil {
		cancel()
		log.Fatal(err) //nolint:gocritic // cancel() is called before Fatal
	}

	resp, err := client.Do(req)
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		// May be context deadline exceeded if retries take too long
		log.Printf("Request failed: %v", err)
		return
	}
	defer resp.Body.Close()

	fmt.Println("Request completed within timeout")
}

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func DefaultRetryableChecker

func DefaultRetryableChecker(err error, resp *http.Response) bool

DefaultRetryableChecker is the default implementation for determining retryable errors It retries on network errors and 5xx/429 status codes

Types

type Attribute added in v0.10.0

type Attribute struct {
	Key   string
	Value any
}

Attribute represents a key-value pair attribute

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is an HTTP client with automatic retry logic using exponential backoff

func NewAggressiveClient added in v0.7.0

func NewAggressiveClient(opts ...Option) (*Client, error)

NewAggressiveClient creates a client with aggressive retry behavior. This preset attempts many retries with short delays, suitable for scenarios where transient failures are common and quick recovery is expected.

Configuration:

  • Max retries: 10 (many retry attempts)
  • Initial delay: 100ms (very short initial delay)
  • Max delay: 5s (moderate maximum delay)
  • Per-attempt timeout: 10s (prevent slow requests)
  • Jitter: enabled (prevent synchronized retries)

Use cases:

  • Highly unreliable networks
  • Services with frequent transient failures
  • Scenarios where eventual success is expected

func NewBackgroundClient added in v0.7.0

func NewBackgroundClient(opts ...Option) (*Client, error)

NewBackgroundClient creates a client optimized for background tasks. This preset is designed for non-time-sensitive operations where reliability is more important than speed, such as batch processing, scheduled jobs, or data sync.

Configuration:

  • Max retries: 10 (persistent retries for reliability)
  • Initial delay: 5s (longer initial backoff)
  • Max delay: 60s (up to 1 minute between retries)
  • Backoff multiplier: 3.0 (aggressive exponential backoff)
  • Per-attempt timeout: 30s (generous timeout per attempt)
  • Jitter: enabled (prevent synchronized retries)

Use cases:

  • Batch data synchronization
  • Scheduled/cron jobs
  • Data export/import operations
  • Async task processing

func NewClient

func NewClient(opts ...Option) (*Client, error)

NewClient creates a new retry-enabled HTTP client with the given options. Returns an error if any option encounters an error.

func NewConservativeClient added in v0.7.0

func NewConservativeClient(opts ...Option) (*Client, error)

NewConservativeClient creates a client with conservative retry behavior. This preset uses fewer retries with longer delays, suitable for scenarios where you want to be cautious about retry storms or when failures are likely permanent.

Configuration:

  • Max retries: 2 (few retry attempts)
  • Initial delay: 5s (long initial delay)
  • Per-attempt timeout: 20s (generous timeout)
  • Jitter: enabled (prevent synchronized retries)

Use cases:

  • External APIs with strict limits
  • Operations where failures are likely permanent
  • Preventing retry storms during outages

func NewCriticalClient added in v0.9.0

func NewCriticalClient(opts ...Option) (*Client, error)

NewCriticalClient creates a client for mission-critical operations. This preset ensures maximum effort to succeed, suitable for operations that absolutely must complete, such as payment processing or critical data sync.

Configuration:

  • Max retries: 15 (many retry attempts)
  • Initial delay: 1s (reasonable initial backoff)
  • Max delay: 120s (up to 2 minutes between retries)
  • Backoff multiplier: 2.0 (standard exponential backoff)
  • Per-attempt timeout: 60s (generous timeout per attempt)
  • Jitter: enabled (prevent synchronized retries)
  • Respect Retry-After: enabled (honor server guidance)

Use cases:

  • Payment processing
  • Order confirmation
  • Critical data synchronization
  • Operations that cannot fail

func NewFastFailClient added in v0.9.0

func NewFastFailClient(opts ...Option) (*Client, error)

NewFastFailClient creates a client optimized for fast failure scenarios. This preset minimizes retry delays and attempts, suitable for operations where you need to know about failures quickly.

Configuration:

  • Max retries: 1 (single retry attempt)
  • Initial delay: 50ms (minimal delay)
  • Max delay: 200ms (very short maximum)
  • Per-attempt timeout: 1s (short timeout)
  • Jitter: enabled (prevent synchronized retries)

Use cases:

  • Health checks
  • Service discovery
  • Quick availability probes
  • Circuit breaker implementations

func NewMicroserviceClient added in v0.7.0

func NewMicroserviceClient(opts ...Option) (*Client, error)

NewMicroserviceClient creates a client optimized for internal microservice communication. This preset uses very short delays suitable for high-speed internal networks where services are geographically close (e.g., within the same Kubernetes cluster).

Configuration:

  • Max retries: 3 (moderate retry count)
  • Initial delay: 50ms (very short delay for internal network)
  • Max delay: 500ms (sub-second maximum)
  • Per-attempt timeout: 2s (fast timeout for internal calls)
  • Jitter: enabled (prevent synchronized retries)

Use cases:

  • Kubernetes pod-to-pod communication
  • Internal service mesh calls
  • Low-latency internal APIs
  • gRPC fallback to HTTP

func NewRateLimitedClient added in v0.7.0

func NewRateLimitedClient(opts ...Option) (*Client, error)

NewRateLimitedClient creates a client optimized for APIs with strict rate limits. This preset respects server-provided Retry-After headers and uses jitter to prevent thundering herd problems when multiple clients retry simultaneously.

Configuration:

  • Max retries: 5 (balanced retry count)
  • Initial delay: 2s (moderate initial backoff)
  • Max delay: 30s (reasonable maximum wait)
  • Per-attempt timeout: 15s (prevent slow requests)
  • Respect Retry-After: enabled (honor server guidance)
  • Jitter: enabled (prevent synchronized retries)

Use cases:

  • Third-party APIs (GitHub, Stripe, AWS, etc.)
  • Services returning 429 Too Many Requests
  • APIs with published rate limits
  • Services providing Retry-After headers

func NewRealtimeClient added in v0.7.0

func NewRealtimeClient(opts ...Option) (*Client, error)

NewRealtimeClient creates a client optimized for realtime user-facing requests. This preset is designed for scenarios where fast response times are critical, such as user interactions, search suggestions, or API calls triggered by UI actions.

Configuration:

  • Max retries: 2 (quick failure for better UX)
  • Initial delay: 100ms (minimal wait time)
  • Max delay: 1s (short maximum delay)
  • Per-attempt timeout: 3s (prevent slow requests)

Use cases:

  • User-initiated API calls
  • Real-time search and autocomplete
  • Interactive UI operations requiring fast failure

func NewWebhookClient added in v0.9.0

func NewWebhookClient(opts ...Option) (*Client, error)

NewWebhookClient creates a client optimized for webhook/callback scenarios. This preset is designed for outbound webhook calls where the sender typically has its own retry mechanism, so quick failure is preferred over aggressive retries.

Configuration:

  • Max retries: 1 (single retry attempt)
  • Initial delay: 500ms (quick retry)
  • Max delay: 1s (short maximum delay)
  • Per-attempt timeout: 5s (reasonable timeout)
  • Jitter: enabled (prevent synchronized retries)

Use cases:

  • Sending webhooks to external services
  • Third-party integration callbacks
  • Event notification systems
  • Outbound webhook deliveries

func (*Client) Delete added in v0.7.0

func (c *Client) Delete(
	ctx context.Context,
	url string,
	opts ...RequestOption,
) (*http.Response, error)

Delete is a convenience method for making DELETE requests with retry logic. It creates a DELETE request for the specified URL and executes it with the configured retry behavior.

func (*Client) Do

func (c *Client) Do(req *http.Request) (*http.Response, error)

Do executes an HTTP request with automatic retry logic using exponential backoff. This method is compatible with the standard http.Client interface. The context is taken from the request via req.Context().

For large file uploads or streaming data, set req.GetBody to enable retries:

file, _ := os.Open("large-file.dat")
req, _ := http.NewRequestWithContext(ctx, "POST", url, file)
req.GetBody = func() (io.ReadCloser, error) {
    return os.Open("large-file.dat")  // Reopen for each retry
}
resp, err := client.Do(req)

See the large_file_upload example for complete implementation patterns.

func (*Client) DoWithContext added in v0.8.0

func (c *Client) DoWithContext(ctx context.Context, req *http.Request) (*http.Response, error)

DoWithContext executes an HTTP request with automatic retry logic using exponential backoff. Use this when you need explicit control over the context separate from the request.

func (*Client) Get added in v0.7.0

func (c *Client) Get(
	ctx context.Context,
	url string,
	opts ...RequestOption,
) (*http.Response, error)

Get is a convenience method for making GET requests with retry logic. It creates a GET request for the specified URL and executes it with the configured retry behavior.

func (*Client) Head added in v0.7.0

func (c *Client) Head(
	ctx context.Context,
	url string,
	opts ...RequestOption,
) (*http.Response, error)

Head is a convenience method for making HEAD requests with retry logic. It creates a HEAD request for the specified URL and executes it with the configured retry behavior.

func (*Client) Patch added in v0.7.0

func (c *Client) Patch(
	ctx context.Context,
	url string,
	opts ...RequestOption,
) (*http.Response, error)

Patch is a convenience method for making PATCH requests with retry logic. It creates a PATCH request with the specified URL and executes it with the configured retry behavior. Use WithBody() to add a request body and content type.

func (*Client) Post added in v0.7.0

func (c *Client) Post(
	ctx context.Context,
	url string,
	opts ...RequestOption,
) (*http.Response, error)

Post is a convenience method for making POST requests with retry logic. It creates a POST request with the specified URL and executes it with the configured retry behavior. Use WithBody() to add a request body and content type.

func (*Client) Put added in v0.7.0

func (c *Client) Put(
	ctx context.Context,
	url string,
	opts ...RequestOption,
) (*http.Response, error)

Put is a convenience method for making PUT requests with retry logic. It creates a PUT request with the specified URL and executes it with the configured retry behavior. Use WithBody() to add a request body and content type.

type Logger added in v0.10.0

type Logger interface {
	Debug(msg string, args ...any)
	Info(msg string, args ...any)
	Warn(msg string, args ...any)
	Error(msg string, args ...any)
}

Logger defines the structured logging interface (slog-compatible)

type MetricsCollector added in v0.10.0

type MetricsCollector interface {
	// RecordAttempt records a single request attempt
	RecordAttempt(method string, statusCode int, duration time.Duration, err error)

	// RecordRetry records a retry event
	RecordRetry(method string, reason string, attemptNumber int)

	// RecordRequestComplete records request completion (including all retries)
	RecordRequestComplete(
		method string,
		statusCode int,
		totalDuration time.Duration,
		totalAttempts int,
		success bool,
	)
}

MetricsCollector defines the interface for collecting metrics (thread-safe)

type OnRetryFunc added in v0.3.0

type OnRetryFunc func(info RetryInfo)

OnRetryFunc is called before each retry attempt

type Option

type Option func(*Client)

Option configures a Client

func WithHTTPClient

func WithHTTPClient(httpClient *http.Client) Option

WithHTTPClient sets a custom http.Client

func WithInitialRetryDelay

func WithInitialRetryDelay(d time.Duration) Option

WithInitialRetryDelay sets the initial delay before the first retry

func WithJitter added in v0.3.0

func WithJitter(enabled bool) Option

WithJitter enables random jitter to prevent thundering herd problem. When enabled, retry delays will be randomized by ±25% to avoid synchronized retries from multiple clients hitting the server at the same time.

func WithLogger added in v0.10.0

func WithLogger(logger Logger) Option

WithLogger sets the structured logger for observability. The logger will output structured logs for request lifecycle events. By default, the client uses slog.Default() which outputs to stderr at INFO level. If nil is provided, logging will be disabled (no-op), equivalent to WithNoLogging().

func WithMaxRetries

func WithMaxRetries(n int) Option

WithMaxRetries sets the maximum number of retry attempts

func WithMaxRetryDelay

func WithMaxRetryDelay(d time.Duration) Option

WithMaxRetryDelay sets the maximum delay between retries

func WithMetrics added in v0.10.0

func WithMetrics(collector MetricsCollector) Option

WithMetrics sets the metrics collector for observability. The collector will receive metrics events for each request attempt, retry, and completion. If nil is provided, metrics collection will be disabled (no-op).

func WithNoLogging added in v0.10.0

func WithNoLogging() Option

WithNoLogging disables all logging output. By default, the client uses slog.Default() which outputs to stderr. Use this option if you want to suppress all log messages.

func WithOnRetry added in v0.3.0

func WithOnRetry(fn OnRetryFunc) Option

WithOnRetry sets a callback function that will be called before each retry attempt. This is useful for logging, metrics collection, or custom retry logic.

func WithPerAttemptTimeout added in v0.6.0

func WithPerAttemptTimeout(d time.Duration) Option

WithPerAttemptTimeout sets a timeout for each individual retry attempt. This prevents a single slow request from consuming all available retry time. If set to 0 (default), no per-attempt timeout is applied. The per-attempt timeout is independent of the overall context timeout.

func WithRespectRetryAfter added in v0.3.0

func WithRespectRetryAfter(enabled bool) Option

WithRespectRetryAfter enables respecting the Retry-After header from HTTP responses. When enabled, the client will use the server-provided retry delay instead of the exponential backoff delay. This is useful for rate limiting scenarios. The Retry-After header can be either a number of seconds or an HTTP-date.

func WithRetryDelayMultiple

func WithRetryDelayMultiple(multiplier float64) Option

WithRetryDelayMultiple sets the exponential backoff multiplier

func WithRetryableChecker

func WithRetryableChecker(checker RetryableChecker) Option

WithRetryableChecker sets a custom function to determine retryable errors

func WithTracer added in v0.10.0

func WithTracer(tracer Tracer) Option

WithTracer sets the distributed tracer for observability. The tracer will create spans for each request and attempt, providing distributed tracing support. If nil is provided, tracing will be disabled (no-op).

type RequestOption added in v0.7.0

type RequestOption func(*http.Request)

RequestOption is a function that configures an HTTP request

func WithBody added in v0.7.0

func WithBody(contentType string, body io.Reader) RequestOption

WithBody sets the request body and optionally the Content-Type header. If contentType is empty, no Content-Type header will be set.

⚠️ MEMORY USAGE WARNING: To support retries, this function buffers the ENTIRE body in memory using io.ReadAll. This is ideal for small payloads like JSON/XML API requests (typically <1MB), but NOT suitable for large files or streaming data.

Size Guidelines:

  • ✅ Small payloads (<1MB): Safe to use WithBody
  • ⚠️ Medium payloads (1-10MB): Use with caution, consider memory constraints
  • ❌ Large payloads (>10MB): DO NOT use WithBody, use Do() with GetBody instead

For large files or streaming data, use the Do() method directly with a custom GetBody function that can reopen the file/stream for each retry attempt. See the large_file_upload example for proper implementation.

Example (small JSON payload):

jsonData := []byte(`{"user":"john"}`)
resp, err := client.Post(ctx, url,
    retry.WithBody("application/json", bytes.NewReader(jsonData)))

For large files, see Do() method and the large_file_upload example instead.

func WithHeader added in v0.7.0

func WithHeader(key, value string) RequestOption

WithHeader sets a header key-value pair on the request.

func WithHeaders added in v0.7.0

func WithHeaders(headers map[string]string) RequestOption

WithHeaders sets multiple headers on the request.

func WithJSON added in v0.9.0

func WithJSON(v any) RequestOption

WithJSON serializes the given value to JSON and sets it as the request body. It automatically sets the Content-Type header to "application/json".

⚠️ MEMORY USAGE WARNING: Like WithBody, this function buffers the entire JSON payload in memory to support retries. This is ideal for typical API requests with small to medium JSON objects, but NOT suitable for very large JSON documents.

Size Guidelines:

  • ✅ Typical API payloads: Safe to use (most API requests are <100KB)
  • ⚠️ Large JSON (1-10MB): Use with caution
  • ❌ Very large JSON (>10MB): Use Do() with custom GetBody instead

If JSON marshaling fails, the request will fail when executed with an error.

Example (typical API request):

type User struct {
    Name  string `json:"name"`
    Email string `json:"email"`
}
user := User{Name: "John", Email: "john@example.com"}
resp, err := client.Post(ctx, url, retry.WithJSON(user))

For large JSON documents, consider streaming or use Do() with custom GetBody.

type RetryError added in v0.7.0

type RetryError struct {
	Attempts   int           // Total number of attempts made (initial + retries)
	LastErr    error         // The last error that occurred (nil if last attempt had non-retryable status)
	LastStatus int           // HTTP status code from the last attempt (0 if request failed)
	Elapsed    time.Duration // Total time elapsed from first attempt to final failure
}

RetryError is returned when all retry attempts have been exhausted. It provides detailed information about the retry attempts and the final failure.

func (*RetryError) Error added in v0.7.0

func (e *RetryError) Error() string

Error implements the error interface

func (*RetryError) Unwrap added in v0.7.0

func (e *RetryError) Unwrap() error

Unwrap returns the underlying error for error unwrapping

type RetryInfo added in v0.3.0

type RetryInfo struct {
	Attempt      int           // Current attempt number (1-indexed)
	Delay        time.Duration // Delay before this retry
	Err          error         // Error that triggered the retry (nil if retrying due to response status)
	StatusCode   int           // HTTP status code (0 if request failed)
	RetryAfter   time.Duration // Retry-After duration from response header (0 if not present)
	TotalElapsed time.Duration // Total time elapsed since first attempt
}

RetryInfo contains information about a retry attempt

type RetryableChecker

type RetryableChecker func(err error, resp *http.Response) bool

RetryableChecker determines if an error or response should trigger a retry

type SlogAdapter added in v0.10.0

type SlogAdapter struct {
	// contains filtered or unexported fields
}

SlogAdapter adapts log/slog.Logger to the retry.Logger interface

func NewSlogAdapter added in v0.10.0

func NewSlogAdapter(logger *slog.Logger) *SlogAdapter

NewSlogAdapter creates a slog adapter

func (*SlogAdapter) Debug added in v0.10.0

func (s *SlogAdapter) Debug(msg string, args ...any)

func (*SlogAdapter) Error added in v0.10.0

func (s *SlogAdapter) Error(msg string, args ...any)

func (*SlogAdapter) Info added in v0.10.0

func (s *SlogAdapter) Info(msg string, args ...any)

func (*SlogAdapter) Warn added in v0.10.0

func (s *SlogAdapter) Warn(msg string, args ...any)

type Span added in v0.10.0

type Span interface {
	End()
	SetAttributes(attrs ...Attribute)
	SetStatus(code string, description string)
	AddEvent(name string, attrs ...Attribute)
}

Span represents a tracing span (OpenTelemetry-compatible)

type Tracer added in v0.10.0

type Tracer interface {
	StartSpan(ctx context.Context, operationName string, attrs ...Attribute) (context.Context, Span)
}

Tracer defines the distributed tracing interface

Directories

Path Synopsis
_example

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL