client

go-client

This repository contains a generic HTTP client which can be adapted to provide:

• General HTTP methods for GET and POST of data
• Ability to send and receive JSON, plaintext and XML data
• Ability to send files and data of type multipart/form-data
• Ability to send data of type application/x-www-form-urlencoded
• Debugging capabilities to see the request and response data
• Streaming text and JSON events
• OpenTelemetry tracing for distributed observability

API Documentation: https://pkg.go.dev/github.com/mutablelogic/go-client

There are also some example clients which use this library:

• Bitwarden API Client
• Home Assistant API Client
• IPify Client

There are also utility packages for working with multipart file uploads, transport middleware, and OpenTelemetry:

• OpenTelemetry Package
• Transport Middleware Package
• Multipart Package

Compatibility with go version 1.25 and above.

Basic Usage

The following example shows how to decode a response from a GET request to a JSON endpoint:

package main

import (
    "fmt"
    "log"

    client "github.com/mutablelogic/go-client"
)

func main() {
    // Create a new client
    c, err := client.New(client.OptEndpoint("https://api.example.com/api/v1"))
    if err != nil {
        log.Fatal(err)
    }

    // Send a GET request, populating a struct with the response
    var response struct {
        Message string `json:"message"`
    }
    if err := c.Do(nil, &response, client.OptPath("test")); err != nil {
        log.Fatal(err)
    }

    // Print the response
    fmt.Println(response.Message)
}

Various options can be passed to the client New method to control its behaviour:

• OptEndpoint(value string) sets the endpoint for all requests
• OptTimeout(value time.Duration) sets the timeout on any request, which defaults to 30 seconds. Timeouts can be ignored on a request-by-request basis using the OptNoTimeout option (see below).
• OptUserAgent(value string) sets the user agent string on each API request.
• OptTrace(w io.Writer, verbose bool) — Deprecated. Use OptTransport with transport.NewLogging instead. See the Transport Middleware section below for details.
• OptStrict() turns on strict content type checking on anything returned from the API.
• OptRateLimit(value float32) sets the limit on number of requests per second and the API will sleep to regulate the rate limit when exceeded.
• OptReqToken(value Token) sets a request token for all client requests. This can be overridden by the client for individual requests using OptToken (see below).
• OptSkipVerify() skips TLS certificate domain verification.
• OptHeader(key, value string) appends a custom header to each request.
• OptParent(v any) attaches arbitrary context to the client. The stored value is accessible via the Parent field and is used by wrapper types that embed a *Client to store their own state.
• OptTransport(fn func(http.RoundTripper) http.RoundTripper) inserts a transport middleware that wraps every request made by this client. Multiple calls stack in order so the first call becomes the outermost layer. Use this to plug in any pkg/transport middleware.
• OptTracer(tracer trace.Tracer) sets an OpenTelemetry tracer for distributed tracing. Span names default to "METHOD /path" format. See the OpenTelemetry section below for more details.

Redirect Handling

The client automatically follows HTTP redirects (3xx responses) for GET and HEAD requests, up to a maximum of 10 redirects. Unlike the default Go HTTP client behavior:

• The HTTP method is preserved (HEAD stays HEAD, GET stays GET)
• Request headers are preserved across redirects
• For security, Authorization and Cookie headers are stripped when redirecting to a different host

This behavior ensures that redirects work correctly for APIs that use CDNs or load balancers with temporary redirects.

Usage with a payload

The first argument to the Do method is the payload to send to the server, when set. You can create a payload using the following methods:

• client.NewRequest() returns a new empty payload which defaults to GET.
• client.NewRequestEx(method, accept string) returns a new empty payload with an explicit HTTP method and accepted response content type.
• client.NewJSONRequest(payload any) (Payload, error) returns a new POST request with a JSON payload that accepts any response content type.
• client.NewJSONRequestEx(method string, payload any, accept string) (Payload, error) is the extended form that also sets the HTTP method and accepted response content type.
• client.NewMultipartRequest(payload any, accept string) (Payload, error) returns a new request with a Multipart Form data payload which defaults to POST.
• client.NewStreamingMultipartRequest(payload any, accept string) (Payload, error) returns a new request with a Multipart Form data payload that streams data rather than buffering in memory. Useful for large file uploads. The returned payload implements io.Closer and is automatically closed by the HTTP client after the request completes.
• client.NewFormRequest(payload any, accept string) (Payload, error) returns a new request with a Form data payload which defaults to POST.

For example,

package main

import (
    "fmt"
    "log"

    client "github.com/mutablelogic/go-client"
)

func main() {
    // Create a new client
    c, err := client.New(client.OptEndpoint("https://api.example.com/api/v1"))
    if err != nil {
        log.Fatal(err)
    }

    // Send a POST request with JSON payload
    var request struct {
        Prompt string `json:"prompt"`
    }
    var response struct {
        Reply string `json:"reply"`
    }
    request.Prompt = "Hello, world!"
    payload, err := client.NewJSONRequest(request)
    if err != nil {
        log.Fatal(err)
    }
    if err := c.Do(payload, &response, client.OptPath("test")); err != nil {
        log.Fatal(err)
    }

    // Print the response
    fmt.Println(response.Reply)
}

You can also implement your own payload by implementing the Payload interface:

type Payload interface {
  io.Reader

  // The method to use to send the payload
  Method() string

  // The content type of the payload
  Type() string

  // The content type which is accepted as a response, or empty string if any
  Accept() string
}

Request options

The signature of the Do method is as follows:

type Client interface {
    // Perform request and wait for response
    Do(in Payload, out any, opts ...RequestOpt) error

    // Perform request and wait for response, with context for cancellation
    DoWithContext(ctx context.Context, in Payload, out any, opts ...RequestOpt) error
}

If you pass a context to the DoWithContext method, then the request can be cancelled using the context in addition to the timeout. Various options can be passed to modify each individual request when using the Do method:

• OptReqEndpoint(value string) sets the endpoint for the request
• OptPath(value ...any) appends path elements onto a request endpoint
• OptToken(value Token) adds an authorization header (overrides the client OptReqToken option)
• OptQuery(value url.Values) sets the query parameters to a request
• OptReqHeader(name, value string) sets a custom header to the request
• OptNoTimeout() disables the timeout on the request, which is useful for long running requests
• OptReqTransport(fn func(http.RoundTripper) http.RoundTripper) inserts a transport middleware for this single request only. Multiple calls stack in order; the first becomes the outermost. The middleware is applied on a per-request copy of the client and does not affect other requests.
• OptTextStreamCallback(fn TextStreamCallback) allows you to set a callback function to process a streaming text response of type text/event-stream, where TextStreamCallback is func(TextStreamEvent) error. See below for more details.
• OptJsonStreamCallback(fn JsonStreamCallback) allows you to set a callback for JSON streaming responses, where JsonStreamCallback is func(any) error. See below for more details.

Authentication

The authentication token can be set as follows:

package main

import (
    "log"
    "os"

    client "github.com/mutablelogic/go-client"
)

func main() {
    // Create a new client
    c, err := client.New(
        client.OptEndpoint("https://api.example.com/api/v1"),
        client.OptReqToken(client.Token{
            Scheme: "Bearer",
            Value:  os.Getenv("API_TOKEN"),
        }),
    )
    if err != nil {
        log.Fatal(err)
    }

    // Use the client...
    _ = c
}

You can also set the token on a per-request basis using the OptToken option in call to the Do method.

Form submission

You can create a payload with form data:

• client.NewFormRequest(payload any, accept string) (Payload, error) returns a new request with a Form data payload which defaults to POST.
• client.NewMultipartRequest(payload any, accept string) (Payload, error) returns a new request with a Multipart Form data payload which defaults to POST. This is useful for file uploads.
• client.NewStreamingMultipartRequest(payload any, accept string) (Payload, error) returns a new request that streams the multipart data rather than buffering in memory. This is recommended for large file uploads to avoid high memory usage. The returned payload implements io.Closer and is automatically closed by the HTTP client after the request completes.

The payload should be a struct where the fields are converted to form tuples. File uploads require a field of type multipart.File. For example,

package main

import (
    "log"
    "strings"

    client "github.com/mutablelogic/go-client"
    multipart "github.com/mutablelogic/go-client/pkg/multipart"
)

type FileUpload struct {
    File multipart.File `json:"file"`
}

func main() {
    // Create a new client
    c, err := client.New(client.OptEndpoint("https://api.example.com/api/v1"))
    if err != nil {
        log.Fatal(err)
    }

    // Create a file upload request
    request := FileUpload{
        File: multipart.File{
            Path: "helloworld.txt",
            Body: strings.NewReader("Hello, world!"),
        },
    }

    // Upload a file
    var response any
    payload, err := client.NewMultipartRequest(request, "*/*")
    if err != nil {
        log.Fatal(err)
    }
    if err := c.Do(payload, &response, client.OptPath("upload")); err != nil {
        log.Fatal(err)
    }
}

Unmarshalling responses

You can implement your own unmarshalling of responses by implementing the Unmarshaler interface:

type Unmarshaler interface {
  Unmarshal(header http.Header, r io.Reader) error
}

The first argument to the Unmarshal method is the HTTP header of the response, and the second argument is the body of the response. You can return one of the following error values from Unmarshal to indicate how the client should handle the response:

• nil to indicate successful unmarshalling.
• httpresponse.ErrNotImplemented (from github.com/mutablelogic/go-server/pkg/httpresponse) to fall back to the default unmarshaling behaviour. In this case, the body will be unmarshaled based on the Content-Type header:
  • application/json → any JSON-decodable type
  • application/xml or text/xml → any XML-decodable type
  • text/plain → *string (value set to the body text), *[]byte (raw bytes), or io.Writer (body copied into it)
• Any other error to indicate a failure in unmarshaling.

Text Streaming Responses

The client implements a streaming text event callback which can be used to process a stream of text events, as per the Mozilla specification.

In order to process streamed events, pass the OptTextStreamCallback() option to the request with a callback function, which should have the following signature:

func Callback(event client.TextStreamEvent) error {
    // Finish processing successfully
    if event.Event == "close" {
        return io.EOF
    }

    // Decode the data into a JSON object
    var data map[string]any
    if err := event.Json(data); err != nil {
        return err
    }

    // Return success - continue streaming
    return nil
}

The TextStreamEvent object has the following fields:

• Id string — the event ID (id: field)
• Event string — the event type (event: field; defaults to "message")
• Data string — the event data (data: fields joined with \n)
• Retry time.Duration — the server-requested reconnect delay (retry: field)

If you return an error of type io.EOF from the callback, then the stream will be closed. Similarly, if you return any other error the stream will be closed and the error returned.

Usually, you would pair this option with OptNoTimeout to prevent the request from timing out.

SSE Reconnect

For reconnect support, use NewTextStream() and Decode() directly rather than OptTextStreamCallback. After Decode returns, the decoder holds the last event ID and server-requested retry delay:

stream := client.NewTextStream()
if err := stream.Decode(r, callback); err != nil {
    // reconnect: set Last-Event-ID header and wait
    req.Header.Set("Last-Event-ID", stream.LastEventID())
    time.Sleep(stream.RetryDuration())
}

When the Accept type of the payload is text/event-stream or application/x-ndjson, the client automatically adds Cache-Control: no-cache and X-Accel-Buffering: no request headers to prevent intermediate proxies (including Nginx) from buffering the stream.

JSON Streaming Responses

The client decodes JSON streaming responses by passing a callback function to the OptJsonStreamCallback() option. The callback with signature func(any) error is called for each JSON object in the stream, where the argument is the same type as the object in the request.

You can return an error from the callback to stop the stream and return the error, or return io.EOF to stop the stream immediately and return success.

Transport Middleware

The pkg/transport package provides composable http.RoundTripper middleware. All middleware follows the same constructor pattern New*(... , parent http.RoundTripper) and falls back to http.DefaultTransport when parent is nil. Middleware can be composed using OptTransport (client-wide) or OptReqTransport (per-request).

Logging Transport

transport.NewLogging logs every request and response to an io.Writer. When verbose is true the request and response bodies are also printed; text/event-stream bodies are printed line-by-line as events arrive rather than buffered:

import (
    client "github.com/mutablelogic/go-client"
    "github.com/mutablelogic/go-client/pkg/transport"
    "os"
)

c, err := client.New(
    client.OptEndpoint("https://api.example.com"),
    client.OptTransport(func(next http.RoundTripper) http.RoundTripper {
        return transport.NewLogging(os.Stderr, next, true)
    }),
)

The convenience client option OptTrace(w io.Writer, verbose bool) (deprecated) is equivalent and internally calls transport.NewLogging.

Recorder Transport

transport.NewRecorder captures the HTTP status code and response headers of the most recent response. It is safe for concurrent use:

var rec *transport.Recorder

c, err := client.New(
    client.OptEndpoint("https://api.example.com"),
    client.OptTransport(func(next http.RoundTripper) http.RoundTripper {
        rec = transport.NewRecorder(next)
        return rec
    }),
)

// After a request:
fmt.Println(rec.StatusCode())     // e.g. 200
fmt.Println(rec.Header())         // cloned http.Header map
rec.Reset()                       // clear recorded values

OTel Transport

transport.NewTransport wraps an http.RoundTripper so that every hop produces an OpenTelemetry client span. See the OpenTelemetry section below for details.

OpenTelemetry

The pkg/otel package provides OpenTelemetry tracing utilities for both HTTP clients and servers.

Creating a Tracer Provider

Use otel.NewProvider to create a tracer provider that exports spans to an OTLP endpoint:

package main

import (
    "context"
    "log"
    "net/http"

    client "github.com/mutablelogic/go-client"
    "github.com/mutablelogic/go-client/pkg/otel"
    "github.com/mutablelogic/go-client/pkg/transport"
)

func main() {
    // Create a provider with an OTLP endpoint
    // Supports http://, https://, grpc://, and grpcs:// schemes
    provider, err := otel.NewProvider(
        "https://otel-collector.example.com:4318",  // OTLP endpoint
        "api-key=your-api-key",                     // Optional headers (comma-separated key=value pairs)
        "my-service",                               // Service name
        otel.Attr{Key: "environment", Value: "production"},  // Optional attributes
    )
    if err != nil {
        log.Fatal(err)
    }
    defer provider.Shutdown(context.Background())

    // Get a tracer from the provider
    tracer := provider.Tracer("my-service")

    // Use the tracer with go-client via OptTransport
    c, err := client.New(
        client.OptEndpoint("https://api.example.com"),
        client.OptTransport(func(next http.RoundTripper) http.RoundTripper {
            return transport.NewTransport(tracer, next)
        }),
    )
    if err != nil {
        log.Fatal(err)
    }

    // Use the client...
    _ = c
}

HTTP Client Tracing

OptTracer and transport.NewTransport both wrap the client's HTTP transport so that every HTTP call produces a client span — including OAuth token refresh calls and each redirect hop. Span names default to "METHOD /path" format. Attributes captured per span:

• HTTP method, URL, and host
• Request and response body sizes
• HTTP status codes
• Error recording for failed requests

Prefer composing via OptTransport so the transport layer stays explicit:

import (
    client "github.com/mutablelogic/go-client"
    "github.com/mutablelogic/go-client/pkg/transport"
)

c, err := client.New(
    client.OptEndpoint("https://api.example.com"),
    client.OptTransport(func(next http.RoundTripper) http.RoundTripper {
        return transport.NewTransport(tracer, next)
    }),
)

You can also call transport.NewTransport directly on any *http.Client:

httpClient.Transport = transport.NewTransport(tracer, httpClient.Transport)

HTTP Server Middleware

Use otel.HTTPHandler or otel.HTTPHandlerFunc to add tracing to your HTTP server:

package main

import (
    "context"
    "log"
    "net/http"

    "github.com/mutablelogic/go-client/pkg/otel"
)

func main() {
    // Create provider (see "Creating a Tracer Provider" above)
    provider, err := otel.NewProvider("https://otel-collector.example.com:4318", "", "my-server")
    if err != nil {
        log.Fatal(err)
    }
    defer provider.Shutdown(context.Background())

    tracer := provider.Tracer("my-server")

    // Wrap your handler with the middleware
    handler := otel.HTTPHandler(tracer)(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("Hello, World!"))
    }))

    // Or use HTTPHandlerFunc directly
    handlerFunc := otel.HTTPHandlerFunc(tracer)(func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("Hello, World!"))
    })

    http.Handle("/", handler)
    http.Handle("/func", handlerFunc)
    http.ListenAndServe(":8080", nil)
}

The middleware automatically:

• Extracts trace context from incoming request headers (W3C Trace Context)
• Creates server spans with HTTP method, URL, and host attributes
• Captures response status codes
• Marks spans as errors for 4xx and 5xx responses

Custom Spans

Use otel.StartSpan to create custom spans in your application:

ctx, endSpan := otel.StartSpan(tracer, ctx, "MyOperation",
    attribute.String("key", "value"),
)
// Use a closure to capture the final value of err when the function returns.
// defer endSpan(err) would capture err's value NOW (likely nil), not at return time.
defer func() { endSpan(err) }()

// Your code here...

License

This project is licensed under the Apache License, Version 2.0. See the LICENSE file for details, and the NOTICE file for copyright attribution.

What You Can Do

Under the Apache-2.0 license, you are free to:

• Use — Use the software for any purpose, including commercial applications
• Modify — Make changes to the source code
• Distribute — Share the original or modified software
• Sublicense — Grant rights to others under different terms
• Patent Use — Use any patents held by contributors that cover this code

Requirements when redistributing:

• Include a copy of the Apache-2.0 license
• State any significant changes you made
• Preserve copyright, patent, trademark, and attribution notices