httptransport

package module
v1.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 30, 2026 License: MIT Imports: 12 Imported by: 0

README

go.loglayer.dev/transports/http

Go Reference

Generic batched HTTP POST transport for LogLayer. Pluggable encoder, async worker, configurable batching and retries. Default Client refuses cross-host redirects so an Authorization header (or any other credential you set on the request) can't leak to a redirected host. Use it directly to talk to any log-ingestion API; the Datadog transport is built on top of it.

Install

go get go.loglayer.dev/transports/http

Documentation

Full reference and examples: https://go.loglayer.dev/transports/http

Main library: https://go.loglayer.dev

Documentation

Overview

Package httptransport sends log entries to an HTTP endpoint as JSON in configurable batches. The package directory is `transports/http` to mirror other transports; the package name is `httptransport` to avoid colliding with `net/http`.

Mental model: log calls enqueue entries into a buffered channel; a background worker drains the channel into batches and POSTs them. Use Close to drain pending entries on shutdown.

See https://go.loglayer.dev for usage guides and the full API reference.

Index

Examples

Constants

This section is empty.

Variables

View Source
var ErrBufferFull = errors.New("loglayer/transports/http: buffer full, entry dropped")

ErrBufferFull is reported via OnError when SendToLogger drops an entry because the internal buffer is full.

View Source
var ErrClosed = errors.New("loglayer/transports/http: transport closed, entry dropped")

ErrClosed is reported via OnError when SendToLogger is called after Close.

View Source
var ErrURLRequired = errors.New("loglayer/transports/http: Config.URL is required")

ErrURLRequired is returned by Build (and panicked by New) when Config.URL is empty.

Functions

This section is empty.

Types

type Config

type Config struct {
	transport.BaseConfig

	// URL is the endpoint logs are POSTed to. Required.
	//
	// The transport accepts any scheme; an http:// URL is allowed because
	// fluentd-over-loopback, Loki on a private network, and TLS-terminating
	// proxies are legitimate setups. Wrappers that ship a known credential
	// (transports/datadog ships DD-API-KEY) layer their own https-by-default
	// enforcement on top.
	URL string

	// Method is the HTTP method. Defaults to "POST".
	Method string

	// Headers are added to every request. Content-Type is set automatically
	// from the Encoder unless overridden here.
	//
	// Headers are transmitted in plaintext when URL is http://. If you
	// put credentials here (Authorization, X-API-Key, etc.), use an
	// https:// URL or a TLS-terminating proxy. The transport doesn't
	// enforce a scheme so internal endpoints (fluentd over the loopback,
	// Loki on a private network) keep working, but the contract is
	// caller-owned.
	//
	// Header injection (CRLF in a header value) is rejected by Go's
	// net/http at request-send time with an "invalid header field
	// value" error, which surfaces via OnError. The transport doesn't
	// pre-validate at Build time; the stdlib failure is enough.
	Headers map[string]string

	// Encoder serializes one or more entries into the request body. Defaults
	// to JSONArrayEncoder.
	Encoder Encoder

	// Client is the HTTP client used to send requests. Defaults to a client
	// with a 30-second timeout.
	Client *http.Client

	// BatchSize is the maximum number of entries per request. The worker
	// flushes whenever this is reached or BatchInterval elapses, whichever
	// comes first. Defaults to 100.
	BatchSize int

	// BatchInterval is the maximum wait before flushing a non-empty batch.
	// Defaults to 5 seconds.
	BatchInterval time.Duration

	// BufferSize is the size of the internal channel buffering entries before
	// the worker reads them. When full, new entries are dropped (oldest-out
	// is not implemented; the dispatch path stays non-blocking). Defaults to
	// 1024.
	BufferSize int

	// OnError is called when a batch fails to encode or send. The default
	// writes a one-line error to os.Stderr. Use this to plumb send errors
	// into a separate logger or metrics counter.
	OnError func(err error, entries []Entry)
}

Config holds HTTP transport configuration.

type Encoder

type Encoder interface {
	Encode(entries []Entry) (body []byte, contentType string, err error)
}

Encoder serializes a batch of entries into the HTTP request body. The returned content type is set as the Content-Type header (overridable via Config.Headers).

var JSONArrayEncoder Encoder = EncoderFunc(jsonArrayEncode)

JSONArrayEncoder serializes the batch as a JSON array of one object per entry: {level, time, msg, ...data, metadata?}. Map metadata merges at the root; non-map metadata lands under the "metadata" key.

type EncoderFunc

type EncoderFunc func(entries []Entry) ([]byte, string, error)

EncoderFunc adapts an ordinary function to the Encoder interface.

func (EncoderFunc) Encode

func (f EncoderFunc) Encode(entries []Entry) ([]byte, string, error)

Encode implements Encoder.

type Entry

type Entry struct {
	Level    loglayer.LogLevel
	Time     time.Time
	Messages []any
	// Data holds the assembled persistent fields and error map (may be nil).
	Data map[string]any
	// Metadata is the raw value the caller passed to WithMetadata. May be a
	// map, a struct, a scalar, or nil. Encoders decide how to render it.
	Metadata any
	// Groups mirrors [loglayer.TransportParams.Groups].
	Groups []string
	// Schema mirrors [loglayer.TransportParams.Schema].
	Schema loglayer.Schema
}

Entry is the canonical log shape passed to the encoder. Constructed once per log call from loglayer.TransportParams.

type HTTPError

type HTTPError struct {
	StatusCode int
}

HTTPError is reported via OnError when the server responds with a status >= 400. The original entries are still passed to OnError so callers can implement retry/dead-letter behavior.

func (*HTTPError) Error

func (e *HTTPError) Error() string

type Transport

type Transport struct {
	transport.BaseTransport
	// contains filtered or unexported fields
}

Transport implements loglayer.Transport with batched HTTP delivery.

func Build

func Build(cfg Config) (*Transport, error)

Build constructs an HTTP Transport like New but returns ErrURLRequired instead of panicking when cfg.URL is empty. Use this when the URL is loaded at runtime (e.g. from an environment variable) and you want to handle the missing-config case explicitly.

func New

func New(cfg Config) *Transport

New constructs an HTTP Transport and starts its background worker. Panics if cfg.URL is empty. Use Build for an error-returning variant.

Example

New POSTs JSON batches to URL. The worker goroutine starts at construction time; call Close on shutdown to flush pending entries and stop the worker.

t := httptransport.New(httptransport.Config{
	URL: "https://logs.example.com/ingest",
})
defer t.Close()

log := loglayer.New(loglayer.Config{
	Transport:        t,
	DisableFatalExit: true,
})
log.Info("served")

func (*Transport) Close

func (t *Transport) Close() error

Close drains the queue, flushes any pending entries, and stops the worker. Safe to call multiple times. After Close, SendToLogger reports ErrClosed.

closeMu.Lock waits for in-flight SendToLogger calls (each holding an RLock) to complete; once they have, no new SendToLogger can land an entry in the queue (the closed flag is set under the same lock), so the worker's drainAndFlush sees a stable, finite queue.

func (*Transport) GetLoggerInstance

func (t *Transport) GetLoggerInstance() any

GetLoggerInstance returns nil; the HTTP transport has no underlying logger.

func (*Transport) SendToLogger

func (t *Transport) SendToLogger(params loglayer.TransportParams)

SendToLogger enqueues the entry. Reports loss via OnError(ErrClosed, ...) when the transport has been closed and OnError(ErrBufferFull, ...) when the buffer is saturated. The closeMu RLock pairs with Close's Lock so an entry either lands in the queue (and is guaranteed to be drained by the worker during Close) or is reported as ErrClosed; an in-flight SendToLogger can't silently land in the queue after the worker has exited.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL