httpcapture

package
v0.22.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 11, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package httpcapture owns the bounded, in-memory HTTP capture handoff for both directions: inbound webhook requests and outbound client attempts. It does no I/O and starts no goroutines. Only framework packages can submit captures.

Index

Constants

View Source
const (
	ResponseBytes = 4 << 10
	// MetadataBytes includes bounded strings, header entries, slice backing
	// arrays and snapshot bookkeeping, as well as the response head and the
	// bounded initial preread buffer during its one growth allocation.
	MetadataBytes = 96 << 10
)
View Source
const OutboundLaneShare = 4

OutboundLaneShare caps one client at a quarter of the process queue and retained bytes, so at least four clients always make progress.

View Source
const OutboundMetadataBytes = 64 << 10

OutboundMetadataBytes is the allowance one outbound exchange reserves for its bounded URL, headers and bookkeeping. Body buffers grow on demand.

Variables

View Source
var OutboundSinks = foundation.NewCollection[*Outbound]("foundry.outbound-capture")

OutboundSinks is the contribution collection httpclient.Module reads. Its element type is internal, so only framework assembly contributes to it.

Functions

func Clip added in v0.19.0

func Clip(text string, limit int) (string, bool)

Clip copies at most limit bytes of text, so a capture never pins a larger backing string, and reports whether text was cut.

Types

type Attempt added in v0.19.0

type Attempt struct {
	// contains filtered or unexported fields
}

Attempt is the producer-side handle of one exchange. Its methods run on the attempt's goroutine; a nil attempt ignores them.

func (*Attempt) Finish added in v0.19.0

func (a *Attempt) Finish(retried bool, result Result, sent, read int64)

Finish runs once in the attempt's deferred cleanup after both bodies are closed, so their taps are quiescent. sent and read are the owned streams' byte counts.

func (*Attempt) Received added in v0.19.0

func (a *Attempt) Received(response *http.Response, expected int64)

Received records the response head and decides how its body is kept. expected is the body length the client will enforce, or -1.

func (*Attempt) RequestTap added in v0.19.0

func (a *Attempt) RequestTap() *Tap

RequestTap decides, before the first byte, how the request body is kept. A Bytes body is kept by reference instead and needs no tap.

func (*Attempt) ResponseTap added in v0.19.0

func (a *Attempt) ResponseTap() *Tap

ResponseTap is the tap Received prepared; nil before a valid response head.

func (*Attempt) Sending added in v0.19.0

func (a *Attempt) Sending(native *http.Request)

Sending records the final native request: URL and headers after defaults and trace propagation.

func (*Attempt) Trace added in v0.19.0

func (a *Attempt) Trace(ctx context.Context) context.Context

Trace attaches a connection observer to the native request's context only, so a consumer callback never inherits it.

type Body

type Body struct {
	Prefix        []byte // raw, at most cap+1; a later owner applies disclosure policy
	End           BodyEnd
	ObservedBytes int64                 // counts source reads, never replay reads
	MeasuredBytes value.Nullable[int64] // known only after observed EOF
	ReadFailure   ReadFailure
}

type BodyCapture added in v0.19.0

type BodyCapture struct {
	// Bytes is an owned tap buffer or, for a request Bytes body within the
	// limit, the body's immutable copy.
	Bytes []byte
	// Declared is the Content-Length; null when unknown.
	Declared value.Nullable[int64]
	// Transferred counts bytes the transport sent or the caller read.
	Transferred int64
	// Complete means the end of the body was observed: the transport
	// consumed the whole request body, or the caller read the whole response.
	Complete bool
	// Offered means Bytes is a request body the client held whole in memory,
	// so it is the whole body even when the transport consumed less.
	Offered bool
	// Over means more than the capture limit passed.
	Over            bool
	Skip            Skip
	ContentType     string
	ContentEncoding string
}

BodyCapture is one direction's raw body facts.

type BodyEnd

type BodyEnd uint8
const (
	NotRead BodyEnd = iota
	EOF
	OverCaptureLimit
	SourceFailure
)

type Cause added in v0.19.0

type Cause struct {
	RequestID attribution.RequestID
	Trace     tracing.TraceID
	// Route is the matched inbound route's declared name.
	Route string
}

Cause links a call to the request that made it.

type ClientLane added in v0.19.0

type ClientLane struct {
	// contains filtered or unexported fields
}

ClientLane is one client's capture policy and admission lane.

func (*ClientLane) Begin added in v0.19.0

func (l *ClientLane) Begin(ctx context.Context, request RequestFacts) *Operation

Begin starts one operation, or returns nil when the client is not captured or a rule excludes the call. A nil URL is never captured.

type Completion

type Completion struct {
	Route                                 string
	Result                                observability.Result
	Status                                value.Nullable[uint16] // actual committed status; absent on precommit abort/hijack
	ResponseHead                          []byte
	ResponseHeaders                       []Field // the handler's header map at commit, before net/http adds transport headers
	ResponseContentType, ResponseEncoding string
	ResponseMetadataTruncated             bool
	ResponseTruncated                     bool
	ResponseBytes                         int64 // accepted by the HTTP writer, not proof of delivery
	Hijacked                              bool
	Aborted                               bool
	WriteFailed                           bool
	Duration                              time.Duration
}

type Config

type Config struct {
	// BodyBytes bounds one captured body.
	BodyBytes     int
	RetainedBytes int64
	QueueSize     int
	// Rate and Burst admit captures per lane.
	Rate  int
	Burst int
	// LaneShare divides QueueSize and RetainedBytes among lanes: one lane may
	// hold at most 1/LaneShare of either (at least one row). Zero means one.
	LaneShare int
}

Config contains operational limits only; it cannot select paths or body mode.

func DefaultConfig

func DefaultConfig() Config

func (Config) Validate

func (c Config) Validate() error

Validate checks every limit against its hard cap. One lane's share of the retained budget must admit at least one whole body with its metadata.

type DropReason

type DropReason uint8
const (
	DropSealed DropReason = iota
	DropRate
	DropCapacity
	DropBytes
	// DropWrite and DropShutdown are reported by the persistence owner after
	// it relinquishes records it could not store.
	DropWrite
	DropShutdown
	DropReasons
)

type Exchange added in v0.19.0

type Exchange struct {
	ID, Operation      [16]byte
	Client             string
	Attempt            uint8
	Retried, Full      bool
	StartedAt          time.Time
	HeadersAfter       value.Nullable[time.Duration]
	Duration           time.Duration
	Cause              Cause
	Method             string
	Scheme, Host       string
	EscapedPath        string
	RawQuery           string
	PathCut, QueryCut  bool
	RequestHeaders     []Field
	RequestHeadersCut  bool
	Request            BodyCapture
	Status             value.Nullable[uint16]
	Protocol           string
	RemoteIP           netip.Addr
	Reused             value.Nullable[bool]
	ResponseHeaders    []Field
	ResponseHeadersCut bool
	Response           BodyCapture
	Result             Result
}

Exchange is one attempt's raw, bounded snapshot. Disclosure happens later in the writer; formatting never prints it.

type Field

type Field struct {
	Name   string   `json:"name"`
	Values []string `json:"values"`
}

Field is one header, query or form name with its retained values. Capture tables store lists of it as JSON.

func Headers added in v0.19.0

func Headers(header http.Header) ([]Field, bool)

Headers copies at most 128 canonical names, 128 values and 16 KiB in sorted order, so the snapshot and its truncation never depend on map order. Both capture directions bound headers with it.

type Lane added in v0.19.0

type Lane[S any] struct {
	// contains filtered or unexported fields
}

Lane is one producer's admission state. Inbound capture has one lane; outbound capture has one per client name, so a busy client spends only its own tokens and share and its losses reach only its own next record.

func (*Lane[S]) Reserve added in v0.19.0

func (l *Lane[S]) Reserve(initial int64) *Reservation[S]

Reserve never waits for capacity, a consumer, or I/O. initial is charged now; Reservation.Grow charges more later.

type Metadata

type Metadata struct {
	RequestID                                                       attribution.RequestID
	ReceivedAt                                                      time.Time
	Prefix                                                          string
	Full                                                            bool
	Method                                                          string
	EscapedPath                                                     string
	RawQuery                                                        string
	ClientIP                                                        netip.Addr
	Headers                                                         []Field
	ContentType                                                     string
	ContentEncoding                                                 string
	DeclaredBytes                                                   value.Nullable[int64]
	PathTruncated, QueryTruncated, HeadersTruncated                 bool
	MethodTruncated, ContentTypeTruncated, ContentEncodingTruncated bool
}

type Operation added in v0.19.0

type Operation struct {
	// contains filtered or unexported fields
}

Operation is one Do, Stream or Download call. Methods on a nil operation are no-ops, so httpclient needs no branches.

func (*Operation) Attempt added in v0.19.0

func (o *Operation) Attempt(number int) *Attempt

Attempt reserves one exchange, or returns nil when the client's lane refuses admission. The exchange starts with the operation's URL and headers, so an attempt that fails before Sending still stores its call.

func (*Operation) NotSent added in v0.19.0

func (o *Operation) NotSent(result Result)

NotSent stores an attempt-0 row: the client's admission was full or the URL's destination was refused before anything reached the wire.

type Outbound added in v0.19.0

type Outbound struct {
	// contains filtered or unexported fields
}

Outbound is the static, I/O-free recorder for client attempts, built once per application. Each client name gets one lane at construction.

func NewOutbound added in v0.19.0

func NewOutbound(c OutboundConfig) (*Outbound, error)

func (*Outbound) Claimed added in v0.19.0

func (o *Outbound) Claimed(name string) bool

Claimed reports whether a client of this name was constructed.

func (*Outbound) Lane added in v0.19.0

func (o *Outbound) Lane(name string) *ClientLane

Lane returns the client's capture lane, or nil when a client-only rule excludes it. Clients of the same name share one lane.

func (*Outbound) Recorder added in v0.19.0

func (o *Outbound) Recorder() *Recorder[Exchange]

Recorder is the queue the outbound store drains.

type OutboundConfig added in v0.19.0

type OutboundConfig struct {
	Limits Config
	// Full names the clients whose bodies are stored exactly.
	Full  []string
	Rules []Rule
	// Inspect decides from Content-Type and Content-Encoding whether redacted
	// mode can store a body; the disclosure owner supplies it.
	Inspect func(contentType, contentEncoding string) Skip
}

OutboundConfig is the validated policy one application's recorder applies.

type ReadFailure

type ReadFailure string
const (
	ReadFailed       ReadFailure = "read_failed"
	ReadTimeout      ReadFailure = "read_timeout"
	ClientDisconnect ReadFailure = "client_disconnect"
	BodyLimit        ReadFailure = "body_limit"
)

type Record

type Record[S any] struct {
	Snapshot      S
	DroppedBefore uint64
	// contains filtered or unexported fields
}

Record is exclusively owned by the draining consumer, including its buffers. DroppedBefore counts captures its lane lost after the lane's previous record.

func (*Record[S]) Format

func (*Record[S]) Format(s fmt.State, _ rune)

func (*Record[S]) Lose added in v0.19.0

func (r *Record[S]) Lose(reason DropReason)

Lose frees a record the consumer could not store. It counts the record and the loss it carried back to its lane, so the lane's next record reports them.

func (*Record[S]) Release

func (r *Record[S]) Release()

Release frees a stored record.

type Recorder

type Recorder[S any] struct {
	// contains filtered or unexported fields
}

Recorder reserves both a slot and bytes before HTTP allocates capture data. Dequeued snapshots remain charged until their consumer releases them. Every producer admits through its own Lane; the recorder holds only the process caps, the drop counters and the queue.

func New

func New[S any](config Config) (*Recorder[S], error)

func (*Recorder[S]) BodyBytes

func (r *Recorder[S]) BodyBytes() int

func (*Recorder[S]) Done

func (r *Recorder[S]) Done() <-chan struct{}

func (*Recorder[S]) Drain

func (r *Recorder[S]) Drain() *Record[S]

Drain transfers one queued record without waiting. Release it only after all persistence/retry work has relinquished its snapshot.

func (*Recorder[S]) Drops

func (r *Recorder[S]) Drops() [DropReasons]uint64

Drops returns owned cumulative counters indexed by DropReason.

func (*Recorder[S]) NewLane added in v0.19.0

func (r *Recorder[S]) NewLane() *Lane[S]

NewLane returns one producer's admission state: its own token bucket, its share of rows and bytes, and the loss count its next record carries.

func (*Recorder[S]) Ready

func (r *Recorder[S]) Ready() <-chan struct{}

func (*Recorder[S]) Seal

func (r *Recorder[S]) Seal()

Seal refuses new reservations. Existing reservations may still grow and complete.

func (*Recorder[S]) Usage

func (r *Recorder[S]) Usage() (int, int64)

Usage includes active reservations, queued records and records held by the consumer.

type RequestFacts added in v0.19.0

type RequestFacts struct {
	Method  string
	URL     *url.URL
	Headers http.Header
	// Data is a Bytes body's private, immutable copy; nil for no body or a stream.
	Data     []byte
	Stream   bool
	Length   int64
	Declined bool
}

RequestFacts are the immutable request pieces httpclient hands over.

type Reservation

type Reservation[S any] struct {
	// contains filtered or unexported fields
}

Reservation is one admitted capture before it is queued. Only its producer uses it.

func (*Reservation[S]) Complete

func (v *Reservation[S]) Complete(snapshot S, retained int64)

Complete takes exclusive ownership of every snapshot buffer and settles the charge to retained, which never exceeds what was admitted. The producer must never mutate or retain those buffers after this call.

func (*Reservation[S]) Format added in v0.19.0

func (*Reservation[S]) Format(s fmt.State, _ rune)

func (*Reservation[S]) Grow added in v0.19.0

func (v *Reservation[S]) Grow(n int64) bool

Grow charges n more bytes against the lane share and the process budget. false means the caller must stop buffering; nothing waits and no record is lost.

type Result added in v0.19.0

type Result uint8

Result is the closed per-attempt outcome; the public enum mirrors it.

const (
	ResultCompleted Result = iota
	ResultConsumerFailed
	ResultResponseBodyFailed
	ResultResponseTooLarge
	ResultRequestBodyFailed
	ResultTransportFailed
	ResultTimedOut
	ResultCanceled
	ResultDestinationDenied
	ResultOverloaded
)

type Rule added in v0.19.0

type Rule struct {
	Client string
	// Host is lowercase; "*.x" matches any subdomain of x, not x itself.
	Host   string
	Method string
	// PathPrefix is a normalized decoded prefix without a trailing slash; it
	// matches segment by segment.
	PathPrefix string
}

Rule is one validated exclusion. Empty fields match anything.

type Skip added in v0.19.0

type Skip uint8

Skip says why a body's bytes were not kept.

const (
	Kept Skip = iota
	// SkipNotSent: the operation never reached the wire (attempt 0).
	SkipNotSent
	// SkipMedia: redacted mode cannot inspect this media type.
	SkipMedia
	// SkipEncoded: the body is content-encoded in redacted mode.
	SkipEncoded
	// SkipDeclined: Request.WithoutBodyCapture.
	SkipDeclined
	// SkipMemory: the retained budget refused the body buffer.
	SkipMemory
)

type Snapshot

type Snapshot struct {
	Metadata   Metadata
	Body       Body
	Completion Completion
}

Snapshot is one inbound webhook request. Its loss count travels on the Record that carries it.

func (Snapshot) Format

func (Snapshot) Format(s fmt.State, _ rune)

Capture data must never be disclosed through ordinary diagnostic formatting.

type Tap added in v0.19.0

type Tap struct {
	// contains filtered or unexported fields
}

Tap observes one owned stream. The stream calls Observe and End under its read lock, so a tap needs no lock of its own; Finish reads it after Close.

func (*Tap) End added in v0.19.0

func (t *Tap) End(end TapEnd)

End records how the stream ended; only the first end counts.

func (*Tap) Observe added in v0.19.0

func (t *Tap) Observe(p []byte)

Observe keeps up to the capture limit of the bytes a read returned. The first buffer is the declared length when known, else 32 KiB, with one growth to the limit; a refused charge turns the tap count-only.

type TapEnd added in v0.19.0

type TapEnd uint8

TapEnd is how an observed stream ended.

const (
	// TapOpen: closed before its end, or never read.
	TapOpen TapEnd = iota
	TapEOF
	// TapBound: the stream exceeded its byte bound.
	TapBound
	// TapFailed: a read failed.
	TapFailed
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL