http

module
v1.15.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 11, 2026 License: MIT

README

gorib/http

HTTP for gorib services: a chi-backed server with typed handlers, an HTTP client, and an OpenAPI generator that keeps the specification in step with the code.

Package What's in it
server mux and route groups, typed handlers (Bind), middleware chain, error rendering, graceful shutdown, route introspection (Routes() []RouteMeta)
client request building, response decoding, tracing
model content types, transport errors, session
pbjson JSON marshalling, with protobuf messages and enums in the shape the OpenAPI document describes
openapi spec derivation from registered routes, and reconciliation with a hand-edited file
openapi/openapitest the document as a golden file in a test

For the service-side usage — wiring the command, registering routes, choosing middleware — see the waffle docs, docs/tools/http.md and docs/tools/http-handler.md in gitlab.com/gorib/waffle. This file documents the packages themselves: the shapes server is built from, and the openapi generator.

server

One handler shape, and everything else composes around it:

type Handler func(src Source) (Renderer, error)
type Layer   = func(next Handler) Handler

Source is the request as an interface — Path, Query, Header, Cookie, Form, Body, plus the Lookup/LookupAll pair that satisfies scan.Source, which is what lets a whisk-generated binder read an HTTP request the same way it reads an AMQP delivery. Renderer is anything that can write itself to a http.ResponseWriter and report its status.

A handler that wants types rather than a request uses Bind. It runs the whisk-generated Parse, then Validate if the type has one, and the first error short-circuits before the function is called. A handler that returns a nil response and a nil error renders 204.

mux.Post("/login", server.Bind(func(ctx context.Context, req *model.LoginRequest) (*server.Response[Out], error) {
    return server.OK(&Out{...}), nil
}))

Response carries status, headers and a body with a content type: OK, Created, Accepted, NoContent, or JSON/XML/Text/HTML when the type is not the default, and StreamWith for a body written incrementally. File and Redirect are renderers of their own.

A JSON body is encoded as encoding/json would, except that a nil slice or map goes out as [] or {} (a nil []byte as ""), never null, and that every protobuf message and enum in it, at any depth, goes through pbjson: lowerCamel names, unset fields written out (null where the field tracks presence, an unset oneof left out), enums by name, 64-bit integers as numbers rather than protojson's quoted strings, well-known types in their protojson shape (the 64-bit wrappers are plain numbers). An enum number with no name fails the response with a 500 rather than leaking the number.

Source.Body reads the whole body and Source.Peek only its head, leaving the request to carry all of it.

Path matching

The route is chosen by the same string every decision below it is made on: the decoded r.URL.Path. chi on its own matches r.URL.RawPath whenever the client sent a path whose escaping is not canonical, while everything else reaches for URL.Path — a guard comparing a prefix, Health, the span and log attributes, the handler. Where that split cuts is a guarded prefix beside something wider:

mux.Get("/customers/{customerId}/*", auth, server.Handle(serveFile))
mux.Get("/*", server.Handle(serveFile))

GET /%63ustomers/5/secret.pdf misses the guarded route and lands in /*, while the handler goes on to open /customers/5/secret.pdf. So the mux pins the routing path to URL.Path before chi matches, and the service has nothing to remember. The window is narrower than "any percent-encoding" — net/url sets RawPath only for an escaping it would not have produced itself, so Cyrillic and spaces never diverge and the bug is invisible in ordinary traffic and in tests written with ordinary URLs.

Two consequences, both pinned in mux_test.go:

  • a path parameter is now always the decoded value. {code} is msk for /facilities/%6dsk/stat, where it used to be %6dsk — while the same route already gave МСК for /facilities/%D0%9C%D0%A1%D0%9A/stat. Which of the two a handler got depended on how the rest of the path happened to be encoded;
  • %2F inside a segment is a separator like any other. /facilities/a%2Fb/stat reads as /facilities/a/b/stat, one segment more than the route declares, and answers 404 where it used to match with {code} = a%2Fb. This is the price, and it is not recoverable by decoding less: a routing string that keeps %2F is a second string again, and the handler reading URL.Path would still see two segments. A value that can carry a slash belongs in a query parameter or in the body, not in a path segment.

Dot segments and doubled slashes are the same split by another road: chi matches //customers/5/secret.pdf against /*, and http.Dir cleans it back to /customers/5/secret.pdf. So a path that is not clean is never routed at all — it answers 308 to its cleaned form, query kept, as the stdlib ServeMux does.

The raw string is taken out of the routing decision, not destroyed: RawPath and EscapedPath() still carry what came off the wire, which is what a log or an audit record wants. So the invariant has an edge the framework cannot close — a handler that decides on EscapedPath() rather than Path is back to two strings. Clearing RawPath instead would close that edge by rewriting EscapedPath(), String() and RequestURI() for every reader, honest ones included; the edge is the cheaper side of that trade. Decide on URL.Path.

Server defaults

NewServer starts from these and applies With* options on top. A zero means no timeout, which is why three of them are zero on purpose.

Setting Default Why not something else
ReadHeaderTimeout 10s Slowloris protection. A full ReadTimeout would also cut off a slow upload.
ReadTimeout 0 Body size is MaxBody's job; body time is left unbounded so a large upload can finish.
WriteTimeout 0 Streaming, SSE and long responses have to stay possible.
IdleTimeout 120s Stale keep-alives would otherwise accumulate.
MaxHeaderBytes 32 KB Room for a JWT, several cookies and tracing baggage. The stdlib's 1 MiB is DoS-friendly.
ShutdownTimeout 0 The orchestrator's grace period owns this. A hung shutdown is better observed than silently truncated.
The standard chain

The middleware package ships seven global layers, plus NewMaxBody for a single route. The canonical order of the seven, outer to inner:

Layer What
Compress gzips the response when the client asked for it and the content type is worth compressing
Health answers the health path itself, so a probe never reaches session, tracing or the handler
Session request identity into session.Session — correlation id, user agent, source ip, the first 4 KiB of the body — plus the incoming trace context onto the ctx, and echoes the correlation id back on the response
Tracing SERVER span, provider into ctx, http.server.request.duration and http.server.active_requests
Log logger into ctx, then the request and the response at trace, raised to error only when the status is 5xx or nothing rendered at all
Error maps a returned error to a status (server.StatusError; Coder sets the body's code, Detailer adds details) and a rendered body, trying the renderers given to NewErrorRenderer before the default; the error itself rides out on the response for Log and Tracing, and a 500's own text never reaches the client
Recover failure.Catch, so a handler panic becomes an error

The package only ships the layers; assembling them in this order is the caller's job, and the order is the whole content of that job.

The order is load-bearing at three points. Error sits inside Tracing and Log, so both see the status that was actually rendered rather than the bare error — put it outside and every failed request is recorded with the handler's zero status. Recover is innermost, so a panic becomes an error while the span is still open; anything further out would end the span unrecorded. And Tracing sits after Session, so it can replace the context's traceparent with its own span's — otherwise an outgoing call would advertise the upstream caller as its parent instead of this request.

Per-route layers from mux.Get(path, layer, ...) compose inside Recover.

Global layers run before routing, so no per-route limit can bound what they read. That is why they only ever Peek the body, at most 4 KiB, and why MaxBody is per-route and still holds.

client

client.Client builds requests, decodes responses and traces both. The provider comes from the context (hilton.ProviderFromContext), so a call inherits the trace of the request that issued it; it emits a CLIENT span and http.client.request.duration, and injects the trace context into the outgoing headers. It also forwards session.Session's correlation id and the context's traceparent — which is the other end of the ordering rule above: the server's Tracing layer put its own span there so this call points at the current request, not at whoever called it.

For what the metrics look like across all of gorib, see docs/tools/hilton.md in waffle.

NewTransport() is a copy of the default transport, one per neighbour, and WithTransport(t) puts it under the client. A waffle service takes it from the container, so a component test can stand in for that neighbour under the client — see docs/harness.md in specrig-go's waffle adapter.

openapi

The package answers one question: how does a specification stay true to the code without taking the prose away from the humans who wrote it.

The answer is a split of ownership, and the rule that makes it work is stricter than "structure versus prose": anything the generator writes, it has to be able to rewrite. A value it produces once and can never correct goes stale in silence, and the golden check below then certifies a document that lies.

So the line falls here:

Owned by Keys On a later run
the code paths, operations, operationId, request body $ref, schemas, field types, required, nullability, and everything derived from a validate tag — format, enum, pattern, the length/range/item bounds, x-constraints rewritten from the model
the human summary, description, example, extra responses, anything the generator does not produce never touched

Two consequences worth knowing.

A rule OpenAPI has no keyword for lands in x-constraints, not in the description. It used to be appended to the prose, where nothing could correct it — so a rule that changed left the old sentence behind forever. Reconciliation moves such a sentence out of an existing description and says so in its warnings; the rest of the description is left exactly as written.

A derived keyword the generator has no opinion about is left exactly as found, and nothing is said about it. Refining format by hand on a field with no validate tag is legitimate, and by the rule above such a value is the human's — so it is not the generator's to delete, nor to complain about on every run for the rest of the project's life.

The cost is the other reading of the same file: a validate rule deleted from the code leaves its keyword behind, and a run cannot tell that from a refinement. The place to catch it is the diff where the tag was deleted, not a warning that would have to fire for both.

Entry points
func Build(routes []server.RouteMeta) (*Spec, []string)
func Emit(routes []server.RouteMeta, path, title string) ([]string, error)
func Reconcile(routes []server.RouteMeta, path, title string) ([]string, error)
func Components(routes []server.RouteMeta) map[string]*Schema

Build derives the spec model and returns warnings. Emit dumps a fresh document whole (OpenAPI 3.1.0, info.version 1.0.0). Reconcile is what callers normally use: it merges into the file at path, and when that file does not exist it reconciles into an empty skeleton rather than delegating to Emit. The difference matters — everything in a file that does not exist yet was put there by the generator, so it all has to arrive through the same branch a later addition does, carrying the TODO(openapi) that says nobody has written the prose. Components is the schemas alone, for callers that only need the type catalogue.

Both file-writing functions return warnings alongside the error. Reconcile writes the merged document before returning a structural error, so a failing run still leaves something to inspect.

Reconcile is idempotent: a run against an up-to-date document leaves the bytes untouched (TestReconcileIsIdempotent). That is what makes the check below possible — without it every build would report drift.

Media types

A route can answer XML: server.OK writes JSON, server.XML writes XML. The schema describes the struct by its JSON shape; xml tags are not read.

What the generator writes into content is application/json, and only that. The reason is the error renderer: its body is an open map — a failure payload carries arbitrary keys — and encoding/xml cannot marshal a map at all, so a document that declared XML for a route could not be honest about that route's error responses.

Which format a handler actually answers with is not readable from here, so an XML route declares its own content once, by hand. Reconcile only ever adds application/json and leaves every other media type where it is — a hand-written one survives every later run.

Keeping the file honest

Nothing above forces anyone to run Reconcile, and an obligation that is only written down gets forgotten. Make the document a golden file instead:

func TestOpenAPIIsUpToDate(t *testing.T) {
    mux := server.New()
    myRouter.Setup(mux)

    openapitest.AssertUpToDate(t, mux.Routes(), "docs/openapi.yaml", "My API")
}

go test checks, go test -openapi.update refreshes — the same shape as every other golden in these repositories. openapitest is a package of its own because it imports testing, which has no business in the graph of a package a server links. It reconciles into a copy, so a run that reports drift does not also erase the evidence of what drifted. Two failure modes come out of it:

  • structure drifted — a route was added and the file was not refreshed;
  • prose is missing — a TODO(openapi) marker is still in the file, so somebody added an endpoint and wrote nothing about it. The marker is removed by hand, and removing it is the signature that says "I looked".

Nobody checks that what a person wrote is useful: the marker can be deleted and nothing written in its place. This catches forgetting, not sloppiness.

One mux per document. Routes accumulate in a slice shared with every group derived from the mux, so reusing one mux for two routers merges two specifications into one file. A service with several APIs — one command per audience, say — runs the loop once per router with a fresh server.New() each time.

How many documents to keep is decided by where the prose diverges, not where the configuration does. Two platforms sharing most of their handlers are one document, not two: the generator would write both for free, but a person would maintain the same descriptions twice. When they do diverge, Reconcile says so — operation X: in doc, not in code — which is the moment to decide, and it will not pass silently.

Where the types come from

server.Bind records the request and response types of the handler it wraps; the mux stores them in RouteMeta along with the method, the path pattern and the handler's program counter. server.Handle takes an untyped handler, so its RouteMeta carries nil types: the route reaches the document without schemas or an operationId, and needs prose until AssertUpToDate passes.

This is why the generator needs no build-time step and no annotations: the route table already knows the types, and reflection reads the rest.

Deriving schemas

A named struct, slice or map becomes a component named pkg.Type, referenced by $ref; anonymous structs are inlined. Two distinct types that share pkg.Type — a gateway's model/warehouse.Security and a generated pb/warehouse.Security — would be one component, so each takes as many package path segments as it needs to stand apart: model.warehouse.Security, pb.warehouse.Security. []byte is string/byte rather than an array, a map becomes an object with additionalProperties, an interface becomes the empty (any) schema. Self-referential types terminate because the component slot is reserved before its body is built.

Struct fields are read as follows:

Tag Effect
scan:"path=…" / query=… / header=… / cookie=… a parameter in that location; path is implicitly required
scan:"form=…" not described — the run warns that a urlencoded body is outside the document
scan:"embed" the embedded struct's properties and parameters are lifted into the parent
scan:"-" a body property when an untagged field makes whisk unmarshal the body; otherwise a parse() hook fills it, and unless json:"-" keeps it out of any body, the body is left for you to describe (see below)
json:"name", or a protobuf json= name body property name
no json tag at all the Go field name, which is what encoding/json emits
json:"-" not a property
validate:"required" in a request, added to required
json:",omitempty" / json:",omitzero" in a response, left out of required

validate rules are mapped onto the schema where OpenAPI can express them — email/uuid/url/hostname/datetime to format, oneof to enum, min/max/len to the length, range or item bounds appropriate to the type. Anything unmappable is not dropped: it lands in x-constraints, so the rule stays visible and the generator can still correct it.

A bound needs to know what it is counting, and a field whose type is named is a $ref carrying no type of its own — so the referenced schema answers instead. min=1 therefore means the same thing on []string and on a type Tags []string, and the keyword is written beside the $ref, which 3.1 evaluates as a sibling. A bound that means nothing for the referenced shape — a count on an object — falls through to the description like any other unmappable rule.

Three more sources: a type exposing Values() []string yields an enum; a generated protobuf enum yields a string enum of its value names, which is what pbjson writes — including the zero value; and well-known types (time.Time, uuid.UUID, protobuf timestamps, durations, wrappers, structpb, emptypb, anypb, fieldmaskpb) map to their standard representations.

A generated protobuf message in a response is read from its descriptor rather than its Go fields, because that is what pbjson walks: properties carry the lowerCamel JSON names, and a oneof's members stand in for the Go field that holds them. Any other google.protobuf message goes to protojson whole, so the run warns instead of guessing its shape.

Required and nullable

A request and a response answer "may this be absent" from different places. In a request it is the binder: validate:"required" lists the field, and omitempty means nothing, because decoding ignores it. In a response it is the encoder: every field is listed unless omitempty or omitzero lets it drop out. A protobuf message, as the body or anywhere inside it, goes through pbjson, which ignores json tags and writes every field, so every field is listed except the members of a real oneof, which are written only when set.

Nullability follows the same split. In a request, a pointer that isn't required is nullable — the binder's presence check is what required means there, so a required pointer is not. In a response, a pointer without omitempty is nullable, since a nil goes out as null; with omitempty the field is absent instead, never null. A slice or map is never nullable, since a nil one goes out empty. In a protobuf message, a field that tracks presence — a message, or a proto3 optional — is nullable, since pbjson writes it as null while unset.

OpenAPI keeps one required list per schema, whichever way it travels. A named type reached from both a request and a response is built once per side; when the two disagree, the document follows the request and the run warns, naming the fields. The fix is to give the response its own type.

Emission follows 3.1 rather than the 3.0 nullable keyword: a scalar becomes type: [string, "null"], and a $ref — which cannot carry a type — becomes anyOf: [{$ref}, {type: "null"}]. Reconciliation converts an existing field to the same form and strips a legacy nullable key. Whatever constrains the ref stays alongside the anyOf rather than moving under it.

Parameters

Parameters are hoisted into components/parameters and referenced from operations, so a query parameter shared by twenty routes is described once.

Sharing is conditional on agreement. Occurrences of one name are compared by location, structural type and requiredness; identical ones collapse into a single component, divergent ones are split into pkg.ReqType.name and reported as a warning. Merging them would quietly publish one contract for two different parameters — the split is deliberately noisy instead.

The component key is sanitized (sort[] → sort) while the parameter's own name keeps its original spelling. Path placeholders that no field declares are synthesized as inline required string parameters, since OpenAPI requires every placeholder to be declared.

operationId

Ids come from the handler function itself, escalating only as far as uniqueness demands: the method name, then package + method, then adding the path, then adding the HTTP method. A handler that is an anonymous closure has no meaningful name and gets no id.

The id is generator-owned and rewritten on every run, so renaming a handler heals the specification instead of leaving a stale id behind.

Reconciliation

For each schema, parameter and operation the code produces:

  • absent from the file — appended, with a TODO(openapi) comment naming what a human still owes it (semantics, description, examples, error responses);
  • present — compared structurally, then required, nullability and operationId are synchronized, and an operation gets every parameter the code binds. Parameters are matched by name and location, so a $ref and an inline entry are the same one; a parameter only the file has is warned about and kept. Nothing else is touched;
  • structurally incompatible — recorded as an error: a field whose type differs from the code, a field or property the code no longer has, a parameter whose in or schema disagrees.

Entries in the file that the code does not produce are warnings, never deletions. They are legitimate for schemas and parameters written by hand for a Handle route, and a useful smell after a rename.

The structural comparison deliberately ignores what people own. A string's format, its enum values and its description are not compared, and in a 3.1 nullable type array the "null" member is invisible to the comparison and handled separately.

What it does not do
  • Media types are declared, not derived — see above. A route that speaks a format the list does not name is documented as speaking the ones it does.
  • One response is scaffolded, 200 OK. Other statuses are yours to write, and survive.
  • A request body is scaffolded as required: true.
  • A request body whisk does not unmarshal into fields — the type decodes it itself (UnmarshalJSON, own or promoted), or a parse() hook fills a scan:"-" field, in the type or a scan:"embed" one — has a shape its fields do not show: its component is left empty for you to describe, the operation still refers to it, reconciliation keeps what you write, and the run warns.
  • info, servers, security and tags are outside the merge — hand-written and preserved.
  • A route registered with Handle reaches the document without schemas or an operationId, and needs prose until AssertUpToDate passes.
  • The route table is built at runtime, so generating a spec means starting the application far enough to construct the router. "Far enough" is the router, not a listening socket: the waffle server command takes --openapi <path>, writes the file and exits without calling Serve, and a test reaches the same point more cheaply.
Tests
File What it pins down
build_test.go synthesized path parameters, component key sanitizing, splitting divergent parameters
walk_test.go components derived from a route table
reconcile_test.go hand-written content survives a merge; a type mismatch is an error; a fresh document is a checklist of markers; reconciling twice changes nothing; an operation gets the parameter it lacks
nullable_test.go the 3.1 nullable forms, and wrapping/unwrapping a $ref round-trips
required_test.go required and nullability per side, the warning for a type both sides disagree on, a stale required dropped on reconcile, two types under one short name kept apart
pbjson_test.go a protobuf message in a response is described as pbjson writes it: every key required, every null nullable, a oneof member optional
features_test.go operationId derivation and collisions, header parameters, enums and time formats
emit_test.go a fresh document is self-contained

Directories

Path Synopsis
internal
openapitest
Separate from openapi because it imports testing, which has no business in the graph of a package a server links.
Separate from openapi because it imports testing, which has no business in the graph of a package a server links.
pb

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL