http

module
v1.13.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 24, 2026 License: MIT

README

gorib/http

HTTP for gorib services: a chi-backed server with typed handlers, an HTTP client, and an OpenAPI generator that keeps the specification in step with the code.

Package What's in it
server mux and route groups, typed handlers (Bind), middleware chain, error rendering, graceful shutdown, route introspection (Routes() []RouteMeta)
client request building, response decoding, tracing
model content types, transport errors, session
pbjson protobuf ⇄ JSON marshalling
openapi spec derivation from registered routes, and reconciliation with a hand-edited file
openapi/openapitest the document as a golden file in a test

For the service-side usage — wiring the command, registering routes, choosing middleware — see the waffle docs, docs/tools/http.md and docs/tools/http-handler.md in gitlab.com/gorib/waffle. This file documents the packages themselves: the shapes server is built from, and the openapi generator.

server

One handler shape, and everything else composes around it:

type Handler func(src Source) (Renderer, error)
type Layer   = func(next Handler) Handler

Source is the request as an interface — Path, Query, Header, Cookie, Form, Body, plus the Lookup/LookupAll pair that satisfies scan.Source, which is what lets a whisk-generated binder read an HTTP request the same way it reads an AMQP delivery. Renderer is anything that can write itself to a http.ResponseWriter and report its status.

A handler that wants types rather than a request uses Bind. It runs the whisk-generated Parse, then Validate if the type has one, and the first error short-circuits before the function is called. A handler that returns a nil response and a nil error renders 204.

mux.Post("/login", server.Bind(func(ctx context.Context, req *model.LoginRequest) (*server.Response[Out], error) {
    return server.OK(&Out{...}), nil
}))

Response carries status, headers and a body with a content type: OK, Created, Accepted, NoContent, or JSON/XML/Text/HTML when the type is not the default, and StreamWith for a body written incrementally.

Source.Body reads the whole body and Source.Peek only its head, leaving the request to carry all of it.

Path matching

The route is chosen by the same string every decision below it is made on: the decoded r.URL.Path. chi on its own matches r.URL.RawPath whenever the client sent a path whose escaping is not canonical, while everything else reaches for URL.Path — a guard comparing a prefix, Health, the span and log attributes, the handler. Where that split cuts is a guarded prefix beside something wider:

mux.Get("/customers/{customerId}/*", auth, server.Handle(serveFile))
mux.Get("/*", server.Handle(serveFile))

GET /%63ustomers/5/secret.pdf misses the guarded route and lands in /*, while the handler goes on to open /customers/5/secret.pdf. So the mux pins the routing path to URL.Path before chi matches, and the service has nothing to remember. The window is narrower than "any percent-encoding" — net/url sets RawPath only for an escaping it would not have produced itself, so Cyrillic and spaces never diverge and the bug is invisible in ordinary traffic and in tests written with ordinary URLs.

Two consequences, both pinned in mux_test.go:

  • a path parameter is now always the decoded value. {code} is msk for /facilities/%6dsk/stat, where it used to be %6dsk — while the same route already gave МСК for /facilities/%D0%9C%D0%A1%D0%9A/stat. Which of the two a handler got depended on how the rest of the path happened to be encoded;
  • %2F inside a segment is a separator like any other. /facilities/a%2Fb/stat reads as /facilities/a/b/stat, one segment more than the route declares, and answers 404 where it used to match with {code} = a%2Fb. This is the price, and it is not recoverable by decoding less: a routing string that keeps %2F is a second string again, and the handler reading URL.Path would still see two segments. A value that can carry a slash belongs in a query parameter or in the body, not in a path segment.

Dot segments and doubled slashes are the same split by another road: chi matches //customers/5/secret.pdf against /*, and http.Dir cleans it back to /customers/5/secret.pdf. So a path that is not clean is never routed at all — it answers 308 to its cleaned form, query kept, as the stdlib ServeMux does.

The raw string is taken out of the routing decision, not destroyed: RawPath and EscapedPath() still carry what came off the wire, which is what a log or an audit record wants. So the invariant has an edge the framework cannot close — a handler that decides on EscapedPath() rather than Path is back to two strings. Clearing RawPath instead would close that edge by rewriting EscapedPath(), String() and RequestURI() for every reader, honest ones included; the edge is the cheaper side of that trade. Decide on URL.Path.

Server defaults

NewServer starts from these and applies With* options on top. A zero means no timeout, which is why two of them are zero on purpose.

Setting Default Why not something else
ReadHeaderTimeout 10s Slowloris protection. A full ReadTimeout would also cut off a slow upload.
ReadTimeout 0 Body size is MaxBody's job; body time is left unbounded so a large upload can finish.
WriteTimeout 0 Streaming, SSE and long responses have to stay possible.
IdleTimeout 120s Stale keep-alives would otherwise accumulate.
MaxHeaderBytes 32 KB Room for a JWT, several cookies and tracing baggage. The stdlib's 1 MiB is DoS-friendly.
ShutdownTimeout 0 The orchestrator's grace period owns this. A hung shutdown is better observed than silently truncated.
The standard chain

The middleware package ships seven layers. Their canonical order, outer to inner:

Layer What
Compress gzips the response when the client asked for it and the content type is worth compressing
Health answers the health path itself, so a probe never reaches session, tracing or the handler
Session request identity into session.Session — correlation id, user agent, source ip, the first 4 KiB of the body — plus the incoming trace context onto the ctx, and echoes the correlation id back on the response
Tracing SERVER span, provider into ctx, http.server.request.duration and http.server.active_requests
Log logger into ctx, then the request and the response at trace, raised to error only when the status is 5xx or nothing rendered at all
Error maps a returned error to a status and a rendered body; the error itself rides out on the response for Log and Tracing, and a 500's own text never reaches the client
Recover failure.Catch, so a handler panic becomes an error

The package only ships the layers; assembling them in this order is the caller's job, and the order is the whole content of that job.

The order is load-bearing at three points. Error sits inside Tracing and Log, so both see the status that was actually rendered rather than the bare error — put it outside and every failed request is recorded with the handler's zero status. Recover is innermost, so a panic becomes an error while the span is still open; anything further out would end the span unrecorded. And Tracing sits after Session, so it can replace the context's traceparent with its own span's — otherwise an outgoing call would advertise the upstream caller as its parent instead of this request.

Per-route layers from mux.Get(path, layer, ...) compose inside Recover.

Global layers run before routing, so no per-route limit can bound what they read. That is why they only ever Peek the body, at most 4 KiB, and why MaxBody is per-route and still holds.

client

client.Client builds requests, decodes responses and traces both. The provider comes from the context (hilton.ProviderFromContext), so a call inherits the trace of the request that issued it; it emits a CLIENT span and http.client.request.duration, and injects the trace context into the outgoing headers. It also forwards session.Session's correlation id and the context's traceparent — which is the other end of the ordering rule above: the server's Tracing layer put its own span there so this call points at the current request, not at whoever called it.

For what the metrics look like across all of gorib, see docs/tools/hilton.md in waffle.

openapi

The package answers one question: how does a specification stay true to the code without taking the prose away from the humans who wrote it.

The answer is a split of ownership, and the rule that makes it work is stricter than "structure versus prose": anything the generator writes, it has to be able to rewrite. A value it produces once and can never correct goes stale in silence, and the golden check below then certifies a document that lies.

So the line falls here:

Owned by Keys On a later run
the code paths, operations, operationId, request body $ref, schemas, field types, required, nullability, and everything derived from a validate tag — format, enum, pattern, the length/range/item bounds, x-constraints rewritten from the model
the human summary, description, example, extra responses, anything the generator does not produce never touched

Two consequences worth knowing.

A rule OpenAPI has no keyword for lands in x-constraints, not in the description. It used to be appended to the prose, where nothing could correct it — so a rule that changed left the old sentence behind forever. Reconciliation moves such a sentence out of an existing description and says so in its warnings; the rest of the description is left exactly as written.

A derived keyword the generator has no opinion about is left exactly as found, and nothing is said about it. Refining format by hand on a field with no validate tag is legitimate, and by the rule above such a value is the human's — so it is not the generator's to delete, nor to complain about on every run for the rest of the project's life.

The cost is the other reading of the same file: a validate rule deleted from the code leaves its keyword behind, and a run cannot tell that from a refinement. The place to catch it is the diff where the tag was deleted, not a warning that would have to fire for both.

Entry points
func Build(routes []server.RouteMeta) (*Spec, []string)
func Emit(routes []server.RouteMeta, path, title string) ([]string, error)
func Reconcile(routes []server.RouteMeta, path, title string) ([]string, error)
func Components(routes []server.RouteMeta) map[string]*Schema

Build derives the spec model and returns warnings. Emit dumps a fresh document whole (OpenAPI 3.1.0, info.version 1.0.0). Reconcile is what callers normally use: it merges into the file at path, and when that file does not exist it reconciles into an empty skeleton rather than delegating to Emit. The difference matters — everything in a file that does not exist yet was put there by the generator, so it all has to arrive through the same branch a later addition does, carrying the TODO(openapi) that says nobody has written the prose. Components is the schemas alone, for callers that only need the type catalogue.

Both file-writing functions return warnings alongside the error. Reconcile writes the merged document before returning a structural error, so a failing run still leaves something to inspect.

Reconcile is idempotent: a run against an up-to-date document leaves the bytes untouched (TestReconcileIsIdempotent). That is what makes the check below possible — without it every build would report drift.

Media types

A route can answer XML: server.OK writes JSON, server.XML writes XML, and the schema carries both serializations — element names, attributes and namespaces come off the encoding/xml tags of the same struct.

What the generator writes into content is application/json, and only that. The reason is the error renderer: its body is an open map — a failure payload carries arbitrary keys — and encoding/xml cannot marshal a map at all, so a document that declared XML for a route could not be honest about that route's error responses.

Which format a handler actually answers with is not readable from here, so an XML route declares its own content once, by hand. Reconcile only ever adds application/json and leaves every other media type where it is — a hand-written one survives every later run.

Keeping the file honest

Nothing above forces anyone to run Reconcile, and an obligation that is only written down gets forgotten. Make the document a golden file instead:

func TestOpenAPIIsUpToDate(t *testing.T) {
    mux := server.New()
    myRouter.Setup(mux)

    openapitest.AssertUpToDate(t, mux.Routes(), "docs/openapi.yaml", "My API")
}

go test checks, go test -openapi.update refreshes — the same shape as every other golden in these repositories. openapitest is a package of its own because it imports testing, which has no business in the graph of a package a server links. It reconciles into a copy, so a run that reports drift does not also erase the evidence of what drifted. Two failure modes come out of it:

  • structure drifted — a route was added and the file was not refreshed;
  • prose is missing — a TODO(openapi) marker is still in the file, so somebody added an endpoint and wrote nothing about it. The marker is removed by hand, and removing it is the signature that says "I looked".

Nobody checks that what a person wrote is useful: the marker can be deleted and nothing written in its place. This catches forgetting, not sloppiness.

One mux per document. Routes accumulate in a slice shared with every group derived from the mux, so reusing one mux for two routers merges two specifications into one file. A service with several APIs — one command per audience, say — runs the loop once per router with a fresh server.New() each time.

How many documents to keep is decided by where the prose diverges, not where the configuration does. Two platforms sharing most of their handlers are one document, not two: the generator would write both for free, but a person would maintain the same descriptions twice. When they do diverge, Reconcile says so — operation X: in doc, not in code — which is the moment to decide, and it will not pass silently.

Where the types come from

server.Bind records the request and response types of the handler it wraps; the mux stores them in RouteMeta along with the method, the path pattern and the handler's program counter. server.Handle takes an untyped handler, so its RouteMeta carries nil types and contributes nothing to the spec.

This is why the generator needs no build-time step and no annotations: the route table already knows the types, and reflection reads the rest.

Deriving schemas

A named struct, slice or map becomes a component named pkg.Type, referenced by $ref; anonymous structs are inlined. []byte is string/byte rather than an array, a map becomes an object with additionalProperties, an interface becomes the empty (any) schema. Self-referential types terminate because the component slot is reserved before its body is built.

Struct fields are read as follows:

Tag Effect
scan:"path=…" / query=… / header=… a parameter in that location; path is implicitly required
scan:"embed" the embedded struct's properties and parameters are lifted into the parent
json:"name", or a protobuf json= name body property name
no json tag at all the Go field name, which is what encoding/json emits
json:"-" not a property
validate:"required" in a request, added to required
json:",omitempty" / json:",omitzero" in a response, left out of required

validate rules are mapped onto the schema where OpenAPI can express them — email/uuid/url/hostname/datetime to format, oneof to enum, min/max/len to the length, range or item bounds appropriate to the type. Anything unmappable is not dropped: it lands in x-constraints, so the rule stays visible and the generator can still correct it.

A bound needs to know what it is counting, and a field whose type is named is a $ref carrying no type of its own — so the referenced schema answers instead. min=1 therefore means the same thing on []string and on a type Tags []string, and the keyword is written beside the $ref, which 3.1 evaluates as a sibling. A bound that means nothing for the referenced shape — a count on an object — falls through to the description like any other unmappable rule.

Three more sources: a type exposing Values() []string yields an enum; a generated protobuf enum yields a string enum of its value names, which is what pbjson writes — including the zero value; and well-known types (time.Time, uuid.UUID, protobuf timestamps, durations, wrappers, structpb) map to their standard representations.

XML

A property name says nothing about the element it becomes, which is what the xml object is for. It is derived from encoding/xml tags, so the two serializations of one struct are described from one place:

Tag In the document
xml:"reference" xml: {name: reference}, and only when it differs from the json key
xml:"id,attr" xml: {attribute: true}
xml:"http://ns name" + namespace: http://ns
xml:"lines>line" on a slice xml: {name: lines, wrapped: true}, and xml: {name: line} on items
XMLName xml.Name with a tag the element the struct itself becomes
no xml tag the Go field name, which is what encoding/xml emits

There is no prefix: encoding/xml has no way to declare one, and a keyword nothing can populate is worse than a missing one.

,chardata, ,innerxml, ,comment, ,cdata and ,any describe content that is not an element with a name, so there is nothing to write and the run says so.

One limitation the single-schema model cannot express: xml:"-" on a field that still has a json key means the two serializations carry different property sets, and a schema describes one set. Such a field is documented as present.

Required and nullable

A request and a response answer "may this be absent" from different places. In a request it is the binder: validate:"required" lists the field, and omitempty means nothing, because decoding ignores it. In a response it is the encoder: every field is listed unless omitempty or omitzero lets it drop out. The exception is a gRPC message returned as is, without conversion: it goes through pbjson, which ignores json tags, so the rule holds for it only when the build strips omitempty from the generated tags.

Nullability follows the same split. In a request, a pointer that isn't required is nullable — the binder's presence check is what required means there, so a required pointer is not. In a response, a pointer without omitempty is nullable, since a nil goes out as null; with omitempty the field is absent instead, never null.

OpenAPI keeps one required list per schema, whichever way it travels. A named type reached from both a request and a response is built once per side; when the two disagree, the document follows the request and the run warns, naming the fields. The fix is to give the response its own type.

Emission follows 3.1 rather than the 3.0 nullable keyword: a scalar becomes type: [string, "null"], and a $ref — which cannot carry a type — becomes anyOf: [{$ref}, {type: "null"}]. Reconciliation converts an existing field to the same form and strips a legacy nullable key. Whatever constrains the ref stays alongside the anyOf rather than moving under it.

Parameters

Parameters are hoisted into components/parameters and referenced from operations, so a query parameter shared by twenty routes is described once.

Sharing is conditional on agreement. Occurrences of one name are compared by location, structural type and requiredness; identical ones collapse into a single component, divergent ones are split into pkg.ReqType.name and reported as a warning. Merging them would quietly publish one contract for two different parameters — the split is deliberately noisy instead.

The component key is sanitized (sort[] → sort) while the parameter's own name keeps its original spelling. Path placeholders that no field declares are synthesized as inline required string parameters, since OpenAPI requires every placeholder to be declared.

operationId

Ids come from the handler function itself, escalating only as far as uniqueness demands: the method name, then package + method, then adding the path, then adding the HTTP method. A handler that is an anonymous closure has no meaningful name and gets no id.

The id is generator-owned and rewritten on every run, so renaming a handler heals the specification instead of leaving a stale id behind.

Reconciliation

For each schema, parameter and operation the code produces:

  • absent from the file — appended, with a TODO(openapi) comment naming what a human still owes it (semantics, description, examples, error responses);
  • present — compared structurally, then required, nullability and operationId are synchronized. Nothing else is touched;
  • structurally incompatible — recorded as an error: a field whose type differs from the code, a field or property the code no longer has, a parameter whose in or schema disagrees.

Entries in the file that the code does not produce are warnings, never deletions. They are legitimate for routes registered with Handle, and a useful smell after a rename.

The structural comparison deliberately ignores what people own. A string's format, its enum values and its description are not compared, and in a 3.1 nullable type array the "null" member is invisible to the comparison and handled separately.

What it does not do
  • Media types are declared, not derived — see above. A route that speaks a format the list does not name is documented as speaking the ones it does.
  • One response is scaffolded, 200 OK. Other statuses are yours to write, and survive.
  • A request body is scaffolded as required: true.
  • info, servers, security and tags are outside the merge — hand-written and preserved.
  • Routes registered with Handle never appear.
  • The route table is built at runtime, so generating a spec means starting the application far enough to construct the router. "Far enough" is the router, not a listening socket: the waffle server command takes --openapi <path>, writes the file and exits without calling Serve, and a test reaches the same point more cheaply.
Tests
File What it pins down
build_test.go synthesized path parameters, component key sanitizing, splitting divergent parameters
walk_test.go components derived from a route table
reconcile_test.go hand-written content survives a merge; a type mismatch is an error; a fresh document is a checklist of markers; reconciling twice changes nothing
nullable_test.go the 3.1 nullable forms, and wrapping/unwrapping a $ref round-trips
required_test.go required and nullability per side, the warning for a type both sides disagree on, a stale required dropped on reconcile
features_test.go operationId derivation and collisions, header parameters, enums and time formats
emit_test.go a fresh document is self-contained

Directories

Path Synopsis
internal
openapitest
Separate from openapi because it imports testing, which has no business in the graph of a package a server links.
Separate from openapi because it imports testing, which has no business in the graph of a package a server links.
Package pbjson marshals proto messages to JSON the way the OpenAPI contract describes them: 64-bit integers are JSON numbers, not the quoted strings the canonical proto3 JSON mapping demands, and an enum number with no name is an error rather than a bare number.
Package pbjson marshals proto messages to JSON the way the OpenAPI contract describes them: 64-bit integers are JSON numbers, not the quoted strings the canonical proto3 JSON mapping demands, and an enum number with no name is an error rather than a bare number.
pb

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL