README
¶
gorib/http
HTTP for gorib services: a chi-backed server with typed handlers, an HTTP client, and an OpenAPI generator that keeps the specification in step with the code.
| Package | What's in it |
|---|---|
server |
mux and route groups, typed handlers (Bind), middleware chain, error rendering, graceful shutdown, route introspection (Routes() []RouteMeta) |
client |
request building, response decoding, tracing |
model |
content types, transport errors, session |
pbjson |
protobuf ⇄ JSON marshalling |
openapi |
spec derivation from registered routes, and reconciliation with a hand-edited file |
openapi/openapitest |
the document as a golden file in a test |
For the service-side usage — wiring the command, registering routes, choosing middleware — see the waffle docs, docs/tools/http.md and docs/tools/http-handler.md in gitlab.com/gorib/waffle. This file documents the packages themselves: the shapes server is built from, and the openapi generator.
server
One handler shape, and everything else composes around it:
type Handler func(src Source) (Renderer, error)
type Layer = func(next Handler) Handler
Source is the request as an interface — Path, Query, Header, Cookie, Form, Body, plus the Lookup/LookupAll pair that satisfies scan.Source, which is what lets a whisk-generated binder read an HTTP request the same way it reads an AMQP delivery. Renderer is anything that can write itself to a http.ResponseWriter and report its status.
A handler that wants types rather than a request uses Bind. It runs the whisk-generated Parse, then Validate if the type has one, and the first error short-circuits before the function is called. A handler that returns a nil response and a nil error renders 204.
mux.Post("/login", server.Bind(func(ctx context.Context, req *model.LoginRequest) (*server.Response[Out], error) {
return server.OK(&Out{...}), nil
}))
Response carries status, headers and a body with a content type: OK, Created, Accepted, NoContent, or JSON/XML/Text/HTML when the type is not the default, and StreamWith for a body written incrementally.
Path matching
The route is chosen by the same string every decision below it is made on: the decoded r.URL.Path. chi on its own matches r.URL.RawPath whenever the client sent a path whose escaping is not canonical, while everything else reaches for URL.Path — a guard comparing a prefix, Health, the span and log attributes, the handler. Where that split cuts is a guarded prefix beside something wider:
mux.Get("/customers/{customerId}/*", auth, server.Handle(serveFile))
mux.Get("/*", server.Handle(serveFile))
GET /%63ustomers/5/secret.pdf misses the guarded route and lands in /*, while the handler goes on to open /customers/5/secret.pdf. So the mux pins the routing path to URL.Path before chi matches, and the service has nothing to remember. The window is narrower than "any percent-encoding" — net/url sets RawPath only for an escaping it would not have produced itself, so Cyrillic and spaces never diverge and the bug is invisible in ordinary traffic and in tests written with ordinary URLs.
Two consequences, both pinned in mux_test.go:
- a path parameter is now always the decoded value.
{code}ismskfor/facilities/%6dsk/stat, where it used to be%6dsk— while the same route already gaveМСКfor/facilities/%D0%9C%D0%A1%D0%9A/stat. Which of the two a handler got depended on how the rest of the path happened to be encoded; %2Finside a segment is a separator like any other./facilities/a%2Fb/statreads as/facilities/a/b/stat, one segment more than the route declares, and answers 404 where it used to match with{code}=a%2Fb. This is the price, and it is not recoverable by decoding less: a routing string that keeps%2Fis a second string again, and the handler readingURL.Pathwould still see two segments. A value that can carry a slash belongs in a query parameter or in the body, not in a path segment.
The raw string is taken out of the routing decision, not destroyed: RawPath and EscapedPath() still carry what came off the wire, which is what a log or an audit record wants. So the invariant has an edge the framework cannot close — a handler that decides on EscapedPath() rather than Path is back to two strings. Clearing RawPath instead would close that edge by rewriting EscapedPath(), String() and RequestURI() for every reader, honest ones included; the edge is the cheaper side of that trade. Decide on URL.Path.
Server defaults
NewServer starts from these and applies With* options on top. A zero means no
timeout, which is why two of them are zero on purpose.
| Setting | Default | Why not something else |
|---|---|---|
ReadHeaderTimeout |
10s | Slowloris protection. A full ReadTimeout would also cut off a slow upload. |
ReadTimeout |
0 | Body size is MaxBody's job; body time is left unbounded so a large upload can finish. |
WriteTimeout |
0 | Streaming, SSE and long responses have to stay possible. |
IdleTimeout |
120s | Stale keep-alives would otherwise accumulate. |
MaxHeaderBytes |
32 KB | Room for a JWT, several cookies and tracing baggage. The stdlib's 1 MiB is DoS-friendly. |
ShutdownTimeout |
0 | The orchestrator's grace period owns this. A hung shutdown is better observed than silently truncated. |
The standard chain
The middleware package ships seven layers. Their canonical order, outer to inner:
| Layer | What |
|---|---|
Compress |
gzips the response when the client asked for it and the content type is worth compressing |
Health |
answers the health path itself, so a probe never reaches session, tracing or the handler |
Session |
request identity into session.Session — correlation id, user agent, source ip — plus the incoming trace context onto the ctx, and echoes the correlation id back on the response |
Tracing |
SERVER span, provider into ctx, http.server.request.duration and http.server.active_requests |
Log |
logger into ctx, then the request and the response at trace, raised to error only when the status is 5xx or nothing rendered at all |
Error |
maps a returned error to a status and a rendered body |
Recover |
failure.Catch, so a handler panic becomes an error |
The package only ships the layers; assembling them in this order is the caller's job, and the order is the whole content of that job.
The order is load-bearing at three points. Error sits inside Tracing and Log, so both see the status that was actually rendered rather than the bare error — put it outside and every failed request is recorded with the handler's zero status. Recover is innermost, so a panic becomes an error while the span is still open; anything further out would end the span unrecorded. And Tracing sits after Session, so it can replace the context's traceparent with its own span's — otherwise an outgoing call would advertise the upstream caller as its parent instead of this request.
Per-route layers from mux.Get(path, layer, ...) compose inside Recover.
client
client.Client builds requests, decodes responses and traces both. The provider comes from the context (hilton.ProviderFromContext), so a call inherits the trace of the request that issued it; it emits a CLIENT span and http.client.request.duration, and injects the trace context into the outgoing headers. It also forwards session.Session's correlation id and the context's traceparent — which is the other end of the ordering rule above: the server's Tracing layer put its own span there so this call points at the current request, not at whoever called it.
For what the metrics look like across all of gorib, see docs/tools/hilton.md in waffle.
openapi
The package answers one question: how does a specification stay true to the code without taking the prose away from the humans who wrote it.
The answer is a split of ownership, and the rule that makes it work is stricter than "structure versus prose": anything the generator writes, it has to be able to rewrite. A value it produces once and can never correct goes stale in silence, and the golden check below then certifies a document that lies.
So the line falls here:
| Owned by | Keys | On a later run |
|---|---|---|
| the code | paths, operations, operationId, request body $ref, schemas, field types, required, nullability, and everything derived from a validate tag — format, enum, pattern, the length/range/item bounds, x-constraints |
rewritten from the model |
| the human | summary, description, example, extra responses, anything the generator does not produce |
never touched |
Two consequences worth knowing.
A rule OpenAPI has no keyword for lands in x-constraints, not in the
description. It used to be appended to the prose, where nothing could correct it —
so a rule that changed left the old sentence behind forever. Reconciliation moves
such a sentence out of an existing description and says so in its warnings; the
rest of the description is left exactly as written.
A derived keyword the generator has no opinion about is left exactly as found, and
nothing is said about it. Refining format by hand on a field with no validate
tag is legitimate, and by the rule above such a value is the human's — so it is not
the generator's to delete, nor to complain about on every run for the rest of the
project's life.
The cost is the other reading of the same file: a validate rule deleted from the
code leaves its keyword behind, and a run cannot tell that from a refinement. The
place to catch it is the diff where the tag was deleted, not a warning that would
have to fire for both.
Entry points
func Build(routes []server.RouteMeta) (*Spec, []string)
func Emit(routes []server.RouteMeta, path, title string) ([]string, error)
func Reconcile(routes []server.RouteMeta, path, title string) ([]string, error)
func Components(routes []server.RouteMeta) map[string]*Schema
Build derives the spec model and returns warnings. Emit dumps a fresh document whole (OpenAPI 3.1.0, info.version 1.0.0). Reconcile is what callers normally use: it merges into the file at path, and when that file does not exist it reconciles into an empty skeleton rather than delegating to Emit. The difference matters — everything in a file that does not exist yet was put there by the generator, so it all has to arrive through the same branch a later addition does, carrying the TODO(openapi) that says nobody has written the prose. Components is the schemas alone, for callers that only need the type catalogue.
Both file-writing functions return warnings alongside the error. Reconcile writes the merged document before returning a structural error, so a failing run still leaves something to inspect.
Reconcile is idempotent: a run against an up-to-date document leaves the bytes untouched (TestReconcileIsIdempotent). That is what makes the check below possible — without it every build would report drift.
Media types
A route can answer XML: server.OK writes JSON, server.XML writes XML, and
the schema carries both serializations — element names, attributes and
namespaces come off the encoding/xml tags of the same struct.
What the generator writes into content is application/json, and only that.
The reason is the error renderer: its body is an open map — a failure payload
carries arbitrary keys — and encoding/xml cannot marshal a map at all, so a
document that declared XML for a route could not be honest about that route's
error responses.
Which format a handler actually answers with is not readable from here, so an
XML route declares its own content once, by hand. Reconcile only ever adds
application/json and leaves every other media type where it is — a
hand-written one survives every later run.
Keeping the file honest
Nothing above forces anyone to run Reconcile, and an obligation that is only written down gets forgotten. Make the document a golden file instead:
func TestOpenAPIIsUpToDate(t *testing.T) {
mux := server.New()
myRouter.Setup(mux)
openapitest.AssertUpToDate(t, mux.Routes(), "docs/openapi.yaml", "My API")
}
go test checks, go test -openapi.update refreshes — the same shape as every other golden in these repositories. openapitest is a package of its own because it imports testing, which has no business in the graph of a package a server links. It reconciles into a copy, so a run that reports drift does not also erase the evidence of what drifted. Two failure modes come out of it:
- structure drifted — a route was added and the file was not refreshed;
- prose is missing — a
TODO(openapi)marker is still in the file, so somebody added an endpoint and wrote nothing about it. The marker is removed by hand, and removing it is the signature that says "I looked".
Nobody checks that what a person wrote is useful: the marker can be deleted and nothing written in its place. This catches forgetting, not sloppiness.
One mux per document. Routes accumulate in a slice shared with every group derived from the mux, so reusing one mux for two routers merges two specifications into one file. A service with several APIs — one command per audience, say — runs the loop once per router with a fresh server.New() each time.
How many documents to keep is decided by where the prose diverges, not where the configuration does. Two platforms sharing most of their handlers are one document, not two: the generator would write both for free, but a person would maintain the same descriptions twice. When they do diverge, Reconcile says so — operation X: in doc, not in code — which is the moment to decide, and it will not pass silently.
Where the types come from
server.Bind records the request and response types of the handler it wraps; the mux stores them in RouteMeta along with the method, the path pattern and the handler's program counter. server.Handle takes an untyped handler, so its RouteMeta carries nil types and contributes nothing to the spec.
This is why the generator needs no build-time step and no annotations: the route table already knows the types, and reflection reads the rest.
Deriving schemas
A named struct, slice or map becomes a component named pkg.Type, referenced by $ref; anonymous structs are inlined. []byte is string/byte rather than an array, a map becomes an object with additionalProperties, an interface becomes the empty (any) schema. Self-referential types terminate because the component slot is reserved before its body is built.
Struct fields are read as follows:
| Tag | Effect |
|---|---|
scan:"path=…" / query=… / header=… |
a parameter in that location; path is implicitly required |
scan:"embed" |
the embedded struct's properties and parameters are lifted into the parent |
json:"name", or a protobuf json= name |
body property name |
no json tag at all |
the Go field name, which is what encoding/json emits |
json:"-" |
not a property |
validate:"required" |
added to required |
validate rules are mapped onto the schema where OpenAPI can express them — email/uuid/url/hostname/datetime to format, oneof to enum, min/max/len to the length, range or item bounds appropriate to the type. Anything unmappable is not dropped: it lands in x-constraints, so the rule stays visible and the generator can still correct it.
A bound needs to know what it is counting, and a field whose type is named is a
$ref carrying no type of its own — so the referenced schema answers instead.
min=1 therefore means the same thing on []string and on a type Tags []string,
and the keyword is written beside the $ref, which 3.1 evaluates as a sibling. A
bound that means nothing for the referenced shape — a count on an object — falls
through to the description like any other unmappable rule.
Two more sources: a type exposing Values() []string yields an enum, and well-known types (time.Time, uuid.UUID, protobuf timestamps, durations, wrappers, structpb) map to their standard representations.
XML
A property name says nothing about the element it becomes, which is what the
xml object is for. It is derived from encoding/xml tags, so the two
serializations of one struct are described from one place:
| Tag | In the document |
|---|---|
xml:"reference" |
xml: {name: reference}, and only when it differs from the json key |
xml:"id,attr" |
xml: {attribute: true} |
xml:"http://ns name" |
+ namespace: http://ns |
xml:"lines>line" on a slice |
xml: {name: lines, wrapped: true}, and xml: {name: line} on items |
XMLName xml.Name with a tag |
the element the struct itself becomes |
no xml tag |
the Go field name, which is what encoding/xml emits |
There is no prefix: encoding/xml has no way to declare one, and a keyword
nothing can populate is worse than a missing one.
,chardata, ,innerxml, ,comment, ,cdata and ,any describe content that is
not an element with a name, so there is nothing to write and the run says so.
One limitation the single-schema model cannot express: xml:"-" on a field that
still has a json key means the two serializations carry different property sets,
and a schema describes one set. Such a field is documented as present.
Nullability
A pointer field that isn't required is nullable — the binder's presence check is what required means here, so a required pointer is not treated as nullable.
Emission follows 3.1 rather than the 3.0 nullable keyword: a scalar becomes type: [string, "null"], and a $ref — which cannot carry a type — becomes anyOf: [{$ref}, {type: "null"}]. Reconciliation converts an existing field to the same form and strips a legacy nullable key. Whatever constrains the ref stays alongside the anyOf rather than moving under it.
Parameters
Parameters are hoisted into components/parameters and referenced from operations, so a query parameter shared by twenty routes is described once.
Sharing is conditional on agreement. Occurrences of one name are compared by location, structural type and requiredness; identical ones collapse into a single component, divergent ones are split into pkg.ReqType.name and reported as a warning. Merging them would quietly publish one contract for two different parameters — the split is deliberately noisy instead.
The component key is sanitized (sort[] → sort) while the parameter's own name keeps its original spelling. Path placeholders that no field declares are synthesized as inline required string parameters, since OpenAPI requires every placeholder to be declared.
operationId
Ids come from the handler function itself, escalating only as far as uniqueness demands: the method name, then package + method, then adding the path, then adding the HTTP method. A handler that is an anonymous closure has no meaningful name and gets no id.
The id is generator-owned and rewritten on every run, so renaming a handler heals the specification instead of leaving a stale id behind.
Reconciliation
For each schema, parameter and operation the code produces:
- absent from the file — appended, with a
TODO(openapi)comment naming what a human still owes it (semantics, description, examples, error responses); - present — compared structurally, then
required, nullability andoperationIdare synchronized. Nothing else is touched; - structurally incompatible — recorded as an error: a field whose type differs from the code, a field or property the code no longer has, a parameter whose
inor schema disagrees.
Entries in the file that the code does not produce are warnings, never deletions. They are legitimate for routes registered with Handle, and a useful smell after a rename.
The structural comparison deliberately ignores what people own. A string's format, its enum values and its description are not compared, and in a 3.1 nullable type array the "null" member is invisible to the comparison and handled separately.
What it does not do
- Media types are declared, not derived — see above. A route that speaks a format the list does not name is documented as speaking the ones it does.
- One response is scaffolded,
200 OK. Other statuses are yours to write, and survive. - A request body is scaffolded as
required: true. info,servers,securityandtagsare outside the merge — hand-written and preserved.- Routes registered with
Handlenever appear. - The route table is built at runtime, so generating a spec means starting the application far enough to construct the router. "Far enough" is the router, not a listening socket: the waffle server command takes
--openapi <path>, writes the file and exits without callingServe, and a test reaches the same point more cheaply.
Tests
| File | What it pins down |
|---|---|
build_test.go |
synthesized path parameters, component key sanitizing, splitting divergent parameters |
walk_test.go |
components derived from a route table |
reconcile_test.go |
hand-written content survives a merge; a type mismatch is an error; a fresh document is a checklist of markers; reconciling twice changes nothing |
nullable_test.go |
the 3.1 nullable forms, and wrapping/unwrapping a $ref round-trips |
features_test.go |
operationId derivation and collisions, header parameters, enums and time formats |
emit_test.go |
a fresh document is self-contained |
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
openapitest
Separate from openapi because it imports testing, which has no business in the graph of a package a server links.
|
Separate from openapi because it imports testing, which has no business in the graph of a package a server links. |
|
Package pbjson marshals proto messages to JSON the way the OpenAPI contract describes them: 64-bit integers are JSON numbers, not the quoted strings the canonical proto3 JSON mapping demands.
|
Package pbjson marshals proto messages to JSON the way the OpenAPI contract describes them: 64-bit integers are JSON numbers, not the quoted strings the canonical proto3 JSON mapping demands. |