Documentation
¶
Overview ¶
Package httpapi holds the HTTP layer for both Relay planes.
- app/httpapi/inference — data plane: /v1/*, /healthz
- app/httpapi/control — admin plane: /auth/*, CRUD, /version, etc.
Each subpackage exposes a typed Deps and a Mount(chi.Router, Deps) huma.API entrypoint. The top-level package owns shared concerns: the OpenAI-shape error envelope used by both planes, the huma↔chi middleware adapter, and the build/version string.
Index ¶
Constants ¶
const DefaultMaxInflight = 2048
DefaultMaxInflight is the derived cap when RELAY_MAX_INFLIGHT is unset (or <= 0). Justification from the 2026-07-08 load test (12-core ARM pod): healthy load cleared ~1k rps at ~20ms → ~20 concurrent, and the achieved knee (~2,856 rps) sat far below this while still fast. So this cap never throttles healthy load — it only engages once latency degrades enough that concurrency climbs three orders of magnitude toward the observed 53k- goroutine / 1.3GB blowup, converting that spiral into a bounded ceiling. It is a fixed number, not 256×GOMAXPROCS: the binding constraint is per-pod memory (set by GOMEMLIMIT / the container limit), which does not scale with core count, so a per-core multiple is the wrong axis and runs loose on big pods.
const RetryAfterShed = "1"
RetryAfterShed is the Retry-After header value (seconds) sent with a shed 429. One second is far longer than the ~20ms a healthy request path takes, so the transient saturation has cleared by the time a well-behaved client retries.
Variables ¶
var ErrShed = errors.New("in-flight capacity reached")
ErrShed is returned by PreFlight when the in-flight cap is reached. The inference dispatch maps it to a retriable 429 + Retry-After; any other pre-flight error stays a 500.
var Version = "dev"
Version is the human-facing build version surfaced in the OpenAPI Info block, /version, and /config.json. Stamped at build time via
-ldflags "-X github.com/wyolet/relay/app/httpapi.Version=<tag>"
(the image build threads the release tag through docker-bake.hcl → Dockerfile). Unstamped builds report "dev".
Functions ¶
func HumaAuth ¶
HumaAuth adapts a net/http middleware into a huma per-operation middleware. Used by both planes to attach auth (or any http.Handler-shaped middleware) to specific huma operations rather than the whole chi router.
Pattern: pass any `func(http.Handler) http.Handler` and it composes onto a huma.Operation via Middlewares = huma.Middlewares{HumaAuth(mw), ...}.
func Install ¶
func Install()
Install installs the process-global huma overrides: OpenAI-compatible error envelope, and a schema namer that prefixes type names with their package's last segment (e.g. provider_Spec vs host_Spec) so the catalog kinds' uniform Spec sub-structs don't collide in the OpenAPI schema registry.
Idempotent — safe to call from every Mount entrypoint.
func InstallErrorRewriter ¶
func InstallErrorRewriter()
InstallErrorRewriter is the legacy alias for Install(). Retained until callers migrate.
func NewRegistry ¶
NewRegistry returns a huma schema Registry whose namer produces clean PascalCase schema ids in the generated OpenAPI:
- The entity types Provider / Host / Model / HostKey / RateLimit / Policy / Pricing / RelayKey keep their bare names.
- Sub-types defined inside an entity package get the entity name prepended so the 8 colliding `Spec` types become `ProviderSpec`, `HostSpec`, `ModelSpec`, etc. without renaming the Go types.
- Types in non-entity packages (meta.Metadata, meta.Owner, adapter.Name, …) keep their bare names — they don't collide.
Plane Mount() functions install the returned registry on their huma.Config.
func ScalarHandler ¶
func ScalarHandler(title, specURL string) http.HandlerFunc
ScalarHandler returns an HTML page that renders the OpenAPI spec at specURL using Scalar API Reference. Use it to replace huma's default Stoplight Elements docs UI: set cfg.DocsPath = "" before humachi.New(...) and register this handler on the chi router instead.
Types ¶
type Admission ¶ added in v0.9.8
type Admission struct {
// contains filtered or unexported fields
}
Admission bounds concurrent in-flight inference requests via a buffered channel used as a counting semaphore.
func NewAdmission ¶ added in v0.9.8
NewAdmission returns an Admission capped at max concurrent in-flight requests. max <= 0 uses DefaultMaxInflight.
func (*Admission) Collect ¶ added in v0.9.8
Collect is a lifecycle.Collector: it releases the slot this request holds. Runs on every Finalize (which fires at response-body close), so a streamed request holds its slot for the whole stream. Gated on the admit mark so a shed request is a no-op; the non-blocking receive is a belt-and-suspenders guard against ever draining below zero.
func (*Admission) PreFlight ¶ added in v0.9.8
func (a *Admission) PreFlight(_ context.Context, lc *lifecycle.Context, _ *lifecycle.PreFlightEvent) error
PreFlight is a lifecycle.PreFlightMiddleware: it acquires a slot without blocking. On success it marks lc so the Collector knows to release; when the cap is reached it counts the shed and returns ErrShed for the dispatch to map to a 429. A request whose Context can't carry the release mark is admitted rather than held (never take a slot we can't return).
type OpenAIError ¶
type OpenAIError struct {
Err OpenAIErrorInner `json:"error"`
HTTPStatus int `json:"-"`
}
OpenAIError implements huma.StatusError with the OpenAI-compatible shape:
{ "error": { "type": "...", "code": "...", "message": "..." } }
func (*OpenAIError) ContentType ¶
func (e *OpenAIError) ContentType(_ string) string
func (*OpenAIError) Error ¶
func (e *OpenAIError) Error() string
func (*OpenAIError) GetStatus ¶
func (e *OpenAIError) GetStatus() int
type OpenAIErrorInner ¶
type OpenAIErrorInner struct {
Type string `json:"type"`
Code string `json:"code,omitempty"`
Message string `json:"message"`
}
OpenAIErrorInner is the inner object of the OpenAI error envelope.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
OIDC login for the control plane: GET /auth/oidc/start redirects to the configured provider's hosted login; GET /auth/oidc/callback exchanges the authorization code and mints the same scs session password login mints, so everything downstream of login is identical for both paths.
|
OIDC login for the control plane: GET /auth/oidc/start redirects to the configured provider's hosted login; GET /auth/oidc/callback exchanges the authorization code and mints the same scs session password login mints, so everything downstream of login is identical for both paths. |
|
Package inference's Dispatch is the shape-agnostic per-request flow.
|
Package inference's Dispatch is the shape-agnostic per-request flow. |