gateway

type AmbiguousCodecMatchError ¶

type AmbiguousCodecMatchError struct {
	Method string
	Path   string
	Count  int
}

AmbiguousCodecMatchError reports that more than one configured ServerCodec matched the same concrete request. This indicates a caller misconfiguration (overlapping MatchRequest implementations registered together), not a client error.

func (e *AmbiguousCodecMatchError) Error() string

type AuthenticationError ¶

type AuthenticationError struct{}

AuthenticationError reports that a request failed gateway-local authentication: a missing Authorization header, a non-Bearer scheme, an oversized header, or a token that does not match. Every one of those causes is reported identically -- by design, AuthenticationError carries no distinguishing detail -- so a response body or log line built from this error can never leak *why* authentication failed, only that it did.

func (e *AuthenticationError) Error() string

type Authenticator ¶

type Authenticator interface {
	// Authenticate reports whether req carries a valid credential. It must
	// not consume req.Body. A non-nil error is always an
	// *AuthenticationError: this interface deliberately has exactly one
	// failure mode so a caller never needs to distinguish "why" auth failed.
	Authenticate(req *http.Request) error
}

Authenticator authenticates one inbound HTTP request against the gateway's own local, inbound credential. This is a distinct, separate concept from inference/auth's Authenticator: that package authorizes OUTBOUND requests to a provider with provider credentials; this interface authenticates INBOUND requests reaching the gateway's own HTTP surface, and is scoped to this package.

func StaticToken(token string) Authenticator

type Binding ¶

type Binding struct {
	BaseURL string
	Token   string
}

Binding reports a bound Server's loopback base URL and bearer token. It is returned by value from Server.Binding -- see that method's doc comment for why Binding() itself returns a bare (string, string, bool) tuple instead of this struct.

type ConcurrencyLimitExceededError ¶

type ConcurrencyLimitExceededError struct{}

ConcurrencyLimitExceededError reports that Config.MaxConcurrent in-flight requests were already admitted when this request arrived. Admission is reject-on-full, never queued.

func (e *ConcurrencyLimitExceededError) Error() string

type Config ¶

type Config struct {
	// Resolver maps (ingress format, requested model alias) to a Target.
	// Required.
	Resolver Resolver

	// Codecs is every ingress dialect this Handler serves, keyed by the
	// model.APIFormat it decodes/encodes. The key doubles as the Ingress
	// value passed to Resolver.Resolve when that codec's MatchRequest wins
	// route selection for a request -- codec.ServerCodec itself carries no
	// APIFormat accessor, so Config is the one place that association is
	// made. At least one entry is required.
	Codecs map[model.APIFormat]codec.ServerCodec

	// Authenticate authenticates the gateway's own local inbound token.
	// Required.
	Authenticate Authenticator

	// ContextCounter serves the Anthropic-dialect
	// POST /v1/messages/count_tokens auxiliary route (matched via
	// anthropicapi.MatchCountTokensRequest, which lives outside the
	// generic codec.ServerCodec surface -- see handler.go). This field is
	// NOT part of the design doc's abbreviated Config sketch
	// (Resolver/Codecs/Authenticate/MaxRequestBody/MaxConcurrent); it was
	// added here because Task 6 requires the count_tokens route to call a
	// configured contextcount.ContextCounter, and the sketch had nowhere to
	// configure one. Optional: a nil ContextCounter is a valid
	// configuration -- a count_tokens request then fails cleanly with a
	// typed *CountTokensUnavailableError (503) instead of panicking on a
	// nil interface call.
	ContextCounter contextcount.ContextCounter

	// MaxRequestBody bounds a request body in bytes, applied before JSON
	// decoding. Zero means DefaultMaxRequestBody.
	MaxRequestBody int64

	// MaxConcurrent bounds global in-flight admission (Target.Client.Invoke/
	// Stream and ContextCounter.CountContext calls). Admission is
	// reject-on-full (a *ConcurrencyLimitExceededError), never queued. Zero
	// means DefaultMaxConcurrent.
	MaxConcurrent int
}

Config configures a Handler. Every field except MaxRequestBody, MaxConcurrent, and ContextCounter is required; New validates the required fields and rejects an invalid Config with a *ConfigError.

type ConfigError ¶

type ConfigError struct {
	Location string
	Reason   string
	Err      error
}

ConfigError reports invalid gateway routing configuration supplied to NewMux or Fixed. It covers every "invalid configuration" case this package rejects at construction time:

• a nil Client on any Target (in Routes, FormatDefaults, or Default);
• a Target.Model that fails model.Model.Validate (Err wraps the underlying *model.ValidationError, reachable via errors.As/errors.Is);
• a duplicate logical route: the same non-empty Target.ID bound, across Routes/FormatDefaults/Default, to two Targets whose Model values are not equal (see NewMux's doc comment for the full rule).

Location identifies where in the configuration the problem was found (e.g. "Routes[anthropic/primary]", "FormatDefaults[openai]", "Default"), for diagnosis; it is not part of any equality contract.

func (e *ConfigError) Error() string

func (e *ConfigError) Unwrap() error

type CountTokensUnavailableError ¶

type CountTokensUnavailableError struct{}

CountTokensUnavailableError reports that the count_tokens route was invoked but the Handler was constructed with a nil Config.ContextCounter.

func (e *CountTokensUnavailableError) Error() string

type ExactResolver ¶

type ExactResolver interface {
	ResolveExact(ctx context.Context, ingress model.APIFormat, requestedModel string) (Target, error)
}

ExactResolver is the optional exact-registration contract consumed by Strict. ResolveExact must return a Target only for the exact (ingress, requestedModel) pair it registered, without applying wildcard or fallback behavior. It must return an *UnknownRouteError for an unregistered pair. Custom resolvers can implement this contract alongside Resolver to opt into strict resolution.

type FixedResolver ¶

type FixedResolver struct {
	// contains filtered or unexported fields
}

FixedResolver is a Resolver that always resolves to the same Target, ignoring the requested ingress format and model alias entirely. Construct with Fixed. FixedFor additionally records an exact harness route for use by Strict; direct Resolve remains wildcard-compatible for both constructors.

func Fixed(client inference.Client, m model.Model) (*FixedResolver, error)

func FixedFor(client inference.Client, m model.Model, ingress model.APIFormat, alias string) (*FixedResolver, error)

func (f *FixedResolver) Resolve(ctx context.Context, ingress model.APIFormat, requestedModel string) (Target, error)

func (f *FixedResolver) ResolveExact(ctx context.Context, ingress model.APIFormat, requestedModel string) (Target, error)

type Handler ¶

type Handler struct {
	// contains filtered or unexported fields
}

Handler is an http.Handler implementing the gateway's request-processing pipeline (see package doc). It is built with New and is safe for concurrent use by multiple goroutines, as any http.Handler must be.

func New(config Config) (*Handler, error)

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request)

type MethodNotAllowedError ¶

type MethodNotAllowedError struct{ Method string }

MethodNotAllowedError reports an HTTP method this gateway does not accept for any configured route. Every inference route this gateway serves is POST-only, so method validation happens once, generically, before codec selection.

func (e *MethodNotAllowedError) Error() string

type Mux ¶

type Mux struct {
	Routes         map[RouteKey]Target
	FormatDefaults map[model.APIFormat]Target
	Default        *Target
}

Mux is an immutable Resolver keyed by ingress API format and the harness-requested model alias, with a per-format default and a single ultimate default target. The zero value is not usable directly as a Resolver in production; construct with NewMux, which validates and defensively copies its input.

Resolution precedence, applied by Resolve:

1. an exact match in Routes for (ingress, requestedModel);
2. FormatDefaults[ingress], if present;
3. Default, if non-nil;
4. otherwise, a *RouteNotFoundError.

There is deliberately no fuzzy, prefix, glob, or provider-name matching.

func NewMux(cfg Mux) (*Mux, error)

func (m *Mux) Resolve(ctx context.Context, ingress model.APIFormat, requestedModel string) (Target, error)

func (m *Mux) ResolveExact(ctx context.Context, ingress model.APIFormat, requestedModel string) (Target, error)

type NoMatchingCodecError ¶

type NoMatchingCodecError struct {
	Method string
	Path   string
}

NoMatchingCodecError reports that no configured ServerCodec recognized a request's method and path.

func (e *NoMatchingCodecError) Error() string

type RequestTooLargeError ¶

type RequestTooLargeError struct{ Limit int64 }

RequestTooLargeError reports a request body exceeding Config.MaxRequestBody.

func (e *RequestTooLargeError) Error() string

type Resolver ¶

type Resolver interface {
	Resolve(ctx context.Context, ingress model.APIFormat, requestedModel string) (Target, error)
}

Resolver maps an ingress API dialect and the harness-requested model alias -- untrusted, caller-supplied input -- to a fully bound Target.

Implementations must not perform unbounded work: Resolve is called on the request path. The built-in implementation is Mux, which performs a bounded set of map lookups with no fuzzy, prefix, glob, or provider-name matching. A resolver that needs that kind of matching implements Resolver directly.

func Strict(inner Resolver) Resolver

type ResponseEncodeError ¶

type ResponseEncodeError struct{ Err error }

ResponseEncodeError reports that a codec's WriteResponse failed after a successful upstream invocation. By the time this can occur the codec has typically already written HTTP headers (and possibly a partial body), so there is usually nothing further this package can do over the wire; the type exists for classification completeness and for callers that inspect it via errors.As for internal diagnostics.

func (e *ResponseEncodeError) Error() string

func (e *ResponseEncodeError) Unwrap() error

type RouteKey ¶

type RouteKey struct {
	Ingress model.APIFormat
	Model   string
}

RouteKey identifies one exact route: the ingress API dialect and the harness-requested model alias, matched verbatim (case-sensitive, no normalization). An empty Model is an ordinary, unreserved alias like any other: it matches only a request that supplies an empty requested-model string for that ingress format. It carries no implicit "format default" meaning -- use FormatDefaults for that, and note that both may be configured for the same ingress format at once without conflict (an exact "" route and a FormatDefaults entry answer different requests: literally empty vs. anything else unmatched).

type RouteNotFoundError ¶

type RouteNotFoundError struct {
	Ingress model.APIFormat
	Model   string
}

RouteNotFoundError reports a Resolve miss: no exact route matched Ingress and Model, no FormatDefaults entry matched Ingress, and no global Default was configured.

func (e *RouteNotFoundError) Error() string

type Server ¶

type Server struct {
	// contains filtered or unexported fields
}

Server runs one loopback-only HTTP listener wrapping a ServerConfig.Handler behind a per-instance, cryptographically random bearer token (see the package doc above for why token generation lives here and not in Config/Authenticate). It is safe for concurrent use by multiple goroutines, including concurrent Start/Binding/Close calls.

Every Server built by NewServer owns an entirely independent listener, *http.Server, token, and lifecycle state -- constructing two Servers, even from the same ServerConfig.Handler value, never shares state between them (see server_test.go's isolation tests). Server never touches http.DefaultServeMux or any other package-level global.

func NewServer(config ServerConfig) (*Server, error)

func (s *Server) Binding() (baseURL, token string, ready bool)

func (s *Server) Close(ctx context.Context) error

func (s *Server) Start(ctx context.Context) error

type ServerConfig ¶

type ServerConfig struct {
	// Handler is served behind Server's own generated-token auth
	// middleware. It is never mutated and may be shared by more than one
	// Server at once (see the package doc above and Server's doc comment).
	Handler http.Handler

	// ShutdownTimeout bounds Close's graceful-drain window. Zero means
	// DefaultShutdownTimeout. Must not be negative.
	ShutdownTimeout time.Duration
}

ServerConfig configures a Server. Handler is required; ShutdownTimeout is optional (zero means DefaultShutdownTimeout).

There is deliberately no address/port field: Server always binds 127.0.0.1 on an ephemeral port (see Start). This is the design's fixed security posture, not an oversight -- a Server is a local trust-boundary primitive, not a configurable network listener.

type ServerStateError ¶

type ServerStateError struct {
	Op    string
	State string
}

ServerStateError reports an invalid Server lifecycle transition -- in practice, Start called while the Server is already running, closing, or closed. Close is never the source of a ServerStateError: it is idempotent by design (see Close's doc comment) rather than erroring on repeated calls.

func (e *ServerStateError) Error() string

type ShutdownTimeoutError ¶

type ShutdownTimeoutError struct {
	Timeout time.Duration
}

ShutdownTimeoutError reports that Close's bounded graceful shutdown did not complete before its effective deadline (the caller's context combined with ServerConfig.ShutdownTimeout -- see Close), so the listener and any still-in-flight connections were force-closed instead. This is one of the design doc's listed local-server error categories ("shutdown timeout"); no earlier task owns it, so it is added here, in the one file that needs it.

func (e *ShutdownTimeoutError) Error() string

type Target ¶

type Target struct {
	ID     string
	Client inference.Client
	Model  model.Model

	// AuthoritativeEffort makes Model.Sampling.Effort authoritative for every
	// request served by this target. The handler will stamp it after ingress
	// decode and before validation/encoding, replacing ingress effort even when
	// the target effort is model.EffortNone. Other override fields remain
	// untouched. False preserves current behavior.
	AuthoritativeEffort bool
}

Target is a fully bound inference destination.

ID is a stable, secret-free diagnostic identity used for logs and metrics only: it is never sent upstream and is not a substitute for Model's own identity (Model.Key). An empty ID asserts no diagnostic identity and is a valid configuration.

Client is already bound to its own credentials and connection policy -- the gateway never handles secrets directly, so Client must never be nil.

Model is the secret-free model descriptor sent with each request routed to this target.

type UnknownRouteError ¶

type UnknownRouteError struct {
	Ingress model.APIFormat
	Alias   string
}

UnknownRouteError reports a strict-resolution miss. It includes only the untrusted request key; resolved targets and their provider/endpoint details are intentionally absent.

func (e *UnknownRouteError) Error() string

func (e *UnknownRouteError) Is(target error) bool

type UnsupportedContentTypeError ¶

type UnsupportedContentTypeError struct{ ContentType string }

UnsupportedContentTypeError reports a request whose Content-Type is not application/json. Every currently bundled dialect is JSON-bodied, so this gateway enforces the constraint once, generically, before codec selection -- ahead of, and instead of, any per-codec Content-Type check a codec's own DecodeRequest might otherwise perform (which would classify as 400, not the 415 this boundary requires).

func (e *UnsupportedContentTypeError) Error() string

type UpstreamInvocationError ¶

type UpstreamInvocationError struct {
	Err              error
	DeadlineExceeded bool
}

UpstreamInvocationError reports that Target.Client.Invoke, Target.Client.Stream, or a configured contextcount.ContextCounter.CountContext returned an error, or that Target.Client.Invoke never returned before the request's context deadline. Its Error() message is deliberately generic -- it never includes the wrapped error's own message -- because that message originates from an upstream provider client the gateway does not control and may contain upstream-secret material (e.g. an echoed Authorization header from a misbehaving transport). Callers that need the underlying detail for internal diagnostics use errors.Unwrap/errors.As, never the HTTP response.

func (e *UpstreamInvocationError) Error() string

func (e *UpstreamInvocationError) Unwrap() error