ext

type Authentication ¶ added in v0.1.71

type Authentication struct {
	PrincipalID     string
	UserPath        string
	Labels          []string
	DashboardAccess bool
	// Method is a short, stable audit identifier such as "oidc" or "saml".
	// Core normalizes safe identifiers and records "extension" when it is empty
	// or invalid.
	Method string
}

Authentication describes an identity established by an extension. PrincipalID must be a stable, non-secret identifier within the authenticator's namespace. UserPath is the existing GoModel authorization and accounting subject; it is deliberately separate because a login identity and a policy hierarchy are not the same thing.

func AuthenticationFromContext(ctx context.Context) (Authentication, bool)

type AuthenticationEvent ¶ added in v0.1.71

type AuthenticationEvent struct {
	Timestamp   time.Time
	Type        string
	Outcome     string
	Method      string
	PrincipalID string
	UserPath    string
	RequestID   string
	ClientIP    string
	HTTPMethod  string
	Path        string
	UserAgent   string
	Reason      string
}

AuthenticationEvent is a security-audit record emitted by an extension authentication flow. Reason must be a short, stable machine identifier; it must not contain provider responses, tokens, claims, or other secrets.

type AuthenticationEventRecorder ¶ added in v0.1.71

type AuthenticationEventRecorder interface {
	RecordAuthenticationEvent(AuthenticationEvent)
}

AuthenticationEventRecorder persists authentication lifecycle events in Core's audit trail. Implementations must be safe for concurrent use and must not block the authentication flow on durable storage I/O.

type AuthenticationEventRecorderAware ¶ added in v0.1.71

type AuthenticationEventRecorderAware interface {
	SetAuthenticationEventRecorder(AuthenticationEventRecorder)
}

AuthenticationEventRecorderAware is optionally implemented by a request authenticator that emits login, rejection, and logout lifecycle events. Core installs the recorder after its audit subsystem has initialized.

type Capability ¶ added in v0.1.77

type Capability string

Capability names an optional core behavior that an extension can enable. Core remains the source of truth for enforcing capabilities; extensions only declare which behaviors their startup configuration has unlocked.

const (
	// CapabilityQuotaTemplates enables per-child user-path templates for
	// budgets and rate limits.
	CapabilityQuotaTemplates Capability = "quota_templates"
	// CapabilityCustomerMarkups enables per-customer markups by user path
	// (pricing.user_path_markups).
	CapabilityCustomerMarkups Capability = "customer_markups"
	// CapabilityInvoiceExport enables the per-customer invoice export of
	// billed usage.
	CapabilityInvoiceExport Capability = "invoice_export"
	// CapabilityBilledBudgets enables budgets enforced on billed cost (provider
	// cost plus markup) instead of provider cost.
	CapabilityBilledBudgets Capability = "billed_budgets"
)

type Endpoint ¶

type Endpoint string

Endpoint identifies an inference endpoint whose raw JSON body can be rewritten before core parses it.

const (
	EndpointChatCompletions Endpoint = "/v1/chat/completions"
	EndpointMessages        Endpoint = "/v1/messages"
	EndpointResponses       Endpoint = "/v1/responses"
)

type HealthChecker ¶ added in v0.1.100

type HealthChecker interface {
	Name() string
	CheckHealth(ctx context.Context) HealthStatus
}

HealthChecker contributes a component to GET /health/ready. Liveness (GET /health) never consults it, so a failing dependency never restarts the process.

Name is the component key in the readiness response; it must be non-empty and must not collide with a core component ("storage", "cache", "models") or another checker. CheckHealth runs on every readiness probe under a short timeout, so it should report recently observed state rather than call slow remote services. An unrecognized status is reported as degraded, as is a check that panics or does not return before the deadline; core runs at most one call per checker at a time. Implementations must be safe for concurrent use.

type HealthStatus ¶ added in v0.1.100

type HealthStatus string

HealthStatus is the state an extension reports for one of its dependencies.

const (
	// HealthOK means the dependency works normally.
	HealthOK HealthStatus = "ok"
	// HealthDegraded means the extension still serves requests, for example
	// from stale cached values. Readiness reports degraded with HTTP 200, so
	// the instance stays in rotation.
	HealthDegraded HealthStatus = "degraded"
	// HealthDown means the extension cannot serve requests. Readiness reports
	// not_ready with HTTP 503, taking the instance out of rotation.
	HealthDown HealthStatus = "down"
)

type Inference ¶ added in v0.1.109

type Inference interface {
	pluginapi.Inference
	pluginapi.Decisions
}

Inference calls models through the gateway on behalf of an extension: chat completions and decision-model questions. Model resolution, access policies, failover, usage accounting, and audit apply as for a plugin's internal calls. Calls carry origin "plugin", skip guardrails, and are scoped to "<request user path>/extensions/<extension name>", so usage and the audit trail attribute them to the extension; '/' and ':' in the name become '-'. A rewriter runs before its request is admitted, so every call is admitted on its own against that path's rate limits and budgets.

type InferenceAware ¶ added in v0.1.109

type InferenceAware interface {
	SetInference(Inference)
}

InferenceAware is optionally implemented by a request rewriter that calls models while it rewrites, such as a router asking a decision model how hard a request is. Core installs the handle once the gateway's internal executors are built, before serving, and installs a fresh one for every configuration generation; requests of the previous generation may still be running, so implementations must swap it atomically. Until a handle is installed, a rewriter must treat inference as unavailable.

type Input ¶

type Input struct {
	Endpoint Endpoint
	// Body is the raw JSON request body, already bounded by the server's
	// body-size limit.
	Body []byte
	// Header is a clone of the inbound request headers with credential
	// values (Authorization, cookies, API keys, ...) redacted. Rewriters
	// run post-auth; use UserPath for identity.
	Header http.Header
	// UserPath is the canonical authenticated user path, when present.
	UserPath string
	// RequestID is the request correlation ID (X-Request-ID).
	RequestID string
	// SessionID is the detected client session, when present, already scoped
	// by the effective user path. Session detection runs before rewriters, so
	// a rewriter can keep its decisions stable across a conversation.
	SessionID string
}

Input is the raw inbound request handed to a rewriter before core parses it. Body and Header are snapshots owned by the middleware; rewriters must treat them as read-only and return new values in Result when changing anything.

type License ¶ added in v0.1.108

type License struct {
	// Badge is the short label shown after the GoModel name, e.g. "PRO".
	// An empty Badge means no license is shown.
	Badge string
	// Trial marks an evaluation license, whose days are always counted down.
	Trial bool
	// ExpiresAt is when the license term ends. Zero shows no countdown.
	ExpiresAt time.Time
	// ValidUntil is when the license stops working, after any grace period.
	// Zero, or a time before ExpiresAt, means there is no grace period.
	ValidUntil time.Time
	// NoticeWindow is how long before ExpiresAt the dashboard starts counting
	// down a license that is not a trial. Zero counts it down only once the
	// term has ended.
	NoticeWindow time.Duration
	// WarningWindow is how long before ExpiresAt the countdown switches to
	// the warning color. Zero never shows the warning color before the end.
	WarningWindow time.Duration
}

License describes the commercial license a distribution runs under, for display only: the dashboard shows Badge after the GoModel name and counts down the days of a trial, or of a license close to its end. Core enforces nothing here and decides none of the policy: validating the license, and when its end is worth pointing out, stay with the distribution.

type PluginFactory ¶ added in v0.1.90

type PluginFactory func() pluginapi.Plugin

PluginFactory builds a fresh plugin value. Every configured instance of the plugin type gets its own value, so plugins can keep per-instance state.

type ProxyRequest ¶ added in v0.1.96

type ProxyRequest struct {
	Provider     string
	ProviderType string
	URL          *url.URL
}

ProxyRequest describes one upstream request about to leave the gateway, so a ProxySelector can choose its egress. Provider is the configured provider instance name ("openai-eu"), ProviderType its adapter ("openai"), and URL the upstream endpoint being called.

type ProxySelector ¶ added in v0.1.96

type ProxySelector interface {
	Name() string
	SelectProxy(req ProxyRequest) (*url.URL, error)
}

ProxySelector chooses the outbound proxy for upstream provider requests. Core consults it on every request of a provider that has no explicit `proxy_url` of its own; a provider's own setting always wins. Returning a nil URL and nil error means "no opinion", and core then falls back to the process-wide HTTP_PROXY / HTTPS_PROXY / NO_PROXY environment. Returning an error fails the request before it is sent.

Selection runs on the request path, so implementations must be safe for concurrent use and return quickly. The returned URL may carry credentials; core never logs it.

type Registry ¶

type Registry struct {
	// contains filtered or unexported fields
}

Registry collects extensions to be consumed by the gateway at startup. Register everything before the server is constructed (before run.Run or app.New); core snapshots each registration list during initialization.

func (r *Registry) AddPublicPaths(paths ...string)

func (r *Registry) AdminRoutes() []func(*echo.Group)

func (r *Registry) Authenticators() []RequestAuthenticator

func (r *Registry) EnableCapability(capability Capability)

func (r *Registry) HasCapability(capability Capability) bool

func (r *Registry) HealthCheckers() []HealthChecker

func (r *Registry) License() (License, bool)

func (r *Registry) LogHandlerWrappers() []func(slog.Handler) slog.Handler

func (r *Registry) Middleware() []echo.MiddlewareFunc

func (r *Registry) OuterMiddleware() []echo.MiddlewareFunc

func (r *Registry) Plugins() []PluginFactory

func (r *Registry) ProxySelector() ProxySelector

func (r *Registry) PublicPaths() []string

func (r *Registry) RegisterAdminRoutes(fn func(g *echo.Group))

func (r *Registry) RegisterAuthenticator(authenticator RequestAuthenticator)

func (r *Registry) RegisterHealthChecker(checker HealthChecker)

func (r *Registry) RegisterPlugin(factory PluginFactory)

func (r *Registry) RegisterProxySelector(sel ProxySelector)

func (r *Registry) RegisterRewriter(rw RequestRewriter)

func (r *Registry) RegisterRouteSelector(sel RouteSelector)

func (r *Registry) RegisterRoutes(fn func(e *echo.Echo))

func (r *Registry) RegisterSetting(setting RuntimeSetting)

func (r *Registry) Rewriters() []RequestRewriter

func (r *Registry) RouteSelector() RouteSelector

func (r *Registry) Routes() []func(*echo.Echo)

func (r *Registry) SetLicense(license License)

func (r *Registry) SetUpdateCheckLicense(license UpdateCheckLicense)

func (r *Registry) Settings() []RuntimeSetting

func (r *Registry) UpdateCheckLicense() UpdateCheckLicense

func (r *Registry) UseMiddleware(m echo.MiddlewareFunc)

func (r *Registry) UseOuterMiddleware(m echo.MiddlewareFunc)

func (r *Registry) WrapLogHandler(wrap func(slog.Handler) slog.Handler)

type RejectionError ¶

type RejectionError struct {
	Status  int
	Code    string
	Message string
}

RejectionError rejects the request with a client-visible status code and machine-readable error code, rendered in the endpoint's native error dialect (OpenAI or Anthropic envelope).

func (e *RejectionError) Error() string

type RequestAuthenticator ¶ added in v0.1.71

type RequestAuthenticator interface {
	Name() string
	AuthenticateRequest(ctx context.Context, request *http.Request) (*Authentication, error)
}

RequestAuthenticator authenticates requests using a mechanism other than the core bearer-token authenticators (for example an OIDC browser session). A nil result with a nil error means the mechanism does not apply to the request. Implementations must be safe for concurrent use.

type RequestRewriter ¶

type RequestRewriter interface {
	Name() string
	Rewrite(ctx context.Context, in Input) (*Result, error)
}

RequestRewriter rewrites raw JSON request bodies at ingress, after authentication and before model resolution, so body changes (including the "model" field) affect routing, failover, guardrails, budgets, and caching.

Rewriters run once per request in registration order; each receives the previous rewriter's output. Implementations must be safe for concurrent use. Errors fail the request (fail-closed): return a *RejectionError for a client-visible status, any other error maps to HTTP 500.

type ResponseFeedbackFilter ¶ added in v0.1.72

type ResponseFeedbackFilter interface {
	WantsResponseFeedback(in Input, result *Result) bool
}

ResponseFeedbackFilter lets an observer control registration per request. Core calls it after Rewrite and registers the observer only when it returns true. Observers without this optional interface receive feedback for every successful rewritten request. A panic is isolated and treated as false so this optional hook cannot abort inference.

type ResponseFeedbackObserver ¶ added in v0.1.72

type ResponseFeedbackObserver interface {
	ObserveResponse(
		ctx context.Context,
		requestID string,
		endpoint Endpoint,
		sessionID string,
		model string,
		providerType string,
		providerName string,
		inputTokens int,
		cachedInputTokens int,
		cacheWriteInputTokens int,
		usageObserved bool,
	)
}

ResponseFeedbackObserver receives content-free feedback after a rewritten request completes successfully at a provider. For ordinary responses the callback runs after the provider returns; for streaming responses it runs after the stream closes. Failed requests do not produce feedback.

It is an optional companion to RequestRewriter: core detects implementations structurally. usageObserved distinguishes a provider-confirmed zero from a provider or stream that returned no usage breakdown. Implementations can be called concurrently, must therefore be concurrency-safe, and must return promptly so they do not delay completion of the client request.

The flat signature intentionally uses only long-standing extension types so extensions can implement the hook while supporting older core releases; older cores simply never call it.

type Result ¶

type Result struct {
	Body []byte
	// ResponseHeader entries are merged into the HTTP response so rewriters
	// can annotate what they did (for example X-GoModel-Pro-Tokens-Saved).
	ResponseHeader http.Header
	// Detail optionally carries a JSON-serializable summary of what the
	// rewriter changed. It is recorded in the audit trail's request-revision
	// chain and never sent upstream; it must never contain secrets or
	// request credentials.
	Detail any
	// TokensSaved is the rewriter's estimate of prompt tokens its body
	// change removed from the request. When positive and the rewritten body
	// is applied, core adds it to the request's usage record together with
	// the input cost those tokens would have incurred, and the dashboard
	// aggregates both as rewrite savings. Leave zero when the rewrite does
	// not shrink the prompt.
	TokensSaved int
}

Result carries a rewritten body and response-header annotations. A nil Result (or nil Body) means the request is unchanged.

type RouteCandidate ¶ added in v0.1.65

type RouteCandidate struct {
	// Provider is the configured provider name (e.g. "openai", "azure-eu").
	Provider string
	// Model is the provider-native model ID (e.g. "gpt-4o").
	Model string
	// Qualified is "provider/model", the stable key selection answers with.
	Qualified string
	// Weight is the operator-configured target weight; 0 means unset (treat
	// as 1).
	Weight        float64
	InputPerMtok  *float64
	OutputPerMtok *float64
}

RouteCandidate is one currently viable target of a load-balanced virtual model, offered to a RouteSelector. Pricing comes from the model registry and is per million tokens; nil means the registry has no price for the target.

type RouteOutcome ¶ added in v0.1.65

type RouteOutcome struct {
	RouteTarget
	// Source is the virtual model originally addressed when this attempt was
	// selected through one. SessionID is the detected client session. Together
	// they let selectors update cache affinity only after a successful attempt.
	Source    string
	SessionID string
	// Endpoint is the upstream API endpoint (e.g. "/chat/completions").
	Endpoint string
	// StatusCode is the final upstream HTTP status; 0 on a network error.
	StatusCode int
	// Duration is the call duration, including any transport-level retries.
	// For streaming requests it measures time to stream establishment, not
	// the full stream lifetime.
	Duration time.Duration
	Stream   bool
	// Err is the client-layer error, nil on success.
	Err error
}

RouteOutcome describes one completed upstream call. Every call is reported — primaries and failover attempts alike — so selectors learn from traffic they did not steer. Transport-level retries inside the provider client are aggregated into their call's single outcome: StatusCode and Err reflect the final result, and Duration spans the whole call including retry backoff, so a target that only succeeds after internal retries still scores slower than a target that succeeds at once.

type RouteRequest ¶ added in v0.1.65

type RouteRequest struct {
	// Source is the virtual model name the request addressed.
	Source string
	// SessionID is the detected client session, when present.
	SessionID string
	// SessionTarget is the qualified target that already served this
	// session, when the redirect keeps session affinity and a pin exists.
	// It is empty on a session's first request, when affinity is off, and
	// when the pinned target has left the candidate set.
	//
	// Core does not enforce the pin for the adaptive strategy: it hands it
	// over here and records whatever the selector answers, so a selector
	// that knows a target is failing can move the session where core —
	// which only tests candidate membership — would hold it for the pin's
	// whole lifetime. Honour it while the target is healthy; the operator
	// asked for affinity, and moving a session costs prompt-cache warmth.
	SessionTarget string
	Candidates    []RouteCandidate
}

RouteRequest asks a RouteSelector to pick one target for a request routed through a load-balanced virtual model. Candidates are the targets that are catalog-supported and have rate-limit capacity right now, in declared order; there are always at least two (single-candidate picks bypass the selector so an alias behaves identically with and without one).

type RouteSelector ¶ added in v0.1.65

type RouteSelector interface {
	Name() string
	Select(req RouteRequest) (qualified string, ok bool)
	OnAttemptStart(target RouteTarget)
	OnAttemptEnd(outcome RouteOutcome)
}

RouteSelector steers load balancing for virtual models using the "adaptive" strategy. Core consults the selector to pick among currently viable targets and, for a session-affine redirect, to decide whether the session stays on its pinned target (see RouteRequest.SessionTarget); rate-limit capacity, failover chains, and retries remain core's responsibility.

Select must be fast and must not block: it runs on the request path before the upstream call. Implementations must be safe for concurrent use. A (_, false) answer — and any answer naming a model outside Candidates — falls back to weighted round robin, so selectors fail open by declining.

OnAttemptStart and OnAttemptEnd observe the upstream client lifecycle, once per upstream call (transport-level retries within a call are aggregated — see RouteOutcome). For streaming requests OnAttemptEnd fires when the stream is established, not when it closes.

type RouteTarget ¶ added in v0.1.65

type RouteTarget struct {
	Provider string
	Model    string
}

RouteTarget identifies a provider/model pair as seen by the upstream client layer.

func (t RouteTarget) Qualified() string

type RuntimeSetting ¶ added in v0.1.69

type RuntimeSetting interface {
	Descriptor() SettingDescriptor
	Apply(value string) error
}

RuntimeSetting is a mutable, deployment-wide extension setting. Core owns persistence and the admin API; the extension owns validation and applying the value to its live runtime state. Apply receives only a value advertised in Descriptor.Options. Implementations must be safe for concurrent use.

type SettingDescriptor ¶ added in v0.1.69

type SettingDescriptor struct {
	Key         string          `json:"key"`
	Label       string          `json:"label"`
	Description string          `json:"description,omitempty"`
	Value       string          `json:"value"`
	Locked      bool            `json:"locked"`
	ManagedBy   string          `json:"managed_by,omitempty"`
	Options     []SettingOption `json:"options"`
}

SettingDescriptor describes one deployment-wide extension setting. Locked settings are controlled by an environment variable and remain visible, but cannot be changed through the admin API. Options must list every accepted value for an unlocked setting, including Value; registration fails when that contract is not met.

type SettingOption ¶ added in v0.1.69

type SettingOption struct {
	Value       string `json:"value"`
	Label       string `json:"label"`
	Description string `json:"description,omitempty"`
}

SettingOption is one allowed value for a dashboard-editable extension setting. Label and Description are safe to expose in the admin UI.

type SharedEntry ¶ added in v0.1.109

type SharedEntry struct {
	Key   string
	Value []byte
	// ExpiresAt is the entry's expiry at second precision; zero means it
	// never expires.
	ExpiresAt time.Time
}

SharedEntry is one live entry of a shared store namespace.

type SharedStore ¶ added in v0.1.109

type SharedStore interface {
	// Put stores value under key. Only the zero time.Time keeps the entry
	// until it is deleted; any other expiresAt that is not in the future,
	// the unix epoch included, removes the entry.
	Put(ctx context.Context, namespace, key string, value []byte, expiresAt time.Time) error
	// Get returns the live value under key, and whether there was one.
	Get(ctx context.Context, namespace, key string) ([]byte, bool, error)
	// Delete removes key. Deleting a missing key is not an error.
	Delete(ctx context.Context, namespace, key string) error
	// List returns every live entry of namespace, in no particular order.
	List(ctx context.Context, namespace string) ([]SharedEntry, error)
}

SharedStore is a small key/value store that extensions share across gateway replicas. Core keeps it in the configured storage backend, so every replica on the same PostgreSQL or MongoDB database sees the same entries and they survive restarts. With SQLite it is durable but local to the instance that owns the file.

It is meant for small coordination state (revocation lists, leases, markers), not bulk data: List reads a whole namespace, so keep namespaces small. Name namespaces after the extension and its purpose (for example "sso.revoked") so extensions do not collide.

Entries are invisible once their expiry passes and are swept periodically. A Put replaces the entry; a Put with a past expiry therefore removes it. Calls go to the database: keep them off the hot request path or bound them with a context deadline. Implementations are safe for concurrent use.

type SharedStoreAware ¶ added in v0.1.109

type SharedStoreAware interface {
	SetSharedStore(SharedStore)
}

SharedStoreAware is optionally implemented by a registered extension component: an authenticator, request rewriter, runtime setting, health checker, route selector, or proxy selector. Core calls SetSharedStore once the configuration generation that owns the store has built successfully, and again with the new store after every reload, so an implementation must accept repeated calls and always use the most recent store. Until the first call there is no store; extensions must keep working without one.

type UpdateCheckLicense ¶ added in v0.1.108

type UpdateCheckLicense struct {
	// Kind is the license the gateway runs on: "paid", "trial", "beta" (a
	// build-time license every copy of a beta build shares) or "none".
	Kind string
	// Account identifies the license holder: 32 lowercase hex characters,
	// the same for every deployment one holder runs, or empty. It must be a
	// keyed digest that only the license issuer can match to a holder, never
	// the holder's name.
	Account string
}

UpdateCheckLicense is what a licensed distribution adds to the daily update check, so the release host can count license holders rather than only deployments: one organization often runs several gateways, each with its own database and therefore its own install id.

Core sends both values only when they have exactly the shapes documented below and drops them otherwise, so an extension cannot use the check to carry anything else.