plugins

package
v0.1.95 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 20, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Overview

Package plugins hosts the runtime side of GoModel's plugin system: the catalog of plugin types, configured instances, per-phase chains and the Host implementation handed to plugins. It maps between core request types and the pluginapi contract through the exchange subpackage.

Index

Constants

View Source 
const (
	CodePluginFailure = "plugin_failure"
	CodeBlocked       = "guardrail_blocked"
	CodeWarn          = "guardrail_warning"
)

Codes used when a plugin does not set its own.

View Source 
const (
	// HealthOK is a healthy instance, and every instance of a plugin that
	// does not implement pluginapi.HealthChecker.
	HealthOK = "ok"
	// HealthDegraded is an instance whose last probe returned an error,
	// panicked, or ran past its deadline.
	HealthDegraded = "degraded"
)

Health statuses of an instance.

View Source 
const GuardrailHeader = "X-GoModel-Guardrail"

GuardrailHeader is the response header carrying warn decisions.

View Source 
const PluginHTTPTimeout = 60 * time.Second

PluginHTTPTimeout is the request timeout of DefaultHTTPClient. A plugin call to an external service is normally bounded by the instance timeout through the hook context; this is the backstop when none is set.

View Source 
const SecretMask = "********"

SecretMask is what a stored secret is rendered as in admin responses. A client sending it back unchanged keeps the stored value; sending "" clears it.

Variables

View Source 
var DefaultHTTPClient = sync.OnceValue(func() *http.Client {
	cfg := httpclient.DefaultConfig()
	cfg.Timeout = PluginHTTPTimeout
	cfg.ResponseHeaderTimeout = PluginHTTPTimeout
	return httpclient.NewHTTPClient(&cfg)
})

DefaultHTTPClient returns the process-wide client plugins use for external calls when HostDeps.HTTP is nil: the gateway's transport settings (proxy from the environment, connection pool, dial and TLS timeouts) with PluginHTTPTimeout as the request and response-header timeout.

View Source 
var ErrAbandoned = errors.New("plugin call abandoned")

ErrAbandoned marks a hook call the runtime stopped waiting for because the instance timeout expired or the request ended. The hook may still be running: it received a cancelled context and is expected to return soon, but nothing it does from then on reaches the request.

View Source 
var ErrHistoryUnavailable = errors.New("plugins: history is not available in this host version")

ErrHistoryUnavailable is returned by History in this host version.

View Source 
var ErrInferenceUnavailable = errors.New("plugins: internal inference is not available")

ErrInferenceUnavailable is returned when no ChatCompleter is wired.

View Source 
var ErrInstanceClosed = errors.New("plugins: instance is closed")

ErrInstanceClosed is returned by Call for an instance that was closed.

View Source 
var ErrRouteResolverClosed = errors.New("plugins: routing-strategy resolver is closed")

ErrRouteResolverClosed is returned by Strategy after Close.

View Source 
var PhaseKinds = []pluginapi.Kind{pluginapi.KindPrompt, pluginapi.KindResponse, pluginapi.KindStream}

PhaseKinds are the hook kinds a workflow step can reference.

Functions

func Abandoned 

func Abandoned(err error) bool

Abandoned reports whether err, from a chain run or a single hook call, stems from an abandoned call. The hook may still be writing the exchange it ran on, so the caller must not read that exchange (its Prompt, Response, Values or Headers) once the run has returned this error.

func BlockError 

func BlockError(d pluginapi.Decision, defaultStatus int) *core.GatewayError

BlockError renders a block decision as the gateway error the client sees. defaultStatus is used when the decision has no status (400 for request phases, 502 for response phases) or one outside 400-599: a plugin may hand back any integer, and net/http panics on a status it cannot write.

func Call 

func Call[T any](ctx context.Context, inst *Instance, fn func(context.Context) (T, error)) (T, error)

Call runs fn under the instance's timeout with panic recovery. It returns when fn returns or when ctx ends, whichever comes first, so a hook that ignores its context bounds neither request latency nor shutdown. A call that returns after its deadline is reported as abandoned as well.

func ComputeChainHash 

func ComputeChainHash(rules []RuleDescriptor) string

ComputeChainHash computes a deterministic hash for a set of chain members. Each member is represented as "name:type:order:mode:content_hash"; the seeds are sorted, joined and passed through SHA-256. The result is carried on the request context (core.WithGuardrailsHash) and consumed by the response cache as an opaque key component, so configuration changes invalidate cached completions. Empty is reserved for "no chain".

func ConfigHash 

func ConfigHash(raw json.RawMessage) string

ConfigHash returns a short stable digest of a config for chain hashing.

func DefaultBlockStatus 

func DefaultBlockStatus(phase pluginapi.Kind) int

DefaultBlockStatus returns the block status a phase uses when the decision sets none.

func FailureError 

func FailureError(err error) *core.GatewayError

FailureError renders a fail-closed plugin error: HTTP 500 with code plugin_failure. The instance name stays out of the client message.

func ImplementedKinds 

func ImplementedKinds(p pluginapi.Plugin) []pluginapi.Kind

ImplementedKinds reports the hook interfaces p satisfies.

func IsPhaseKind 

func IsPhaseKind(kind pluginapi.Kind) bool

IsPhaseKind reports whether kind is a workflow phase.

func MergeDecision 

func MergeDecision(current, next pluginapi.Decision) pluginapi.Decision

MergeDecision returns the more severe of two decisions; on a tie the first one wins so step order decides.

func MergeSecrets 

func MergeSecrets(schema []pluginapi.Field, incoming, stored json.RawMessage) json.RawMessage

MergeSecrets restores stored secret values where incoming carries the mask. An empty incoming secret clears the stored value.

func MetaFromContext 

func MetaFromContext(ctx context.Context, workflow *core.Workflow) pluginapi.Meta

MetaFromContext snapshots the request facts a hook may read from the request context and the resolved workflow. Attempts are not read here: callers of response phases add them with WithAttempts.

func NewHost 

func NewHost(deps HostDeps, info HostInfo) pluginapi.Host

NewHost builds the Host for one instance. The logger is tagged with the plugin and instance names.

func NormalizeDecision 

func NormalizeDecision(d pluginapi.Decision) pluginapi.Decision

NormalizeDecision maps the zero action to allow.

func PromptEditCaptureEnabled 

func PromptEditCaptureEnabled(ctx context.Context) bool

PromptEditCaptureEnabled reports whether ctx asks for prompt edits to be kept.

func RedactSecrets 

func RedactSecrets(schema []pluginapi.Field, raw json.RawMessage) json.RawMessage

RedactSecrets replaces every non-empty secret value with SecretMask.

func RoutePluginNames 

func RoutePluginNames(catalog *Catalog) []string

RoutePluginNames lists the usable catalog entries implementing the route hook, sorted by name.

func SchemaDefaults 

func SchemaDefaults(schema []pluginapi.Field) json.RawMessage

SchemaDefaults renders the default config object of the instance-scoped fields: each field's Default, or the empty value of its input kind.

func Severity 

func Severity(action pluginapi.Action) int

Severity ranks decisions so concurrent results merge deterministically: block > respond > warn > allow.

func ValidateConfig 

func ValidateConfig(schema []pluginapi.Field, raw json.RawMessage, scope pluginapi.FieldScope) (json.RawMessage, error)

ValidateConfig checks raw against the schema fields of the given scope and returns the canonical config: defaults applied, values coerced to the field's type, keys sorted. Unknown keys are rejected unless the schema has no field in that scope at all, in which case the config passes through.

func WarnHeaderValue 

func WarnHeaderValue(d pluginapi.Decision) string

WarnHeaderValue renders the X-GoModel-Guardrail header for a warn decision.

func WithAttempts 

func WithAttempts(meta pluginapi.Meta, attempts []Attempt) pluginapi.Meta

WithAttempts returns meta with the provider attempts filled in.

func WithPromptEditCapture 

func WithPromptEditCapture(ctx context.Context) context.Context

WithPromptEditCapture marks ctx as wanting a PromptEdit kept for every prompt edit (see RequestState.PromptEdits). Off by default, so batch items and requests without audit capture pay nothing for it.

Types

type Attempt 

type Attempt struct {
	Seq          int
	Kind         string
	ProviderType string
	ProviderName string
	Model        string
	StatusCode   int
	Success      bool
	ErrorCode    string
	Duration     time.Duration
}

Attempt mirrors one provider attempt without importing the gateway package.

type Catalog 

type Catalog struct {
	// contains filtered or unexported fields
}

Catalog is the process-wide set of plugin types.

func NewCatalog 

func NewCatalog() *Catalog

NewCatalog returns an empty catalog.

func (*Catalog) Entries 

func (c *Catalog) Entries() []Entry

Entries returns every entry, failed loads included, sorted by name.

func (*Catalog) Len 

func (c *Catalog) Len() int

Len returns the number of usable plugin types.

func (*Catalog) Lookup 

func (c *Catalog) Lookup(name string) (Entry, bool)

Lookup returns the usable entry registered under name.

func (*Catalog) Names 

func (c *Catalog) Names() []string

Names returns the usable plugin names sorted.

func (*Catalog) Register 

func (c *Catalog) Register(factory Factory, source Source, options ...RegisterOptions) (err error)

Register probes factory once, validates its manifest, and adds it under the manifest name. Every Kind the manifest declares must be backed by the matching hook interface on the probed value.

func (*Catalog) RegisterFailed 

func (c *Catalog) RegisterFailed(name string, source Source, err error)

RegisterFailed records a plugin that could not be loaded so the admin API can report it. A later successful Register under the same name replaces it.

type Chain 

type Chain struct {
	Phase pluginapi.Kind
	Steps []Step
	Hash  string
}

Chain is the ordered set of instances of one phase.

func BuildChain 

func BuildChain(phase pluginapi.Kind, refs []Ref) (*Chain, error)

BuildChain groups refs by step and validates them: every instance must implement the phase hook and a step may hold at most one mutating instance (non-mutating instances of a step run concurrently).

func (*Chain) Acquire 

func (c *Chain) Acquire()

Acquire marks every instance of the chain held; pair it with Release.

func (*Chain) Empty 

func (c *Chain) Empty() bool

Empty reports whether the chain has no instances.

func (*Chain) Instances 

func (c *Chain) Instances() []*Instance

Instances lists the instances in step order.

func (*Chain) Len 

func (c *Chain) Len() int

Len returns the number of instances.

func (*Chain) Release 

func (c *Chain) Release()

Release drops the hold taken by Acquire.

func (*Chain) RunPrompt 

func (c *Chain) RunPrompt(ctx context.Context, x *pluginapi.Exchange) (Outcome, error)

RunPrompt runs the chain's OnPrompt hooks over x.

func (*Chain) RunPromptObserved 

func (c *Chain) RunPromptObserved(ctx context.Context, x *pluginapi.Exchange, observe EditObserver) (Outcome, error)

RunPromptObserved is RunPrompt with an observer of each step's edit.

func (*Chain) RunResponse 

func (c *Chain) RunResponse(ctx context.Context, x *pluginapi.Exchange) (Outcome, error)

RunResponse runs the chain's OnResponse hooks over x.

func (*Chain) RunStreamEnd 

func (c *Chain) RunStreamEnd(ctx context.Context, x *pluginapi.Exchange) (Outcome, error)

RunStreamEnd runs the chain's OnStreamEnd hooks over x.

func (*Chain) StepsOf 

func (c *Chain) StepsOf() []Step

StepsOf returns the steps of a possibly nil chain.

type Chains 

type Chains struct {
	Prompt   *Chain
	Response *Chain
	Stream   *Chain
}

Chains holds the compiled chain of every phase of one workflow.

func (*Chains) Acquire 

func (c *Chains) Acquire()

Acquire marks every instance of every phase held, once per instance even when it serves several phases; pair it with Release. A compiled workflow holds its chains while it is live and a request while it runs them, so a replaced instance is not closed underneath either.

func (*Chains) CacheHash 

func (c *Chains) CacheHash() string

CacheHash returns the hash the response cache keys on: the prompt chain hash alone when no response or stream chain exists (so keys of existing prompt-only workflows are unchanged), otherwise a digest of every phase hash, because cached bodies were produced by the response and stream chains too.

func (*Chains) Empty 

func (c *Chains) Empty() bool

Empty reports whether no phase has a chain.

func (*Chains) Hashes 

func (c *Chains) Hashes() map[string]string

Hashes returns the non-empty chain hashes keyed by phase.

func (*Chains) PromptHash 

func (c *Chains) PromptHash() string

PromptHash returns the prompt chain hash.

func (*Chains) Release 

func (c *Chains) Release()

Release drops the hold taken by Acquire.

type ChatCompleter 

type ChatCompleter interface {
	ChatCompletion(ctx context.Context, req *core.ChatRequest) (*core.ChatResponse, error)
}

ChatCompleter runs a gateway-internal chat completion (routing, usage and budgets apply). The server's InternalChatCompletionExecutor implements it.

type DecisionRecord 

type DecisionRecord struct {
	Phase    pluginapi.Kind
	Instance string
	// Type is the plugin type of the instance, when known.
	Type string
	// Step is the chain step the instance ran in, when known.
	Step     int
	Decision pluginapi.Decision
	Duration time.Duration
	Err      error
	// FailedClosed reports that Err ended the request; a failure without it
	// was a fail-open one and the chain carried on.
	FailedClosed bool
	// Edited reports that the instance's step changed the request or
	// response.
	Edited bool
	// Replaced and Dropped count the stream events an in-flight stream
	// instance rewrote or withheld.
	Replaced int
	Dropped  int
	// BytesBefore and BytesAfter are the encoded request sizes around the
	// edit, when known.
	BytesBefore int
	BytesAfter  int
}

DecisionRecord is one recorded plugin decision.

func DecisionRecordsOf added in v0.1.91 

func DecisionRecordsOf(phase pluginapi.Kind, outcome Outcome, runErr error) []DecisionRecord

DecisionRecordsOf converts one chain run's records into decision records of phase, marking the instance whose failure ended the run (runErr, when a *PluginError) as failed closed.

type EditObserver 

type EditObserver func(instance string, x *pluginapi.Exchange)

EditObserver is told, during a chain run, that instance edited the prompt or response, right after its step completed and before the next step starts, so x is quiet while the observer reads it. An observer that needs the state later must copy it (see pluginapi.Prompt.Clone).

type Entry 

type Entry struct {
	Name     string
	Manifest pluginapi.Manifest
	// Kinds are the hooks the plugin declares and actually implements.
	Kinds   []pluginapi.Kind
	Source  Source
	Factory Factory
	// SingleInstance marks a type that can back only one configured instance
	// (a shared object exporting a plugin variable rather than a constructor).
	SingleInstance bool
	// Health is "ok" for a usable entry and "error" for one that failed to
	// load; Err carries the failure.
	Health string
	Err    error
}

Entry is one plugin type known to the catalog.

func (Entry) HasKind 

func (e Entry) HasKind(kind pluginapi.Kind) bool

HasKind reports whether the entry implements the hook.

type Factory 

type Factory func() pluginapi.Plugin

Factory builds a fresh plugin value. Every configured instance gets its own value so plugins can keep per-instance state.

type FailMode 

type FailMode string

FailMode says what happens when an instance errors, panics, or times out. 

const (
	// FailClosed rejects the request with HTTP 500 and code "plugin_failure".
	FailClosed FailMode = "closed"
	// FailOpen logs the failure and continues as if the instance allowed.
	FailOpen FailMode = "open"
)

func DefaultFailMode 

func DefaultFailMode(phase pluginapi.Kind) FailMode

DefaultFailMode is the fail mode used when the instance does not set one: closed for content phases, open for everything else.

func ParseFailMode 

func ParseFailMode(raw string) (FailMode, error)

ParseFailMode normalizes a configured fail mode. Empty selects the phase default.

type Health added in v0.1.91 

type Health struct {
	Status string
	// Error is the probe's error text when Status is HealthDegraded.
	Error string
	// CheckedAt is when the probe ran; zero for a plugin that is not a
	// HealthChecker.
	CheckedAt time.Time
}

Health is the outcome of an instance's last health probe.

func (Health) Degraded added in v0.1.91 

func (h Health) Degraded() bool

Degraded reports whether the last probe failed.

type HostDeps 

type HostDeps struct {
	Logger *slog.Logger
	// Chat may be nil; Inference().Complete then fails with a clear error.
	Chat ChatCompleter
	// Metrics may be nil; a no-op sink is used.
	Metrics MetricsSink
	// HTTP is the client handed to plugins for external calls. Nil selects
	// [DefaultHTTPClient].
	HTTP *http.Client
}

HostDeps are the gateway services a Host exposes to plugins.

type HostInfo 

type HostInfo struct {
	PluginName   string
	InstanceName string
	// UserPath, when set, scopes the instance's internal inference to that
	// user path instead of the current request's path.
	UserPath string
}

HostInfo identifies the instance a Host serves.

type Instance 

type Instance struct {
	Name     string
	Type     string
	Manifest pluginapi.Manifest
	Kinds    []pluginapi.Kind
	Plugin   pluginapi.Plugin
	FailMode FailMode
	Timeout  time.Duration
	// ConfigHash digests the validated config for chain hashing.
	ConfigHash string
	// contains filtered or unexported fields
}

Instance is one configured, initialized plugin.

func NewInstance 

func NewInstance(ctx context.Context, entry Entry, spec InstanceSpec, host pluginapi.Host) (inst *Instance, err error)

NewInstance validates spec.Config against the entry's schema, builds a fresh plugin value and initializes it. Init runs with a 10s timeout and panic recovery.

func (*Instance) Acquire 

func (i *Instance) Acquire()

Acquire records a holder of the instance; see Release.

func (*Instance) CheckHealth added in v0.1.91 

func (i *Instance) CheckHealth(ctx context.Context) Health

CheckHealth probes the instance when its plugin is a HealthChecker, records the outcome, and returns it. The probe runs with panic recovery under healthTimeout (and the instance timeout, when shorter); one that does not return in time counts as degraded. A plugin that is not a HealthChecker is reported ok without being called.

func (*Instance) Checks added in v0.1.91 

func (i *Instance) Checks() bool

Checks reports whether the plugin implements pluginapi.HealthChecker.

func (*Instance) Close 

func (i *Instance) Close(ctx context.Context) (err error)

Close releases the plugin's resources, recovering panics. It runs once; later calls return nil, and later hook calls fail with ErrInstanceClosed.

func (*Instance) Closed 

func (i *Instance) Closed() bool

Closed reports whether Close ran.

func (*Instance) EditsContent added in v0.1.92 

func (i *Instance) EditsContent() bool

EditsContent reports whether the configured instance edits content. A mutating plugin may be configured only to flag or block — presidio with action "warn", string_replace with on_match "block" — and then leaves the request as it is; such a plugin says so through pluginapi.ContentEditor.

func (*Instance) EffectiveFailMode 

func (i *Instance) EffectiveFailMode(phase pluginapi.Kind) FailMode

EffectiveFailMode resolves the fail mode for a phase.

func (*Instance) FailsOpen 

func (i *Instance) FailsOpen(phase pluginapi.Kind, err error, shared bool) bool

FailsOpen reports whether err from a hook of phase is absorbed under the instance's fail mode. An abandoned call (ErrAbandoned) never fails open when shared says the hook ran on the request's own exchange: it may still be editing it, so the request cannot safely continue.

func (*Instance) HasKind 

func (i *Instance) HasKind(kind pluginapi.Kind) bool

HasKind reports whether the instance implements the hook.

func (*Instance) Health added in v0.1.91 

func (i *Instance) Health() Health

Health returns the outcome of the last probe, or ok for a plugin that is not a HealthChecker or has not been probed yet.

func (*Instance) Held 

func (i *Instance) Held() bool

Held reports whether a compiled workflow or an in-flight request still holds the instance.

func (*Instance) Mutates 

func (i *Instance) Mutates() bool

Mutates reports whether the plugin declares that it edits content.

func (*Instance) Release 

func (i *Instance) Release()

Release drops one holder recorded by Acquire.

func (*Instance) StreamPolicy 

func (i *Instance) StreamPolicy() (policy pluginapi.StreamPolicy)

StreamPolicy returns the stream policy of a stream-hook instance, or the zero policy (observe) for others. A panicking StreamPolicy is logged and treated as observe.

type InstanceSpec 

type InstanceSpec struct {
	Name     string
	Config   json.RawMessage
	FailMode FailMode
	// Timeout bounds every hook call; zero means no per-instance timeout.
	Timeout time.Duration
}

InstanceSpec is the operator-facing configuration of one instance.

type MetricsSink 

type MetricsSink interface {
	Inc(name string, labels map[string]string)
	Observe(name string, value float64, labels map[string]string)
}

MetricsSink receives plugin metrics. Names arrive already prefixed with "plugin_<name>_".

type Outcome 

type Outcome struct {
	// Decision is the most severe decision of the run (allow when nothing
	// objected). Blocking decisions end the chain after their step.
	Decision pluginapi.Decision
	// Instance names the instance that produced Decision, when not allow.
	Instance string
	Records  []Record
}

Outcome is the merged result of a chain run.

type PluginError 

type PluginError struct {
	Instance string
	Phase    pluginapi.Kind
	Err      error
}

PluginError reports a fail-closed instance failure (error, panic, or timeout). The server renders it as HTTP 500 with code plugin_failure and records the instance name in the audit trail only.

func (*PluginError) Error 

func (e *PluginError) Error() string

func (*PluginError) Unwrap 

func (e *PluginError) Unwrap() error

type PromptEdit 

type PromptEdit struct {
	Instance string
	Apply    func() (any, error)
}

PromptEdit is one instance's edit of the prompt, kept for the audit revision chain. Apply builds the request as it stood right after that step, from the phase's original request and a snapshot of the prompt taken when the step completed; it is safe to call off the request path.

type Record 

type Record struct {
	Instance string
	// Type is the plugin type of the instance.
	Type string
	// Step is the chain step the instance ran in.
	Step     int
	Decision pluginapi.Decision
	Duration time.Duration
	// Err is set when the instance failed; a fail-open failure still leaves
	// the chain running.
	Err error
	// Edited reports that the instance changed the prompt or response. Only
	// a mutator can; a mutator that ran and left them alone is not edited.
	Edited bool
}

Record is one instance's contribution to a chain run, kept for audit.

type Ref 

type Ref struct {
	Instance *Instance
	Step     int
}

Ref points a chain at an instance running at a step.

type RegisterOptions 

type RegisterOptions struct {
	SingleInstance bool
}

RegisterOptions tunes Register.

type RequestState 

type RequestState struct {
	Values          pluginapi.Values
	ResponseHeaders http.Header
	Decisions       []DecisionRecord
	// contains filtered or unexported fields
}

RequestState is the per-request plugin state shared by every Exchange built for one request: the Values bag, the response headers plugins add, and the decisions taken so far.

func NewRequestState 

func NewRequestState() *RequestState

NewRequestState allocates an empty state. Its maps are created on first use so a request that runs no plugin pays for nothing but the pointer.

func RequestStateFor 

func RequestStateFor(ctx context.Context) *RequestState

RequestStateFor returns the request's state, creating it on the request's workflow the first time a plugin runs. Every later phase of the request finds the same state through RequestStateFromContext. Without a workflow the state is not shared; it lives only for the caller.

func RequestStateFromContext 

func RequestStateFromContext(ctx context.Context) *RequestState

RequestStateFromContext returns the request's state, or nil when no plugin has run for the request yet.

func WithRequestState 

func WithRequestState(ctx context.Context) (context.Context, *RequestState)

WithRequestState ensures ctx carries a RequestState and returns it. The request path does not need it: RequestStateFor creates the state on the request's workflow, so only tests and callers without a workflow use this.

func (*RequestState) AddPromptEdit 

func (s *RequestState) AddPromptEdit(edit PromptEdit)

AddPromptEdit keeps one prompt edit, in step order.

func (*RequestState) AddResponseHeader 

func (s *RequestState) AddResponseHeader(key, value string)

AddResponseHeader appends a response header value.

func (*RequestState) ApplyRequestHeaders 

func (s *RequestState) ApplyRequestHeaders(dst http.Header) []string

ApplyRequestHeaders replays the request header edits plugins made onto dst (the live request headers) and returns the names it changed. Credential headers and redacted placeholders are never written.

func (*RequestState) ApplyResponseHeaders 

func (s *RequestState) ApplyResponseHeaders(dst http.Header)

ApplyResponseHeaders applies the collected headers to a response. A header whose only value is the empty string is removed from dst (see pluginapi.Headers.Response).

func (*RequestState) Finish 

func (s *RequestState) Finish(x *pluginapi.Exchange)

Finish folds an exchange back into the state after a chain ran: upstream headers are logged (not applied in this version).

func (*RequestState) NewExchange 

func (s *RequestState) NewExchange(ctx context.Context, meta pluginapi.Meta) *pluginapi.Exchange

NewExchange builds an Exchange bound to the request state: Values and Headers.Response are shared with every other Exchange of the request; Headers.Request is a redacted copy of the inbound headers.

func (*RequestState) NoStore added in v0.1.91 

func (s *RequestState) NoStore() bool

NoStore reports whether any recorded decision asked for the response not to be stored in the response cache. It implements core.ResponseCacheVeto, which the cache consults after the handler ran.

func (*RequestState) PromptEdits 

func (s *RequestState) PromptEdits() []PromptEdit

PromptEdits returns a copy of the kept prompt edits, in step order.

func (*RequestState) Record 

func (s *RequestState) Record(records ...DecisionRecord)

Record appends decisions. A decision carrying NoStore vetoes storing the response in the response cache (see NoStore).

func (*RequestState) Snapshot 

func (s *RequestState) Snapshot() []DecisionRecord

Snapshot returns a copy of the recorded decisions.

type RouteConfigSource 

type RouteConfigSource func(name string) (json.RawMessage, bool)

RouteConfigSource returns the instance-scoped config of the route plugin registered under name: the config of a guardrail definition of the same name, when one exists. A missing definition (false) selects an empty config, which is validated against the plugin's instance-scoped fields.

type RouteResolver 

type RouteResolver struct {
	// contains filtered or unexported fields
}

RouteResolver builds one instance per routing-strategy plugin, lazily on first use, and validates virtual model strategy_config values against the plugin's route-scoped fields. Instances are rebuilt when their instance-scoped config changes. It is safe for concurrent use.

func NewRouteResolver 

func NewRouteResolver(catalog *Catalog, deps HostDeps) *RouteResolver

NewRouteResolver returns a resolver over the catalog's route plugins.

func (*RouteResolver) Close 

func (r *RouteResolver) Close(ctx context.Context) error

Close releases every built and retired strategy instance, held or not: it runs at shutdown. It waits for a rebuild in flight, so the instance that rebuild publishes is closed as well, and refuses later lookups.

func (*RouteResolver) Names 

func (r *RouteResolver) Names() []string

Names lists the loaded plugins that implement the route hook, sorted.

func (*RouteResolver) Release 

func (r *RouteResolver) Release(inst *Instance)

Release drops the hold Strategy took on inst and closes the retired instances no call holds any more, so a replaced instance goes away when its last Select returns rather than at the next rebuild or report.

func (*RouteResolver) ReportOutcome 

func (r *RouteResolver) ReportOutcome(outcome pluginapi.RouteOutcome)

ReportOutcome hands one upstream attempt to every built strategy, each call recovered from panics, so strategies learn from traffic they did not steer. Each instance is held for the duration of its call, so a rebuild in the meantime cannot close it underneath OnAttemptEnd.

func (*RouteResolver) SetInstanceConfigs 

func (r *RouteResolver) SetInstanceConfigs(source RouteConfigSource)

SetInstanceConfigs installs the source of instance-scoped configs. It may be called after strategies were built: the next Strategy call rebuilds an instance whose config changed.

func (*RouteResolver) Strategy 

func (r *RouteResolver) Strategy(name string) (pluginapi.RouteStrategy, *Instance, error)

Strategy returns the initialized strategy for the named plugin, building it on first use. It fails when the plugin is not loaded, does not implement the route hook, or its instance-scoped config (from the guardrail definition of the same name, or empty) is rejected by validation or Init; the failure is cached until that config changes, so a broken plugin costs one lookup per request rather than one Init.

The returned instance is held for the caller, who must hand it back to Release after the call: a config change replaces the instance, and the replaced one is closed only once no caller holds it.

func (*RouteResolver) ValidateRouteConfig 

func (r *RouteResolver) ValidateRouteConfig(name string, cfg map[string]any) (json.RawMessage, error)

ValidateRouteConfig checks cfg against the route-scoped fields of the named plugin and returns the canonical JSON: defaults applied, values coerced, keys sorted. Errors name the offending key.

type RuleDescriptor 

type RuleDescriptor struct {
	Name    string
	Type    string
	Order   int
	Mode    string
	Content string
}

RuleDescriptor describes one chain member for hashing. Mode carries the instance fail mode and Content its config digest.

type ShortCircuit 

type ShortCircuit struct {
	Instance   string
	Decision   pluginapi.Decision
	Completion *pluginapi.Completion
}

ShortCircuit is returned through the request path when a prompt-phase plugin answers the request itself (ActionRespond). The server renders Completion in the request's dialect with HTTP 200.

func (*ShortCircuit) Error 

func (e *ShortCircuit) Error() string

type Source 

type Source string

Source says where a plugin type came from: compiled into the binary, added through the ext registry, or loaded from a shared object path. 

const (
	SourceBuiltin    Source = "builtin"
	SourceRegistered Source = "registered"
)

type Step 

type Step struct {
	Order     int
	Instances []*Instance
}

Step groups the instances that run together at one order value.

Source Files

View all Source files

Directories

Path Synopsis
Package builtin lists the plugins compiled into GoModel.
 Package builtin lists the plugins compiled into GoModel.
headeredit
Package headeredit is the built-in header_edit plugin: it sets, adds, and removes HTTP headers on the request, the client response, and the upstream provider call.
 Package headeredit is the built-in header_edit plugin: it sets, adds, and removes HTTP headers on the request, the client response, and the upstream provider call.
llmaltering
Package llmaltering is the built-in llm_based_altering plugin: it rewrites the text of selected message roles through an auxiliary model, both in the prompt phase (before the provider call) and in the response phase.
 Package llmaltering is the built-in llm_based_altering plugin: it rewrites the text of selected message roles through an auxiliary model, both in the prompt phase (before the provider call) and in the response phase.
llmjudge
Package llmjudge is the built-in llm_judge plugin: it asks a second model whether a prompt or a completion violates a policy and blocks, answers, or flags the exchange based on the verdict.
 Package llmjudge is the built-in llm_judge plugin: it asks a second model whether a prompt or a completion violates a policy and blocks, answers, or flags the exchange based on the verdict.
presidio
Package presidio is the built-in presidio plugin: it sends prompt and completion text to a Presidio analyzer sidecar, and anonymizes, flags, or blocks the personal data it finds.
 Package presidio is the built-in presidio plugin: it sends prompt and completion text to a Presidio analyzer sidecar, and anonymizes, flags, or blocks the personal data it finds.
routeexample
Package routeexample is the built-in cheapest_healthy routing strategy: it sends a virtual model's traffic to the cheapest (or fastest) target whose recent error rate is acceptable, learning target health from the attempt outcomes GoModel reports.
 Package routeexample is the built-in cheapest_healthy routing strategy: it sends a virtual model's traffic to the cheapest (or fastest) target whose recent error rate is acceptable, learning target health from the attempt outcomes GoModel reports.
stringreplace
Package stringreplace is the built-in string_replace plugin: it rewrites, flags, or blocks prompt and completion text that matches a list of literal or regular-expression rules, in place for non-streaming responses and in flight for streams.
 Package stringreplace is the built-in string_replace plugin: it rewrites, flags, or blocks prompt and completion text that matches a list of literal or regular-expression rules, in place for non-streaming responses and in flight for streams.
systemprompt
Package systemprompt is the built-in system_prompt plugin: it injects, overrides, or decorates the system message before the provider call.
 Package systemprompt is the built-in system_prompt plugin: it injects, overrides, or decorates the system message before the provider call.
tagreplace
Package tagreplace is the built-in tag_replace plugin: before the provider call it expands {{gomodel.<tag>}} placeholders in prompt text into request facts such as the resolved model, the user path, or the current date.
 Package tagreplace is the built-in tag_replace plugin: before the provider call it expands {{gomodel.<tag>}} placeholders in prompt text into request facts such as the resolved model, the user path, or the current date.
Package exchange maps GoModel's core request and response types to the unified pluginapi types plugins see, and applies plugin edits back.
 Package exchange maps GoModel's core request and response types to the unified pluginapi types plugins see, and applies plugin edits back.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.