Documentation
¶
Overview ¶
Package guardrails provides built-in ProviderHook implementations that bridge the unified eval system to the pipeline's hook infrastructure.
Index ¶
- Constants
- Variables
- func CompileValidators(validators []prompt.ValidatorConfig) ([]hooks.ProviderHook, error)
- func CompileValidatorsWithRegistry(validators []prompt.ValidatorConfig, registry *evals.EvalTypeRegistry) ([]hooks.ProviderHook, error)
- func NewGuardrailHook(typeName string, params map[string]any, opts ...GuardrailOption) (hooks.ProviderHook, error)
- func NewGuardrailHookFromRegistry(typeName string, params map[string]any, registry *evals.EvalTypeRegistry, ...) (hooks.ProviderHook, error)
- func ValidatorsToHooks(validators []prompt.ValidatorConfig) []hooks.ProviderHookdeprecated
- func ValidatorsToHooksWithRegistry(validators []prompt.ValidatorConfig, registry *evals.EvalTypeRegistry) []hooks.ProviderHookdeprecated
- type GuardrailHookAdapter
- func (a *GuardrailHookAdapter) AfterCall(ctx context.Context, req *hooks.ProviderRequest, resp *hooks.ProviderResponse) hooks.Decision
- func (a *GuardrailHookAdapter) BeforeCall(ctx context.Context, req *hooks.ProviderRequest) hooks.Decision
- func (a *GuardrailHookAdapter) Name() string
- func (a *GuardrailHookAdapter) OnChunk(ctx context.Context, chunk *providers.StreamChunk) hooks.Decision
- type GuardrailOption
- type Spec
- func Input(evalType string, params map[string]any, opts ...GuardrailOption) Spec
- func InputFunc(name string, fn func(context.Context, *hooks.InputRequest) hooks.Decision) Spec
- func Output(evalType string, params map[string]any, opts ...GuardrailOption) Spec
- func OutputFunc(name string, fn func(context.Context, *hooks.OutputRequest) hooks.Decision) Spec
Constants ¶
const ( DirectionInput = hooks.DirectionInput DirectionOutput = hooks.DirectionOutput DirectionBoth = hooks.DirectionBoth )
Aliases of the canonical direction constants, retained for existing callers. See runtime/hooks for what each value means; the vocabulary lives there so the pipeline stage need not import a concrete hook implementation to tag a firing.
Variables ¶
var ErrEmptySpec = errors.New(
"uninitialized Spec — build it with Input, Output, InputFunc or OutputFunc")
ErrEmptySpec is returned by Spec.Hook for a zero-value Spec — one that never went through Input, Output, InputFunc or OutputFunc. Reachable from a pre-sized slice (make([]guardrails.Spec, n)) whose entries a branch failed to assign; returning an error keeps that config mistake out of the panic path.
var ErrUnknownGuardrailType = errors.New("unknown guardrail type")
ErrUnknownGuardrailType is returned when a validator names an eval type that is not registered. It is distinguishable from other construction failures because it is treated as fatal: an unknown type has no legitimate use, so dropping it would leave a conversation silently unprotected. See CompileValidators.
Functions ¶
func CompileValidators ¶ added in v1.5.8
func CompileValidators(validators []prompt.ValidatorConfig) ([]hooks.ProviderHook, error)
CompileValidators turns pack-declared validators into ProviderHooks suitable for prepending to a hook registry. Both SDK.Open and Arena's per-turn pipeline use this so guardrails run identically in production and in tests.
Per-validator behavior:
- Validators with Enabled == &false are skipped silently (explicit opt-out). nil Enabled means enabled (spec default).
- All accepted validators enforce: on a hit they rewrite the assistant message in place (truncate or replace). If you want observe-only behavior, declare an eval and assert on it; guardrails always act.
- "message" set on the validator becomes the user-facing blocked text, falling back to Params["message"].
Failure policy, split by how ambiguous the mistake is:
- An **unknown eval type** is FATAL and returns ErrUnknownGuardrailType. A type that is not registered has no legitimate use — it is a typo — and silently dropping it leaves the conversation with no protection while load appears to succeed. That is fail-open on a safety control.
- A validator whose **params** are unusable is logged and skipped, so one bad entry does not break the others. A pack authored against a newer runtime can legitimately carry params this build does not understand; refusing to load would make packs forward-incompatible.
On a fatal error no hooks are returned, so a caller cannot accidentally proceed with a partial guardrail set.
func CompileValidatorsWithRegistry ¶ added in v1.5.8
func CompileValidatorsWithRegistry( validators []prompt.ValidatorConfig, registry *evals.EvalTypeRegistry, ) ([]hooks.ProviderHook, error)
CompileValidatorsWithRegistry is CompileValidators resolving each validator's eval type against the supplied registry instead of the built-in default. Pass the registry a caller configured (sdk.WithEvalRegistry) so a custom handler can back a pack validator; a nil registry means the default one.
Without this the default registry does not know a custom type, construction fails, and — on the lenient path — the guardrail is logged and dropped, which leaves the conversation unprotected while load appears to succeed (#1717).
func NewGuardrailHook ¶
func NewGuardrailHook(typeName string, params map[string]any, opts ...GuardrailOption) (hooks.ProviderHook, error)
NewGuardrailHook creates a guardrail ProviderHook using the default eval registry.
func NewGuardrailHookFromRegistry ¶ added in v1.3.12
func NewGuardrailHookFromRegistry( typeName string, params map[string]any, registry *evals.EvalTypeRegistry, opts ...GuardrailOption, ) (hooks.ProviderHook, error)
NewGuardrailHookFromRegistry creates a guardrail ProviderHook using the eval registry. Any registered eval handler (including aliases) can be used as a guardrail.
If the handler implements evals.ParamValidator, the params are normalised (ApplyDefaults + NormalizeParams) and passed to ValidateParams before the hook is constructed. This surfaces invalid pack validators at SDK load time instead of silently failing every turn — handlers for which params are unusable return an error here, and the SDK's warn-and-skip loop in convertPackValidatorsToHooks logs and drops them.
func ValidatorsToHooks
deprecated
added in
v1.4.9
func ValidatorsToHooks(validators []prompt.ValidatorConfig) []hooks.ProviderHook
ValidatorsToHooks is the lenient form: every unusable validator — including an unknown eval type — is logged and skipped, and the usable ones are still returned.
Deprecated: use CompileValidators. This form cannot report an unknown eval type, so a typo'd validator is silently dropped and the caller proceeds unprotected. Retained unchanged so existing callers keep their behavior.
func ValidatorsToHooksWithRegistry
deprecated
added in
v1.5.8
func ValidatorsToHooksWithRegistry( validators []prompt.ValidatorConfig, registry *evals.EvalTypeRegistry, ) []hooks.ProviderHook
ValidatorsToHooksWithRegistry is the lenient form of CompileValidatorsWithRegistry: every unusable validator — including an unknown eval type — is logged and skipped. A nil registry means the default one.
Deprecated: use CompileValidatorsWithRegistry. This form cannot report an unknown eval type, so a typo'd validator is silently dropped and the caller proceeds unprotected.
Types ¶
type GuardrailHookAdapter ¶ added in v1.3.12
type GuardrailHookAdapter struct {
// contains filtered or unexported fields
}
GuardrailHookAdapter wraps an evals.EvalTypeHandler as a hooks.ProviderHook. This bridges the unified eval system to the pipeline's hook infrastructure, allowing any registered eval handler to be used as a guardrail.
Guardrails always enforce: on a hit the adapter mutates the response (truncate or replace) and returns an Enforced decision so the pipeline continues. If you want observe-only behavior, declare an eval — not a guardrail — and assert on it in scenarios.
func (*GuardrailHookAdapter) AfterCall ¶ added in v1.3.12
func (a *GuardrailHookAdapter) AfterCall( ctx context.Context, req *hooks.ProviderRequest, resp *hooks.ProviderResponse, ) hooks.Decision
AfterCall checks provider output when direction is "output" or "both". When the guardrail triggers, it enforces in-place on resp.Message (truncating or replacing content) and returns an Enforced decision.
func (*GuardrailHookAdapter) BeforeCall ¶ added in v1.3.12
func (a *GuardrailHookAdapter) BeforeCall( ctx context.Context, req *hooks.ProviderRequest, ) hooks.Decision
BeforeCall checks input when direction is "input" or "both".
It evaluates only when the last message is a user message (see lastUserTurn). BeforeCall runs once per round inside the tool loop, where later rounds end in a tool-result message rather than user input — evaluating those would score the wrong content and rebill LLM-judged checks every round. The gate is deliberately content-based rather than round-based: a round check would also misfire on a round whose last message is an assistant message, and round numbering is per-ProviderStage (it restarts in each composition sub-pipeline), so it is not a reliable proxy for "there is new user input".
func (*GuardrailHookAdapter) Name ¶ added in v1.3.12
func (a *GuardrailHookAdapter) Name() string
Name returns the eval type identifier for this guardrail.
func (*GuardrailHookAdapter) OnChunk ¶ added in v1.3.12
func (a *GuardrailHookAdapter) OnChunk( ctx context.Context, chunk *providers.StreamChunk, ) hooks.Decision
OnChunk evaluates streaming chunks via StreamableEvalHandler.EvalPartial. When a guardrail triggers, it truncates the chunk content and returns an Enforced decision so the provider stage can stop reading but continue the pipeline.
type GuardrailOption ¶ added in v1.3.12
type GuardrailOption func(*GuardrailHookAdapter)
GuardrailOption configures a GuardrailHookAdapter.
func WithMessage ¶ added in v1.3.12
func WithMessage(msg string) GuardrailOption
WithMessage sets the user-facing message shown when content is blocked.
type Spec ¶ added in v1.5.8
type Spec struct {
// contains filtered or unexported fields
}
Spec is a declared guardrail, not yet built into a hook. Construction errors (unknown eval type, invalid params) surface from Hook() so callers can declare guardrails inline and report all failures at one point — typically sdk.Open.
func Input ¶ added in v1.5.8
func Input(evalType string, params map[string]any, opts ...GuardrailOption) Spec
Input declares an eval-backed guardrail that gates the user's input before the provider call. Any registered eval handler may be named.
guardrails.Input("pii_leakage", nil)
func InputFunc ¶ added in v1.5.8
InputFunc declares a guardrail from a plain function gating user input. The function runs once per user turn: it is skipped on tool-loop rounds, where the last message is a tool result rather than user input.
guardrails.InputFunc("no-wires", func(ctx context.Context, in *hooks.InputRequest) hooks.Decision {
if strings.Contains(in.UserInput, "wire transfer") {
in.Replacement = "I can't help with transfers."
return hooks.Enforced("wire transfer requested", nil)
}
return hooks.Allow
})
func Output ¶ added in v1.5.8
func Output(evalType string, params map[string]any, opts ...GuardrailOption) Spec
Output declares an eval-backed guardrail that gates the assistant's response.
func OutputFunc ¶ added in v1.5.8
OutputFunc declares a guardrail from a plain function gating the assistant response. Mutate OutputRequest.Message in place and return Enforced to rewrite. Returning Enforced also stops the provider round loop and drops any tool calls the response requested; downstream pipeline stages still run.
func (Spec) Hook ¶ added in v1.5.8
func (s Spec) Hook() (hooks.ProviderHook, error)
Hook builds the ProviderHook this Spec describes against the default eval registry. A zero-value Spec returns ErrEmptySpec rather than panicking.
func (Spec) HookWithRegistry ¶ added in v1.5.8
func (s Spec) HookWithRegistry(registry *evals.EvalTypeRegistry) (hooks.ProviderHook, error)
HookWithRegistry builds the ProviderHook resolving an eval-backed guardrail's type against registry. A nil registry means the default one, so HookWithRegistry(nil) and Hook() are equivalent.
Callers that let a user supply their own evals.EvalTypeRegistry — the SDK's WithEvalRegistry — must use this form. Building against the default registry makes a custom eval type unknown, and the guardrail is then dropped rather than enforced (#1717). Func-backed Specs (InputFunc, OutputFunc) ignore the registry: they carry their own logic.
Source Files