Documentation
¶
Overview ¶
Package evaluation implements typed remote evaluations independently of workflow and ACP.
Index ¶
- func MarshalResult(result Result) ([]byte, error)
- func ValidateConfig(config Config, getenv func(string) string) error
- func ValidateRequest(request Request) error
- func ValidateResult(request Request, result Result) error
- type Answer
- type Config
- type Error
- type ErrorClass
- type Evaluator
- type Request
- type Result
- type Trace
- type Usage
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func MarshalResult ¶
MarshalResult returns deterministic compact JSON for result.
func ValidateConfig ¶
ValidateConfig checks trusted local service configuration without remote I/O.
func ValidateRequest ¶
ValidateRequest checks the completely rendered request before credential lookup.
func ValidateResult ¶
ValidateResult checks a normalized result against its rendered request.
Types ¶
type Answer ¶
type Answer struct {
Type string `json:"type"`
Noul *float64 `json:"noul,omitempty"`
Choice string `json:"choice,omitempty"`
Score *float64 `json:"score,omitempty"`
Legend map[string]any `json:"legend,omitempty"`
Probabilities map[string]float64 `json:"probabilities,omitempty"`
Confidence *float64 `json:"confidence,omitempty"`
}
Answer is the canonical union returned for one question.
type Error ¶
type Error struct {
Class ErrorClass
Op string
Code int
Err error
}
Error is a classified failure that excludes response bodies and secrets.
type ErrorClass ¶
type ErrorClass string
ErrorClass is a bounded operational failure category.
const ( ErrorConfiguration ErrorClass = "configuration" ErrorAuthentication ErrorClass = "authentication" ErrorRateLimit ErrorClass = "rate_limit" ErrorTransport ErrorClass = "transport" ErrorTimeout ErrorClass = "timeout" ErrorProvider ErrorClass = "provider" ErrorResponse ErrorClass = "response" ErrorCanceled ErrorClass = "canceled" )
type Evaluator ¶
type Evaluator interface {
Evaluate(ctx context.Context, config Config, request Request) (Result, Trace, error)
}
Evaluator executes one rendered request through the service selected by kind.
func NewEvaluator ¶
func NewEvaluator() Evaluator
type Request ¶
type Request struct {
Model string
State any
Questions map[string]agent.EvaluationQuestion
}
Request is one fully rendered typed evaluation request.
type Result ¶
type Result struct {
Service string `json:"service"`
Provider string `json:"provider,omitempty"`
RequestID string `json:"requestId,omitempty"`
RequestedModel string `json:"requestedModel"`
Model string `json:"model"`
Answers map[string]Answer `json:"answers"`
Usage *Usage `json:"usage,omitempty"`
}
Result is the service-neutral value published to workflow state.