Documentation
¶
Overview ¶
Package target defines the interface for evaluation targets — the system under test. Targets can be raw LLM APIs, agents, or plain functions.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func WithCallOptions ¶
func WithCallOptions(ctx context.Context, o CallOptions) context.Context
WithCallOptions attaches the run's choices to ctx.
Types ¶
type AgentClient ¶
type AgentClient interface {
Run(ctx context.Context, agentID, personaRef, input string) (*AgentResponse, error)
}
AgentClient is the interface for invoking a Cortex agent and capturing the full run trace for persona-aware evaluation.
type AgentResponse ¶
AgentResponse holds the response from an agent invocation.
type AgentTarget ¶
type AgentTarget struct {
AgentID string
PersonaRef string
Client AgentClient
}
AgentTarget invokes a Cortex agent and captures the full run trace. This enables persona-aware scoring by providing tool calls, steps, and execution metadata to the scorers.
func NewAgentTarget ¶
func NewAgentTarget(client AgentClient, agentID, personaRef string) *AgentTarget
NewAgentTarget creates a target that invokes a Cortex agent.
func (*AgentTarget) Name ¶
func (t *AgentTarget) Name() string
type CallOptions ¶
CallOptions carry what the run decided for this call: the prompt of the suite's current prompt version (or the suite's own prompt), the model and the temperature. A target that builds its own request should prefer these to whatever it was constructed with, so the prompt a run records is the prompt the model actually saw.
func CallOptionsFrom ¶
func CallOptionsFrom(ctx context.Context) (CallOptions, bool)
CallOptionsFrom returns the run's choices, if the engine attached any.
type FuncTarget ¶
type FuncTarget struct {
// contains filtered or unexported fields
}
FuncTarget wraps a plain function as an eval target.
func FromFunc ¶
FromFunc creates a named target from a function. The function receives the input string and returns the output. Latency is measured automatically.
func (*FuncTarget) Name ¶
func (t *FuncTarget) Name() string
type LLMClient ¶
type LLMClient interface {
Complete(ctx context.Context, model, systemPrompt, userInput string, temperature float64) (*LLMResponse, error)
}
LLMClient is the interface for calling a language model. Implementations bridge to specific LLM providers.
type LLMResponse ¶
LLMResponse holds the response from an LLM call.
type LLMTarget ¶
LLMTarget calls a language model directly (traditional eval).
func NewLLMTarget ¶
NewLLMTarget creates a target that calls an LLM directly.
type Output ¶
type Output struct {
Output string // The text response
Latency time.Duration // Response time
Tokens int // Tokens used
Cost float64 // Estimated cost
Trace *evalrun.RunTrace // Agent execution trace (nil for raw LLM targets)
}
Output includes both the text output and optional execution trace.
Source Files