Documentation
¶
Overview ¶
Package stages provides arena-specific pipeline stages for test execution.
Index ¶
- type ArenaAssertionStage
- type ArenaStateStoreSaveStage
- type AssistantTTSFilterStage
- type CompletionInstructionStage
- type HistoryInjectionStage
- type MockScenarioContextStage
- type PersonaAssemblyStage
- type STTUserMessageStage
- type ScenarioContextExtractionStage
- type SelfPlayUserTurnContextStage
- func NewSelfPlayUserTurnContextStageWithHintAndTurnState(scenario *config.Scenario, turnIndexHint int, personaID string, ...) *SelfPlayUserTurnContextStage
- func NewSelfPlayUserTurnContextStageWithTurnState(scenario *config.Scenario, personaID string, turnState *stage.TurnState) *SelfPlayUserTurnContextStage
- type SkillInstructionStage
- type StripToolMessagesStage
- type TTSSentenceAggregatorStage
- type TurnEvalRunner
- type TurnIndexStage
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ArenaAssertionStage ¶
ArenaAssertionStage validates assertions after LLM responses.
func NewArenaAssertionStage ¶
func NewArenaAssertionStage( assertionConfigs []assertions.AssertionConfig, ) *ArenaAssertionStage
NewArenaAssertionStage creates a new assertion stage.
func (*ArenaAssertionStage) Process ¶
func (s *ArenaAssertionStage) Process( ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement, ) error
Process validates assertions on the stream elements.
func (*ArenaAssertionStage) WithTurnEvalRunner ¶ added in v1.3.2
func (s *ArenaAssertionStage) WithTurnEvalRunner(runner TurnEvalRunner, sessionID string) *ArenaAssertionStage
WithTurnEvalRunner sets the eval runner for assertion execution.
type ArenaStateStoreSaveStage ¶
ArenaStateStoreSaveStage saves conversation state with telemetry to ArenaStateStore. This stage captures validation results, turn metrics, and cost information for Arena testing and analysis.
func NewArenaStateStoreSaveStage ¶
func NewArenaStateStoreSaveStage(config *pipeline.StateStoreConfig) *ArenaStateStoreSaveStage
NewArenaStateStoreSaveStage creates a new Arena state store save stage.
func NewArenaStateStoreSaveStageWithTurnState ¶ added in v1.4.7
func NewArenaStateStoreSaveStageWithTurnState( config *pipeline.StateStoreConfig, turnState *stage.TurnState, ) *ArenaStateStoreSaveStage
NewArenaStateStoreSaveStageWithTurnState creates an Arena state store save stage that reads the rendered system prompt from the supplied TurnState. Falls back to config.Metadata["system_prompt"] when the TurnState is empty (used by tests that wire the prompt via config).
func (*ArenaStateStoreSaveStage) Process ¶
func (s *ArenaStateStoreSaveStage) Process( ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement, ) error
Process collects messages and saves them incrementally to Arena state store. Messages are saved after each turn completion (when an element contains a Message). This ensures conversation state is captured in real-time as turns complete.
The conversation state is loaded lazily on the first save and reused for subsequent saves within the same Process call — avoiding O(N) Loads on scenarios with many Message elements.
func (*ArenaStateStoreSaveStage) WithEmitter ¶ added in v1.4.7
func (s *ArenaStateStoreSaveStage) WithEmitter(emitter *events.Emitter) *ArenaStateStoreSaveStage
WithEmitter wires an events.Emitter so the stage broadcasts each message to the event bus the moment it arrives. Live UIs (TUI conversation panel, web SSE relay) can then render turns as they happen instead of waiting for run completion. Stage-level state save remains the source of truth for replay and post-run results.
type AssistantTTSFilterStage ¶ added in v1.5.3
AssistantTTSFilterStage sits between the ArenaStateStoreSaveStage and the TTSStageWithInterruption in the VAD composed pipeline. It drops any Message element whose Role is not "assistant", preventing the user's own transcript (wrapped as a user Message by STTUserMessageStage) from being spoken back through TTS.
All non-Message elements — audio, text, EndOfStream, and control elements — are forwarded unchanged so the TTS stage receives everything it needs.
This is a Filter stage: it passes through assistant Messages and non-Message elements, and drops user/system/tool Messages.
func NewAssistantTTSFilterStage ¶ added in v1.5.3
func NewAssistantTTSFilterStage() *AssistantTTSFilterStage
NewAssistantTTSFilterStage creates a filter that only forwards assistant Message elements (and all non-Message elements) to the TTS stage.
func (*AssistantTTSFilterStage) Process ¶ added in v1.5.3
func (s *AssistantTTSFilterStage) Process( ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement, ) error
Process forwards every element unchanged except Message elements whose Role is not "assistant" — those are silently dropped.
type CompletionInstructionStage ¶ added in v1.3.16
type CompletionInstructionStage struct {
stage.BaseStage
// contains filtered or unexported fields
}
CompletionInstructionStage appends natural termination instructions to the system prompt assembled by PersonaAssemblyStage. The instruction is appended exactly once per Turn — TurnState.SystemPrompt is the source of truth and the stage is idempotent across the elements that flow through it.
func NewCompletionInstructionStageWithTurnState ¶ added in v1.4.7
func NewCompletionInstructionStageWithTurnState( instruction string, turnState *stage.TurnState, ) *CompletionInstructionStage
NewCompletionInstructionStageWithTurnState creates a stage that reads and writes the system prompt through the shared *TurnState.
func (*CompletionInstructionStage) Process ¶ added in v1.3.16
func (s *CompletionInstructionStage) Process(ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement) error
Process appends the completion instruction to the system prompt.
type HistoryInjectionStage ¶
HistoryInjectionStage prepends conversation history to the pipeline stream. This is useful for self-play scenarios where history needs to be explicitly provided before the LLM generates the next user message.
The stage emits all history messages first, then forwards any incoming elements.
When swapRoles is true, user↔assistant roles are swapped so that the self-play LLM sees its own prior outputs as "assistant" and the target's responses as "user". This prevents the self-play LLM from confusing itself with the target assistant.
func NewHistoryInjectionStage ¶
func NewHistoryInjectionStage(history []types.Message) *HistoryInjectionStage
NewHistoryInjectionStage creates a new history injection stage.
func NewHistoryInjectionStageSwapped ¶ added in v1.3.16
func NewHistoryInjectionStageSwapped(history []types.Message) *HistoryInjectionStage
NewHistoryInjectionStageSwapped creates a history injection stage that swaps user↔assistant roles. Use this for self-play generation so the LLM sees the conversation from its own perspective (its prior outputs as "assistant").
func (*HistoryInjectionStage) Process ¶
func (s *HistoryInjectionStage) Process(ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement) error
Process emits history messages first, then forwards all incoming elements.
type MockScenarioContextStage ¶
MockScenarioContextStage adds scenario context to TurnState's ProviderRequestMetadata so MockProvider can select scenario-specific canned responses.
This stage should be placed before ProviderStage in the pipeline when using MockProvider to ensure scenario context is available.
func NewMockScenarioContextStageWithTurnState ¶ added in v1.4.7
func NewMockScenarioContextStageWithTurnState( scenario *config.Scenario, turnState *stage.TurnState, ) *MockScenarioContextStage
NewMockScenarioContextStageWithTurnState creates a stage that writes scenario context into the shared *TurnState's ProviderRequestMetadata.
func (*MockScenarioContextStage) Process ¶
func (s *MockScenarioContextStage) Process(ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement) error
Process writes scenario context into TurnState and forwards all elements unchanged.
type PersonaAssemblyStage ¶
PersonaAssemblyStage assembles persona prompts using the same fragment/template system as PromptAssemblyStage.
This stage mirrors the behavior of PromptAssemblyMiddleware but for personas:
- Uses persona's BuildSystemPrompt() which handles fragment assembly
- Supports template variable substitution with {{variable}} syntax
- Injects persona-specific variables (goals, constraints, style)
- Optionally prepends a TTS characterization rubric when the persona has opted in via style.expressive (see issue #1130)
- Writes the rendered system prompt and base variables into TurnState
func NewPersonaAssemblyStageWithTurnState ¶ added in v1.4.7
func NewPersonaAssemblyStageWithTurnState( persona *config.UserPersonaPack, region string, baseVariables map[string]string, turnState *stage.TurnState, ) *PersonaAssemblyStage
NewPersonaAssemblyStageWithTurnState creates a persona assembly stage that writes the rendered persona system prompt into the shared *TurnState.
func (*PersonaAssemblyStage) Process ¶
func (s *PersonaAssemblyStage) Process(ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement) error
Process assembles the persona prompt and writes it into TurnState. All input elements are forwarded unchanged.
func (*PersonaAssemblyStage) WithProviderRubric ¶ added in v1.4.7
func (s *PersonaAssemblyStage) WithProviderRubric(rubric string) *PersonaAssemblyStage
WithProviderRubric sets the TTS provider's characterization rubric. The stage uses it as the default when the persona opts into expressive output (style.expressive) and does not provide its own override. Empty string disables the default and is the no-op path for non-expressive providers.
type STTUserMessageStage ¶ added in v1.5.3
STTUserMessageStage converts a transcribed text element (as produced by the STT stage) into a user Message element so the downstream prompt-assembly, provider, and state-store-save stages treat the spoken turn exactly like a typed user message. This is the bridge that makes the composed VAD voice pipeline materialize history identically to a text run: ProviderStage only accumulates Message elements, so a bare Text element from STT would otherwise never reach the provider or be persisted.
Text-only elements are wrapped; elements that already carry a Message, plus audio / EndOfStream / non-text elements, are forwarded unchanged.
This is a Transform stage: text element -> user message element (1:1).
func NewSTTUserMessageStage ¶ added in v1.5.3
func NewSTTUserMessageStage() *STTUserMessageStage
NewSTTUserMessageStage creates a stage that wraps STT text into user messages.
func (*STTUserMessageStage) Process ¶ added in v1.5.3
func (s *STTUserMessageStage) Process( ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement, ) error
Process wraps text elements into user message elements and forwards the rest.
type ScenarioContextExtractionStage ¶
type ScenarioContextExtractionStage struct {
stage.BaseStage
// contains filtered or unexported fields
}
ScenarioContextExtractionStage extracts context using scenario metadata and conversation history. This is designed for Arena use where rich scenario metadata is available.
When scenario metadata is present, it uses: - Scenario metadata variables (domain, user role from scenario definition) - Scenario description and task type - Message analysis as fallback
Extracted variables are merged into TurnState.Variables, where TemplateStage reads them when rendering the system prompt.
func NewScenarioContextExtractionStageWithTurnState ¶ added in v1.4.7
func NewScenarioContextExtractionStageWithTurnState( scenario *config.Scenario, turnState *stage.TurnState, ) *ScenarioContextExtractionStage
NewScenarioContextExtractionStageWithTurnState creates a scenario context extraction stage that writes extracted variables into the shared *TurnState.
func (*ScenarioContextExtractionStage) Process ¶
func (s *ScenarioContextExtractionStage) Process(ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement) error
Process extracts scenario context and writes it to TurnState.Variables.
type SelfPlayUserTurnContextStage ¶
type SelfPlayUserTurnContextStage struct {
stage.BaseStage
// contains filtered or unexported fields
}
SelfPlayUserTurnContextStage adds scenario context for the NEXT user turn (completed user turns + 1). Intended only for self-play user generation.
This stage writes scenario/turn coordination keys into TurnState's ProviderRequestMetadata so the mock provider can select the appropriate canned response by scenario id, turn number, and persona.
func NewSelfPlayUserTurnContextStageWithHintAndTurnState ¶ added in v1.4.7
func NewSelfPlayUserTurnContextStageWithHintAndTurnState( scenario *config.Scenario, turnIndexHint int, personaID string, turnState *stage.TurnState, ) *SelfPlayUserTurnContextStage
NewSelfPlayUserTurnContextStageWithHintAndTurnState creates a self-play context stage with an explicit turn index. The turnIndexHint should be the 1-indexed selfplay turn number (first selfplay = 1). Used when the scenario has mixed file-based and selfplay turns.
func NewSelfPlayUserTurnContextStageWithTurnState ¶ added in v1.4.7
func NewSelfPlayUserTurnContextStageWithTurnState( scenario *config.Scenario, personaID string, turnState *stage.TurnState, ) *SelfPlayUserTurnContextStage
NewSelfPlayUserTurnContextStageWithTurnState creates a self-play context stage that writes coordination metadata into the shared *TurnState.
func (*SelfPlayUserTurnContextStage) Process ¶
func (s *SelfPlayUserTurnContextStage) Process(ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement) error
Process writes next-turn self-play context into TurnState's provider request metadata, then forwards all input elements unchanged.
type SkillInstructionStage ¶ added in v1.4.5
SkillInstructionStage appends preloaded skill instructions to the system prompt so the model sees skills marked preload: true from turn 1 without having to call skill__activate. The instructions are appended exactly once per Turn into TurnState.SystemPrompt.
func NewSkillInstructionStageWithTurnState ¶ added in v1.4.7
func NewSkillInstructionStageWithTurnState( instructions string, turnState *stage.TurnState, ) *SkillInstructionStage
NewSkillInstructionStageWithTurnState creates a stage that reads and writes the system prompt through the shared *TurnState.
func (*SkillInstructionStage) Process ¶ added in v1.4.5
func (s *SkillInstructionStage) Process(ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement) error
Process appends the preloaded skill instructions to the system prompt.
type StripToolMessagesStage ¶
StripToolMessagesStage removes tool role messages from the stream. This is used in self-play scenarios before calling the self-play provider.
func NewStripToolMessagesStage ¶
func NewStripToolMessagesStage() *StripToolMessagesStage
NewStripToolMessagesStage creates a new strip tool messages stage.
func (*StripToolMessagesStage) Process ¶
func (s *StripToolMessagesStage) Process( ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement, ) error
Process filters out elements with tool role messages.
type TTSSentenceAggregatorStage ¶ added in v1.5.3
type TTSSentenceAggregatorStage struct {
stage.BaseStage
// contains filtered or unexported fields
}
TTSSentenceAggregatorStage turns a streaming provider's token-by-token text deltas into complete sentences for TTS, so the agent speaks in smooth phrases instead of "word … pause … word".
It sits between the AssistantTTSFilterStage (which has already dropped the user transcript) and the TTS stage. As assistant text deltas arrive it accumulates them and emits each complete sentence (ending in . ! or ?) the moment it forms — while the model is still generating the rest. That overlaps generation with speech: audio starts after the first sentence rather than after the whole reply.
The complete assistant Message that the provider emits at the end of a turn duplicates the deltas, so once any delta has been seen the Message is dropped (after flushing the trailing partial sentence). For a NON-streaming provider (no deltas), the Message's content is split into sentences instead.
Control signals pass through: Interrupt clears the buffer (barge-in discards the half-spoken reply); EndOfTurn / EndOfStream flush the trailing buffer.
func NewTTSSentenceAggregatorStage ¶ added in v1.5.3
func NewTTSSentenceAggregatorStage() *TTSSentenceAggregatorStage
NewTTSSentenceAggregatorStage creates the sentence aggregator.
func (*TTSSentenceAggregatorStage) Process ¶ added in v1.5.3
func (s *TTSSentenceAggregatorStage) Process( ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement, ) error
Process implements the Stage interface.
type TurnEvalRunner ¶ added in v1.3.2
type TurnEvalRunner interface {
RunAssertionsAsEvals(
ctx context.Context,
assertionConfigs []assertions.AssertionConfig,
messages []types.Message,
turnIndex int,
sessionID string,
trigger evals.EvalTrigger,
) []evals.EvalResult
}
TurnEvalRunner is an interface for running assertions as evals during turn execution. EvalOrchestrator in the engine package implements this interface.
type TurnIndexStage ¶
TurnIndexStage computes role-specific turn counters from accumulated messages. It writes the counters into TurnState's ProviderRequestMetadata so downstream stages (e.g. mock provider context) and persisted state.Metadata can read them. Keys written:
- arena_user_completed_turns: number of completed user messages
- arena_user_next_turn: completed user messages + 1 (next user turn to generate)
- arena_assistant_completed_turns: number of completed assistant messages
- arena_assistant_next_turn: completed assistant messages + 1
func NewTurnIndexStage ¶
func NewTurnIndexStage() *TurnIndexStage
NewTurnIndexStage creates a new turn index stage that does not publish into any TurnState. Useful for tests that only need passthrough behavior.
func NewTurnIndexStageWithTurnState ¶ added in v1.4.7
func NewTurnIndexStageWithTurnState(turnState *stage.TurnState) *TurnIndexStage
NewTurnIndexStageWithTurnState creates a turn index stage that publishes the computed counters onto the supplied TurnState's ProviderRequestMetadata.
func (*TurnIndexStage) Process ¶
func (s *TurnIndexStage) Process(ctx context.Context, input <-chan stage.StreamElement, output chan<- stage.StreamElement) error
Process computes role-specific turn counters and forwards elements unchanged.
Source Files
¶
- assertions.go
- assistant_tts_filter.go
- completion_instruction.go
- history_injection.go
- instruction_helpers.go
- mock_scenario_context.go
- persona_assembly.go
- scenario_context_extraction.go
- selfplay_context.go
- skill_instruction.go
- statestore_save_integration.go
- strip_tool_messages.go
- stt_user_message.go
- tts_sentence_aggregator.go
- turn_index.go