selfplay

package
v1.5.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 26, 2026 License: Apache-2.0 Imports: 25 Imported by: 0

Documentation

Overview

Package selfplay provides self-play capabilities for arena testing scenarios. It enables LLM-driven user simulation and audio generation for duplex conversations.

Index

Constants

View Source 
const (
	TTSProviderOpenAI     = "openai"
	TTSProviderElevenLabs = "elevenlabs"
	TTSProviderCartesia   = "cartesia"
)

Supported TTS provider names.

View Source 
const CompletionInstruction = `` /* 339-byte string literal not displayed */

CompletionInstruction is appended to the persona system prompt when natural termination is enabled.

View Source 
const CompletionMarker = "[CONVERSATION_COMPLETE]"

CompletionMarker is the token the self-play LLM emits to signal the conversation is complete.

View Source 
const (
	// TTSProviderMock is the provider name for the mock TTS service.
	TTSProviderMock = "mock"
)

Variables

View Source 
var ErrConversationComplete = errors.New("conversation complete")

ErrConversationComplete is returned when the self-play LLM signals the conversation is done.

Functions

func DetectAndStripCompletion added in v1.3.16 

func DetectAndStripCompletion(content string) (string, bool)

DetectAndStripCompletion checks whether content contains the completion marker. Returns the cleaned content (marker removed) and whether the marker was detected.

func NewCachedTTSService added in v1.4.7 

func NewCachedTTSService(backend tts.Service, dir string) (tts.Service, error)

NewCachedTTSService wraps backend in a disk cache rooted at dir. The directory is created if missing. Returns the backend unwrapped if dir is empty (so callers can pass an env-var lookup directly).

Types

type AudioContentGenerator added in v1.1.6 

type AudioContentGenerator struct {
	// contains filtered or unexported fields
}

AudioContentGenerator wraps a ContentGenerator and adds TTS synthesis.

textGenerator may be nil when constructed for scripted-text turns where no LLM is involved. In that case NextUserTurnAudioStream returns an error; SynthesizeTextStream still works.

func NewAudioContentGenerator added in v1.1.6 

func NewAudioContentGenerator(
	textGenerator *ContentGenerator,
	ttsService base.TTSProvider,
	ttsProvider *config.Provider,
) *AudioContentGenerator

NewAudioContentGenerator creates a new audio content generator backed by a loaded TTS provider config (capability=tts). textGenerator may be nil for TTS-only generators.

When the TTS service exposes a PersonaRubric (see tts.PersonaRubricProvider), it is forwarded to the underlying text generator so that personas opting in via style.expressive get a provider-tuned rubric prepended to their system prompt. Providers that do not implement the interface, or implement it and return the empty string, are no-ops.

func (*AudioContentGenerator) GetTTSService added in v1.1.6 

func (g *AudioContentGenerator) GetTTSService() base.TTSProvider

GetTTSService returns the TTS provider for direct access if needed.

func (*AudioContentGenerator) GetTextGenerator added in v1.1.6 

func (g *AudioContentGenerator) GetTextGenerator() *ContentGenerator

GetTextGenerator returns the underlying text generator.

func (*AudioContentGenerator) NextUserTurnAudioStream added in v1.4.7 

func (g *AudioContentGenerator) NextUserTurnAudioStream(
	ctx context.Context,
	history []types.Message,
	scenarioID string,
	opts *GeneratorOptions,
) (*AudioStreamResult, error)

NextUserTurnAudioStream implements AudioGenerator. Generates user-turn text via the underlying ContentGenerator and returns a streaming reader for the synthesized audio. Memory consumption is bounded regardless of utterance length.

func (*AudioContentGenerator) SynthesizeTextStream added in v1.4.7 

func (g *AudioContentGenerator) SynthesizeTextStream(
	ctx context.Context,
	text string,
) (*AudioStreamResult, error)

SynthesizeTextStream implements AudioGenerator. Skips the LLM text generation step — used for scripted-text duplex turns where the text arrives pre-known from the scenario YAML.

type AudioGenerator added in v1.1.6 

type AudioGenerator interface {
	// NextUserTurnAudioStream generates a user message and returns the
	// synthesized audio as a streaming reader. Caller is responsible for
	// closing the reader. The text result is available before the audio
	// has finished streaming.
	NextUserTurnAudioStream(
		ctx context.Context,
		history []types.Message,
		scenarioID string,
		opts *GeneratorOptions,
	) (*AudioStreamResult, error)

	// SynthesizeTextStream synthesizes pre-known text directly to audio
	// (no LLM-driven text generation). Used by scripted-text duplex turns
	// where the text is from the scenario YAML, not a persona.
	// Caller is responsible for closing Reader.
	SynthesizeTextStream(
		ctx context.Context,
		text string,
	) (*AudioStreamResult, error)
}

AudioGenerator generates user messages with audio output for duplex self-play. It wraps a text ContentGenerator and adds TTS synthesis.

All audio output is streamed through io.ReadCloser — there is no buffered shape. Memory stays bounded by the chunk size the caller reads with, regardless of utterance length, which is required for any production-like workload where TTS output can run minutes long.

Implementations should ensure the reader returned by stream methods arrives at TTS-source rate (faster than playback for real providers, instant for mocks), letting downstream consumers buffer or pace as they see fit.

type AudioProvider added in v1.1.6 

type AudioProvider interface {
	Provider

	// GetAudioContentGenerator returns an AudioGenerator backed by a loaded TTS
	// provider config (capability=tts).
	GetAudioContentGenerator(role, personaID string, ttsProvider *config.Provider) (AudioGenerator, error)
}

AudioProvider extends Provider with audio generation capabilities for duplex mode.

type AudioStreamResult added in v1.4.7 

type AudioStreamResult struct {
	// TextResult is the text generation result. Nil for SynthesizeTextStream
	// where the text was supplied by the caller and no LLM ran.
	TextResult *pipeline.ExecutionResult

	// Text is the synthesized utterance. For NextUserTurnAudioStream it
	// matches TextResult.Response.Content; for SynthesizeTextStream it
	// is the input text.
	Text string

	// Reader is the streaming audio body. Caller must Close it.
	Reader io.ReadCloser

	// AudioFormat describes the audio encoding.
	AudioFormat tts.AudioFormat

	// SampleRate is the sample rate of the audio data in Hz.
	SampleRate int
}

AudioStreamResult is the result of an audio generation. The audio hasn't been synthesized yet when this is returned — the caller drains Reader to consume it as it arrives from the TTS provider.

type CacheKey 

type CacheKey struct {
	Role      string
	PersonaID string
}

CacheKey represents a structured cache key for user generators

type CacheStats 

type CacheStats struct {
	Size        int     // Number of cached entries
	Hits        int     // Number of cache hits
	Misses      int     // Number of cache misses
	HitRate     float64 // Cache hit rate (0.0 to 1.0)
	CachedPairs []CacheKey
}

CacheStats provides observability into cache performance

type CachedTTSService added in v1.4.7 

type CachedTTSService struct {
	// contains filtered or unexported fields
}

CachedTTSService wraps another tts.Service with a content-addressed disk cache. Cache keys hash the backend name, voice, format, and full text together so two backends or two voices for the same text never collide. Cache misses fall through to the backend and persist the returned bytes; cache hits read straight from disk.

Thread-safe: a per-key sync.Mutex prevents two concurrent Synthesize calls for the same key from both hitting the backend, but otherwise requests fan out freely.

func (*CachedTTSService) Close added in v1.4.7 

func (c *CachedTTSService) Close() error

Close proxies to the backend if it satisfies base.Provider.

func (*CachedTTSService) HealthCheck added in v1.4.7 

func (c *CachedTTSService) HealthCheck(ctx context.Context) error

HealthCheck proxies to the backend if it satisfies base.Provider.

func (*CachedTTSService) Init added in v1.4.7 

func (c *CachedTTSService) Init(ctx context.Context) error

Init proxies to the backend if it satisfies base.Provider.

func (*CachedTTSService) Name added in v1.4.7 

func (c *CachedTTSService) Name() string

Name returns the underlying backend's name unchanged so consumers can't tell whether they're hitting the cache.

func (*CachedTTSService) Pricing added in v1.4.7 

func (c *CachedTTSService) Pricing() *base.PricingDescriptor

Pricing proxies to the backend if it satisfies base.Provider; otherwise nil.

func (*CachedTTSService) SupportedFormats added in v1.4.7 

func (c *CachedTTSService) SupportedFormats() []tts.AudioFormat

SupportedFormats proxies to the backend.

func (*CachedTTSService) SupportedVoices added in v1.4.7 

func (c *CachedTTSService) SupportedVoices() []tts.Voice

SupportedVoices proxies to the backend.

func (*CachedTTSService) Synthesize added in v1.4.7 

func (c *CachedTTSService) Synthesize(
	ctx context.Context, text string, cfg tts.SynthesisConfig,
) (io.ReadCloser, error)

Synthesize returns cached audio when available; otherwise calls the backend and persists its output before returning a reader for it.

func (*CachedTTSService) SynthesizeTTS added in v1.4.7 

func (c *CachedTTSService) SynthesizeTTS(ctx context.Context, req base.TTSRequest) (base.TTSStream, error)

SynthesizeTTS implements base.TTSProvider. It serves cached bytes when available (using the default PCM format key), otherwise delegates to the backend's SynthesizeTTS. The returned stream is not itself cached.

func (*CachedTTSService) Type added in v1.4.7 

func (c *CachedTTSService) Type() base.ProviderType

Type returns ProviderTypeTTS for CachedTTSService.

func (*CachedTTSService) Validate added in v1.4.7 

func (c *CachedTTSService) Validate() error

Validate proxies to the backend if it satisfies base.Provider.

type ContentGenerator 

type ContentGenerator struct {
	// contains filtered or unexported fields
}

ContentGenerator generates user messages using an LLM

func NewContentGenerator 

func NewContentGenerator(provider providers.Provider, persona *config.UserPersonaPack) *ContentGenerator

NewContentGenerator creates a new content generator with a specific provider and persona

func (*ContentGenerator) NextUserTurn 

func (cg *ContentGenerator) NextUserTurn(
	ctx context.Context,
	history []types.Message,
	scenarioID string,
	opts *GeneratorOptions,
) (*pipeline.ExecutionResult, error)

NextUserTurn generates a user message using the LLM through a stage pipeline. The opts parameter is optional and can be nil.

func (*ContentGenerator) WithProviderRubric added in v1.4.7 

func (cg *ContentGenerator) WithProviderRubric(rubric string) *ContentGenerator

WithProviderRubric sets the TTS provider's characterization rubric that will be prepended to the persona's system prompt when the persona has opted in via style.expressive. Empty string disables the rubric (no-op for personas that did not opt in, and the explicit "this provider does not support characterization" signal otherwise). See issue #1130.

type Generator 

type Generator interface {
	NextUserTurn(
		ctx context.Context,
		history []types.Message,
		scenarioID string,
		opts *GeneratorOptions,
	) (*pipeline.ExecutionResult, error)
}

Generator generates user messages for self-play scenarios. Each generator is configured with a specific persona and LLM provider, and produces user turns based on conversation history. Returns the full pipeline ExecutionResult which includes trace data, costs, and metadata. The opts parameter is optional and can be nil.

type GeneratorOptions added in v1.1.6 

type GeneratorOptions struct {
	// SelfplayTurnIndex is the 1-indexed selfplay turn number (first selfplay turn = 1).
	// If set, this overrides the automatic turn counting from history.
	// This is important for scenarios with mixed file-based and selfplay turns.
	SelfplayTurnIndex int

	// NaturalTerminationEnabled signals the generator to append completion instructions
	// to the persona system prompt, allowing the self-play LLM to end the conversation.
	NaturalTerminationEnabled bool

	// Emitter is an optional event emitter for provider call telemetry.
	Emitter *events.Emitter
}

GeneratorOptions provides optional configuration for self-play generation.

type MockTTSService added in v1.1.6 

type MockTTSService struct {
	// SampleRate is the audio sample rate (default: 24000).
	SampleRate int

	// AudioFiles is a list of PCM audio files to load and rotate through.
	// If empty, falls back to generating silence.
	AudioFiles []string

	// Latency simulates network/processing delay before returning audio.
	Latency time.Duration
	// contains filtered or unexported fields
}

MockTTSService is a mock TTS service for testing. It loads audio from PCM files to provide realistic speech for testing.

func NewMockTTS added in v1.1.6 

func NewMockTTS() *MockTTSService

NewMockTTS creates a new mock TTS service with default settings.

func NewMockTTSWithFiles added in v1.1.6 

func NewMockTTSWithFiles(audioFiles []string) *MockTTSService

NewMockTTSWithFiles creates a mock TTS service that loads audio from files.

func NewMockTTSWithLatency added in v1.1.6 

func NewMockTTSWithLatency(latency time.Duration) *MockTTSService

NewMockTTSWithLatency creates a mock TTS service with simulated latency.

func (*MockTTSService) Close added in v1.4.7 

func (m *MockTTSService) Close() error

Close releases resources (no-op for mock).

func (*MockTTSService) HealthCheck added in v1.4.7 

func (m *MockTTSService) HealthCheck(_ context.Context) error

HealthCheck reports liveness (no-op for mock).

func (*MockTTSService) Init added in v1.4.7 

func (m *MockTTSService) Init(_ context.Context) error

Init performs asynchronous setup (no-op for mock).

func (*MockTTSService) Name added in v1.1.6 

func (m *MockTTSService) Name() string

Name returns the provider name.

func (*MockTTSService) Pricing added in v1.4.7 

func (m *MockTTSService) Pricing() *base.PricingDescriptor

Pricing returns nil for the mock (free provider, no billing).

func (*MockTTSService) SupportedFormats added in v1.1.6 

func (m *MockTTSService) SupportedFormats() []tts.AudioFormat

SupportedFormats returns supported audio output formats.

func (*MockTTSService) SupportedVoices added in v1.1.6 

func (m *MockTTSService) SupportedVoices() []tts.Voice

SupportedVoices returns available mock voices.

func (*MockTTSService) Synthesize added in v1.1.6 

func (m *MockTTSService) Synthesize(
	ctx context.Context, text string, _ tts.SynthesisConfig,
) (io.ReadCloser, error)

Synthesize converts text to mock PCM audio. If audio files are configured, returns audio from those files (rotating through them). Otherwise falls back to generating silence.

Returns a chunked reader that caps each Read at ~20 ms of audio so a caller using a large buffer (e.g. io.ReadAll) still observes chunk-shaped reads. Pacing — the actual wall-clock cadence — is the AudioPacingStage's job, applied downstream by the duplex pipeline.

func (*MockTTSService) SynthesizeTTS added in v1.4.7 

func (m *MockTTSService) SynthesizeTTS(ctx context.Context, req base.TTSRequest) (base.TTSStream, error)

SynthesizeTTS implements base.TTSProvider. It bridges the base.TTSRequest to the existing Synthesize method and wraps the response in a TTSStream. Cost is nil because the mock has no pricing.

func (*MockTTSService) Type added in v1.4.7 

func (m *MockTTSService) Type() base.ProviderType

Type returns ProviderTypeTTS for MockTTSService.

func (*MockTTSService) Validate added in v1.4.7 

func (m *MockTTSService) Validate() error

Validate performs synchronous config validation (no-op for mock).

type Provider 

type Provider interface {
	GetContentGenerator(role, personaID string) (Generator, error)
}

Provider provides access to content generators for self-play scenarios. This is the main interface that the engine and turn executors use to obtain content generators based on role and persona.

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

Registry manages self-play providers, personas, and user generator creation

func NewRegistry 

func NewRegistry(
	providerRegistry *providers.Registry,
	providerMap map[string]string,
	personas map[string]*config.UserPersonaPack,
	roles []config.SelfPlayRoleGroup,
) *Registry

NewRegistry creates a new self-play registry

func NewRegistryWithTTS added in v1.1.6 

func NewRegistryWithTTS(
	providerRegistry *providers.Registry,
	providerMap map[string]string,
	personas map[string]*config.UserPersonaPack,
	roles []config.SelfPlayRoleGroup,
	ttsRegistry *TTSRegistry,
) *Registry

NewRegistryWithTTS creates a new self-play registry with a custom TTS registry. This is useful for testing or when using pre-configured TTS services.

func (*Registry) ClearCache 

func (r *Registry) ClearCache()

ClearCache clears all cached user generators and resets statistics

func (*Registry) Close 

func (r *Registry) Close() error

Close closes the self-play provider registry and cleans up resources

func (*Registry) GetAudioContentGenerator added in v1.1.6 

func (r *Registry) GetAudioContentGenerator(
	role, personaID string,
	ttsProvider *config.Provider,
) (AudioGenerator, error)

GetAudioContentGenerator implements AudioProvider interface. Returns an AudioGenerator backed by a loaded TTS provider config (capability=tts).

Unlike GetContentGenerator, audio generators are not cached since the provider config may vary across roles.

func (*Registry) GetAvailableProviders 

func (r *Registry) GetAvailableProviders() []string

GetAvailableProviders returns a list of available providers

func (*Registry) GetAvailableRoles 

func (r *Registry) GetAvailableRoles() []string

GetAvailableRoles returns a list of available self-play roles

func (*Registry) GetCacheStats 

func (r *Registry) GetCacheStats() CacheStats

GetCacheStats returns current cache statistics

func (*Registry) GetContentGenerator 

func (r *Registry) GetContentGenerator(role, personaID string) (Generator, error)

GetContentGenerator implements Provider interface Returns a Generator for the given role and persona (cached for efficiency). Uses double-check locking to prevent duplicate creation under concurrency.

func (*Registry) GetPersona added in v1.4.7 

func (r *Registry) GetPersona(personaID string) *config.UserPersonaPack

GetPersona returns the parsed persona pack for the given ID, or nil if no persona by that ID is registered. Used by the duplex executor to surface the persona definition on selfplay user messages so the arena UI can show "what drove this turn?" alongside the generated text.

func (*Registry) GetSTTRegistry added in v1.5.3 

func (r *Registry) GetSTTRegistry() *STTRegistry

GetSTTRegistry returns the shared STT registry.

func (*Registry) GetTTSRegistry added in v1.1.6 

func (r *Registry) GetTTSRegistry() *TTSRegistry

GetTTSRegistry returns the TTS registry for direct access if needed.

func (*Registry) IsValidRole 

func (r *Registry) IsValidRole(role string) bool

IsValidRole checks if a role is configured for self-play

func (*Registry) PrewarmCache 

func (r *Registry) PrewarmCache(rolePairs []CacheKey) error

PrewarmCache creates and caches ContentGenerators for common role+persona combinations This can improve performance by avoiding cold starts during actual execution

type STTRegistry added in v1.5.3 

type STTRegistry struct {
	// contains filtered or unexported fields
}

STTRegistry manages STT service instances by provider type, mirroring TTSRegistry. Used by the interactive voice console's VAD branch to transcribe microphone audio.

func NewSTTRegistry added in v1.5.3 

func NewSTTRegistry() *STTRegistry

NewSTTRegistry creates a new STT registry.

func (*STTRegistry) GetForProvider added in v1.5.3 

func (r *STTRegistry) GetForProvider(p *config.Provider) (stt.Service, error)

GetForProvider returns an stt.Service configured from a loaded STT provider yaml. Validates role == "stt". A service registered via Register for the provider Type wins; otherwise routing is by Type, pinning Model when set.

func (*STTRegistry) Register added in v1.5.3 

func (r *STTRegistry) Register(providerType string, svc stt.Service)

Register adds a pre-configured STT service keyed by provider type, mirroring TTSRegistry.Register. A registered service takes precedence over the built-in vendor adapters in GetForProvider, which lets tests inject a deterministic fake without requiring an API key.

type TTSRegistry added in v1.1.6 

type TTSRegistry struct {
	// contains filtered or unexported fields
}

TTSRegistry manages TTS provider instances by provider name. It supports lazy initialization and caching of TTS providers.

func NewTTSRegistry added in v1.1.6 

func NewTTSRegistry() *TTSRegistry

NewTTSRegistry creates a new TTS registry.

func (*TTSRegistry) Clear added in v1.1.6 

func (r *TTSRegistry) Clear()

Clear removes all cached services.

func (*TTSRegistry) Get added in v1.1.6 

func (r *TTSRegistry) Get(provider string) (base.TTSProvider, error)

Get returns a TTS provider for the given provider name. Providers are lazily initialized on first request and cached.

When the TTS_CACHE_DIR environment variable is set, the returned provider is wrapped in a CachedTTSService rooted at that directory so repeated synthesis of the same text doesn't re-bill the upstream provider. Mock providers are exempt — they're already deterministic and would just bloat the cache.

func (*TTSRegistry) GetForProvider added in v1.4.7 

func (r *TTSRegistry) GetForProvider(p *config.Provider) (base.TTSProvider, error)

GetForProvider returns a base.TTSProvider configured from a loaded TTS provider yaml. Provider routing is by Type (cartesia/elevenlabs/openai/mock); the voice/sample_rate/audio_files/model fields on the provider populate the synthesis config.

Validates that the provider has Role == "tts" — guards against callers passing an LLM provider by mistake.

func (*TTSRegistry) Register added in v1.1.6 

func (r *TTSRegistry) Register(provider string, svc base.TTSProvider)

Register adds a pre-configured TTS provider to the registry. This is useful for testing or when using custom configurations.

func (*TTSRegistry) SupportedProviders added in v1.1.6 

func (r *TTSRegistry) SupportedProviders() []string

SupportedProviders returns a list of supported TTS provider names.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.