Documentation
¶
Overview ¶
Package tts provides a unified interface for Text-to-Speech providers.
This file defines capability interfaces for local TTS providers. Local providers (F5-TTS, Qwen3-TTS, Piper, etc.) may implement these interfaces to expose additional functionality beyond the base Provider.
Use type assertions to check for capability support:
if cloner, ok := provider.(tts.VoiceCloner); ok {
profile, err := cloner.CloneVoice(ctx, req)
}
Package tts provides a unified interface for Text-to-Speech providers.
Index ¶
- Variables
- type Client
- func (c *Client) Hook() observability.TTSHook
- func (c *Client) SetHook(hook observability.TTSHook)
- func (c *Client) Synthesize(ctx context.Context, text string, config SynthesisConfig) (*SynthesisResult, error)
- func (c *Client) SynthesizeStream(ctx context.Context, text string, config SynthesisConfig) (<-chan StreamChunk, error)
- type CloneVoiceRequest
- type Closer
- type HealthChecker
- type HealthStatus
- type LoadModelResult
- type ModelInfo
- type ModelManager
- type PrepareVoiceProfileRequest
- type PreparedProfile
- type ProfileCacher
- type Provider
- type ReferenceSynthesizeRequest
- type ReferenceSynthesizer
- type RuntimeChecker
- type RuntimeInfo
- type StreamChunk
- type StreamingProvider
- type StreamingReferenceSynthesizer
- type SynthesisConfig
- type SynthesisResult
- type UnloadModelResult
- type Voice
- type VoiceCloner
- type VoiceProfile
Constants ¶
This section is empty.
Variables ¶
var ( // ErrNoAvailableProvider is returned when no provider is available. ErrNoAvailableProvider = errors.New("tts: no available provider") // ErrVoiceNotFound is returned when a voice ID is not found. ErrVoiceNotFound = errors.New("tts: voice not found") // ErrInvalidConfig is returned when the synthesis config is invalid. ErrInvalidConfig = errors.New("tts: invalid configuration") // ErrRateLimited is returned when the provider rate limits the request. ErrRateLimited = errors.New("tts: rate limited") // ErrQuotaExceeded is returned when the provider quota is exceeded. ErrQuotaExceeded = errors.New("tts: quota exceeded") // ErrStreamClosed is returned when attempting to use a closed stream. ErrStreamClosed = errors.New("tts: stream closed") )
Functions ¶
This section is empty.
Types ¶
type Client ¶
Client provides a unified interface across multiple TTS providers.
func (*Client) Hook ¶ added in v0.6.0
func (c *Client) Hook() observability.TTSHook
Hook returns the current observability hook.
func (*Client) SetHook ¶ added in v0.6.0
func (c *Client) SetHook(hook observability.TTSHook)
SetHook sets the observability hook for all TTS operations.
func (*Client) Synthesize ¶
func (c *Client) Synthesize(ctx context.Context, text string, config SynthesisConfig) (*SynthesisResult, error)
Synthesize uses the primary provider with smart fallback. Fallback only occurs for permanent (non-retryable) errors. Transient errors like rate limits are expected to be handled by the provider's retry logic.
func (*Client) SynthesizeStream ¶
func (c *Client) SynthesizeStream(ctx context.Context, text string, config SynthesisConfig) (<-chan StreamChunk, error)
SynthesizeStream uses the primary provider with smart fallback. Fallback only occurs for permanent (non-retryable) errors.
type CloneVoiceRequest ¶ added in v0.15.0
type CloneVoiceRequest struct {
// Name is a human-readable name for the voice profile.
Name string
// ReferenceAudio is the audio data (WAV or PCM format).
ReferenceAudio []byte
// ReferenceText is the transcript of the reference audio.
// Must accurately match what is spoken.
ReferenceText string
// Language is the BCP-47 language code (e.g., "en-US").
Language string
}
CloneVoiceRequest contains parameters for voice cloning.
type Closer ¶ added in v0.15.0
Closer is implemented by providers that hold resources requiring cleanup.
type HealthChecker ¶ added in v0.15.0
type HealthChecker interface {
// Health returns the health status of the provider.
Health(ctx context.Context) (*HealthStatus, error)
}
HealthChecker is implemented by providers that support health checks.
type HealthStatus ¶ added in v0.15.0
type HealthStatus struct {
// Healthy indicates whether the provider is operational.
Healthy bool
// ModelLoaded indicates whether the model is loaded.
ModelLoaded bool
// ModelName is the name of the loaded model.
ModelName string
// ModelVersion is the version of the loaded model.
ModelVersion string
// AvailableVoices lists available voice profile IDs.
AvailableVoices []string
}
HealthStatus contains health information for a provider.
type LoadModelResult ¶ added in v0.15.0
type LoadModelResult struct {
// Success indicates whether loading succeeded.
Success bool
// LoadTimeMs is the time taken to load in milliseconds.
LoadTimeMs int64
// MemoryUsedMB is the memory consumed by the model.
MemoryUsedMB int64
// Error contains error details if Success is false.
Error string
}
LoadModelResult contains the result of loading a model.
type ModelInfo ¶ added in v0.15.0
type ModelInfo struct {
// Name is the model name.
Name string
// Version is the model version.
Version string
// ParameterCount is the number of model parameters.
ParameterCount int64
// SupportedLanguages lists supported language codes.
SupportedLanguages []string
}
ModelInfo contains details about a loaded model.
type ModelManager ¶ added in v0.15.0
type ModelManager interface {
// LoadModel loads the TTS model into memory.
LoadModel(ctx context.Context) (*LoadModelResult, error)
// UnloadModel unloads the model from memory.
UnloadModel(ctx context.Context) (*UnloadModelResult, error)
// IsModelLoaded returns whether the model is currently loaded.
IsModelLoaded() bool
}
ModelManager is implemented by providers that support explicit model loading and unloading.
type PrepareVoiceProfileRequest ¶ added in v0.15.0
type PrepareVoiceProfileRequest struct {
// ProfileID is the unique identifier for this profile.
ProfileID string
// ReferenceAudio is the reference speaker's audio.
ReferenceAudio []byte
// ReferenceText is the transcript of the reference audio.
ReferenceText string
// Language is the BCP-47 language code.
Language string
}
PrepareVoiceProfileRequest contains parameters for profile preparation.
type PreparedProfile ¶ added in v0.15.0
type PreparedProfile struct {
// ProfileID is the unique identifier.
ProfileID string
// Cached indicates whether caching succeeded.
Cached bool
// EmbeddingSizeBytes is the size of the cached embedding.
EmbeddingSizeBytes int64
// CreatedAt is when the profile was prepared.
CreatedAt time.Time
}
PreparedProfile represents a cached voice embedding.
type ProfileCacher ¶ added in v0.15.0
type ProfileCacher interface {
// PrepareVoiceProfile pre-computes and caches a voice embedding.
// Subsequent synthesis using this profile ID will be faster.
PrepareVoiceProfile(ctx context.Context, req PrepareVoiceProfileRequest) (*PreparedProfile, error)
// ListPreparedProfiles returns all cached voice profiles.
ListPreparedProfiles(ctx context.Context) ([]*PreparedProfile, error)
// DeletePreparedProfile removes a cached profile.
DeletePreparedProfile(ctx context.Context, profileID string) error
}
ProfileCacher is implemented by providers that support pre-computing voice embeddings for faster subsequent synthesis.
type Provider ¶
type Provider interface {
// Name returns the provider name.
Name() string
// Synthesize converts text to speech and returns audio data.
Synthesize(ctx context.Context, text string, config SynthesisConfig) (*SynthesisResult, error)
// SynthesizeStream converts text to speech with streaming output.
SynthesizeStream(ctx context.Context, text string, config SynthesisConfig) (<-chan StreamChunk, error)
// ListVoices returns available voices from this provider.
ListVoices(ctx context.Context) ([]Voice, error)
// GetVoice returns a specific voice by ID.
GetVoice(ctx context.Context, voiceID string) (*Voice, error)
}
Provider defines the interface for TTS providers.
type ReferenceSynthesizeRequest ¶ added in v0.15.0
type ReferenceSynthesizeRequest struct {
// Text is the text to synthesize.
Text string
// ReferenceAudio is the reference speaker's audio (WAV or PCM).
ReferenceAudio []byte
// ReferenceText is the transcript of the reference audio.
ReferenceText string
// Config contains synthesis configuration (format, speed, etc.).
Config SynthesisConfig
}
ReferenceSynthesizeRequest contains parameters for reference-based synthesis.
type ReferenceSynthesizer ¶ added in v0.15.0
type ReferenceSynthesizer interface {
// SynthesizeWithReference performs synthesis using reference audio inline.
// This is useful for one-off synthesis without creating a persistent profile.
SynthesizeWithReference(ctx context.Context, req ReferenceSynthesizeRequest) (*SynthesisResult, error)
}
ReferenceSynthesizer is implemented by providers that support zero-shot synthesis with reference audio (without pre-creating a profile).
type RuntimeChecker ¶ added in v0.15.0
type RuntimeChecker interface {
// RuntimeInfo returns information about the runtime environment.
RuntimeInfo(ctx context.Context) (*RuntimeInfo, error)
}
RuntimeChecker is implemented by providers that can report runtime environment information.
type RuntimeInfo ¶ added in v0.15.0
type RuntimeInfo struct {
// DeviceType is the compute device ("mlx", "mps", "cpu", "cuda").
DeviceType string
// MemoryUsedMB is the current memory usage.
MemoryUsedMB int64
// MemoryAvailableMB is the available memory.
MemoryAvailableMB int64
// FrameworkVersion is the ML framework version (e.g., MLX version).
FrameworkVersion string
// PythonVersion is the Python version if applicable.
PythonVersion string
// ModelInfo contains loaded model information.
ModelInfo *ModelInfo
}
RuntimeInfo contains details about the runtime environment.
type StreamChunk ¶
type StreamChunk struct {
// Audio is a chunk of audio data.
Audio []byte
// IsFinal indicates if this is the last chunk.
IsFinal bool
// Error contains any error that occurred during streaming.
Error error
}
StreamChunk represents a chunk of streaming audio.
type StreamingProvider ¶
type StreamingProvider interface {
Provider
// SynthesizeFromReader reads text from a reader and streams audio output.
// Useful for streaming LLM output directly to TTS.
SynthesizeFromReader(ctx context.Context, reader io.Reader, config SynthesisConfig) (<-chan StreamChunk, error)
}
StreamingProvider extends Provider with input streaming support.
type StreamingReferenceSynthesizer ¶ added in v0.15.0
type StreamingReferenceSynthesizer interface {
ReferenceSynthesizer
// SynthesizeWithReferenceStream performs streaming synthesis with reference audio.
SynthesizeWithReferenceStream(ctx context.Context, req ReferenceSynthesizeRequest) (<-chan StreamChunk, error)
}
StreamingReferenceSynthesizer extends ReferenceSynthesizer with streaming output.
type SynthesisConfig ¶
type SynthesisConfig struct {
// VoiceID is the voice to use for synthesis.
VoiceID string
// Model is the provider-specific model identifier (optional).
Model string
// OutputFormat specifies the audio format ("mp3", "pcm", "wav", "opus").
OutputFormat string
// SampleRate is the audio sample rate in Hz (e.g., 22050, 44100).
SampleRate int
// Speed is the speech speed multiplier (1.0 = normal).
Speed float64
// Pitch adjusts the voice pitch (-1.0 to 1.0, 0 = normal).
Pitch float64
// Stability controls voice consistency (0.0 to 1.0, provider-specific).
Stability float64
// SimilarityBoost enhances voice similarity (0.0 to 1.0, provider-specific).
SimilarityBoost float64
// Extensions holds provider-specific settings.
// Keys should be namespaced by provider (e.g., "elevenlabs.style", "deepgram.tier").
// Use provider-specific helper functions for type-safe access.
Extensions map[string]any
// Hook provides observability for TTS operations.
// If nil, no hooks are called.
Hook observability.TTSHook
}
SynthesisConfig configures a TTS synthesis request.
type SynthesisResult ¶
type SynthesisResult struct {
// Audio is the synthesized audio data.
Audio []byte
// Format is the audio format of the result.
Format string
// SampleRate is the sample rate of the audio.
SampleRate int
// DurationMs is the duration of the audio in milliseconds.
DurationMs int
// CharacterCount is the number of characters processed.
CharacterCount int
}
SynthesisResult contains the result of a TTS synthesis.
type UnloadModelResult ¶ added in v0.15.0
type UnloadModelResult struct {
// Success indicates whether unloading succeeded.
Success bool
// MemoryFreedMB is the memory freed by unloading.
MemoryFreedMB int64
}
UnloadModelResult contains the result of unloading a model.
type Voice ¶
type Voice struct {
// ID is the provider-specific voice identifier.
ID string
// Name is a human-readable name for the voice.
Name string
// Language is the BCP-47 language code (e.g., "en-US").
Language string
// Gender is the voice gender ("male", "female", "neutral").
Gender string
// Provider is the name of the TTS provider.
Provider string
// Metadata contains provider-specific additional information.
Metadata map[string]any
}
Voice represents a voice configuration for TTS.
type VoiceCloner ¶ added in v0.15.0
type VoiceCloner interface {
// CloneVoice creates a voice profile from reference audio.
// The reference audio should be 5-15 seconds of clear speech.
// The reference text must accurately transcribe the audio.
CloneVoice(ctx context.Context, req CloneVoiceRequest) (*VoiceProfile, error)
}
VoiceCloner is implemented by providers that support voice cloning. Voice cloning creates a voice profile from reference audio that can be used for subsequent synthesis.
type VoiceProfile ¶ added in v0.15.0
type VoiceProfile struct {
// ID is the unique identifier for this profile.
ID string
// Name is the human-readable name.
Name string
// Language is the BCP-47 language code.
Language string
// CreatedAt is when the profile was created.
CreatedAt time.Time
// Provider is the provider that created this profile.
Provider string
// Cached indicates whether the embedding is cached.
Cached bool
// EmbeddingSize is the size of the cached embedding in bytes.
EmbeddingSize int64
}
VoiceProfile represents a cloned voice that can be used for synthesis.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package providertest provides conformance tests for TTS provider implementations.
|
Package providertest provides conformance tests for TTS provider implementations. |