tts

package
v0.16.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 27, 2026 License: MIT Imports: 7 Imported by: 7

Documentation

Overview

Package tts provides a unified interface for Text-to-Speech providers.

This file defines capability interfaces for local TTS providers. Local providers (F5-TTS, Qwen3-TTS, Piper, etc.) may implement these interfaces to expose additional functionality beyond the base Provider.

Use type assertions to check for capability support:

if cloner, ok := provider.(tts.VoiceCloner); ok {
    profile, err := cloner.CloneVoice(ctx, req)
}

Package tts provides a unified interface for Text-to-Speech providers.

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrNoAvailableProvider is returned when no provider is available.
	ErrNoAvailableProvider = errors.New("tts: no available provider")

	// ErrVoiceNotFound is returned when a voice ID is not found.
	ErrVoiceNotFound = errors.New("tts: voice not found")

	// ErrInvalidConfig is returned when the synthesis config is invalid.
	ErrInvalidConfig = errors.New("tts: invalid configuration")

	// ErrRateLimited is returned when the provider rate limits the request.
	ErrRateLimited = errors.New("tts: rate limited")

	// ErrQuotaExceeded is returned when the provider quota is exceeded.
	ErrQuotaExceeded = errors.New("tts: quota exceeded")

	// ErrStreamClosed is returned when attempting to use a closed stream.
	ErrStreamClosed = errors.New("tts: stream closed")
)

Functions

This section is empty.

Types

type Client

type Client struct {
	*provider.Client[Provider]
	// contains filtered or unexported fields
}

Client provides a unified interface across multiple TTS providers.

func NewClient

func NewClient(providers ...Provider) *Client

NewClient creates a new TTS client with the specified providers.

func (*Client) Hook added in v0.6.0

func (c *Client) Hook() observability.TTSHook

Hook returns the current observability hook.

func (*Client) SetHook added in v0.6.0

func (c *Client) SetHook(hook observability.TTSHook)

SetHook sets the observability hook for all TTS operations.

func (*Client) Synthesize

func (c *Client) Synthesize(ctx context.Context, text string, config SynthesisConfig) (*SynthesisResult, error)

Synthesize uses the primary provider with smart fallback. Fallback only occurs for permanent (non-retryable) errors. Transient errors like rate limits are expected to be handled by the provider's retry logic.

func (*Client) SynthesizeStream

func (c *Client) SynthesizeStream(ctx context.Context, text string, config SynthesisConfig) (<-chan StreamChunk, error)

SynthesizeStream uses the primary provider with smart fallback. Fallback only occurs for permanent (non-retryable) errors.

type CloneVoiceRequest added in v0.15.0

type CloneVoiceRequest struct {
	// Name is a human-readable name for the voice profile.
	Name string

	// ReferenceAudio is the audio data (WAV or PCM format).
	ReferenceAudio []byte

	// ReferenceText is the transcript of the reference audio.
	// Must accurately match what is spoken.
	ReferenceText string

	// Language is the BCP-47 language code (e.g., "en-US").
	Language string
}

CloneVoiceRequest contains parameters for voice cloning.

type Closer added in v0.15.0

type Closer interface {
	io.Closer
}

Closer is implemented by providers that hold resources requiring cleanup.

type HealthChecker added in v0.15.0

type HealthChecker interface {
	// Health returns the health status of the provider.
	Health(ctx context.Context) (*HealthStatus, error)
}

HealthChecker is implemented by providers that support health checks.

type HealthStatus added in v0.15.0

type HealthStatus struct {
	// Healthy indicates whether the provider is operational.
	Healthy bool

	// ModelLoaded indicates whether the model is loaded.
	ModelLoaded bool

	// ModelName is the name of the loaded model.
	ModelName string

	// ModelVersion is the version of the loaded model.
	ModelVersion string

	// AvailableVoices lists available voice profile IDs.
	AvailableVoices []string
}

HealthStatus contains health information for a provider.

type LoadModelResult added in v0.15.0

type LoadModelResult struct {
	// Success indicates whether loading succeeded.
	Success bool

	// LoadTimeMs is the time taken to load in milliseconds.
	LoadTimeMs int64

	// MemoryUsedMB is the memory consumed by the model.
	MemoryUsedMB int64

	// Error contains error details if Success is false.
	Error string
}

LoadModelResult contains the result of loading a model.

type ModelInfo added in v0.15.0

type ModelInfo struct {
	// Name is the model name.
	Name string

	// Version is the model version.
	Version string

	// ParameterCount is the number of model parameters.
	ParameterCount int64

	// SupportedLanguages lists supported language codes.
	SupportedLanguages []string
}

ModelInfo contains details about a loaded model.

type ModelManager added in v0.15.0

type ModelManager interface {
	// LoadModel loads the TTS model into memory.
	LoadModel(ctx context.Context) (*LoadModelResult, error)

	// UnloadModel unloads the model from memory.
	UnloadModel(ctx context.Context) (*UnloadModelResult, error)

	// IsModelLoaded returns whether the model is currently loaded.
	IsModelLoaded() bool
}

ModelManager is implemented by providers that support explicit model loading and unloading.

type PrepareVoiceProfileRequest added in v0.15.0

type PrepareVoiceProfileRequest struct {
	// ProfileID is the unique identifier for this profile.
	ProfileID string

	// ReferenceAudio is the reference speaker's audio.
	ReferenceAudio []byte

	// ReferenceText is the transcript of the reference audio.
	ReferenceText string

	// Language is the BCP-47 language code.
	Language string
}

PrepareVoiceProfileRequest contains parameters for profile preparation.

type PreparedProfile added in v0.15.0

type PreparedProfile struct {
	// ProfileID is the unique identifier.
	ProfileID string

	// Cached indicates whether caching succeeded.
	Cached bool

	// EmbeddingSizeBytes is the size of the cached embedding.
	EmbeddingSizeBytes int64

	// CreatedAt is when the profile was prepared.
	CreatedAt time.Time
}

PreparedProfile represents a cached voice embedding.

type ProfileCacher added in v0.15.0

type ProfileCacher interface {
	// PrepareVoiceProfile pre-computes and caches a voice embedding.
	// Subsequent synthesis using this profile ID will be faster.
	PrepareVoiceProfile(ctx context.Context, req PrepareVoiceProfileRequest) (*PreparedProfile, error)

	// ListPreparedProfiles returns all cached voice profiles.
	ListPreparedProfiles(ctx context.Context) ([]*PreparedProfile, error)

	// DeletePreparedProfile removes a cached profile.
	DeletePreparedProfile(ctx context.Context, profileID string) error
}

ProfileCacher is implemented by providers that support pre-computing voice embeddings for faster subsequent synthesis.

type Provider

type Provider interface {
	// Name returns the provider name.
	Name() string

	// Synthesize converts text to speech and returns audio data.
	Synthesize(ctx context.Context, text string, config SynthesisConfig) (*SynthesisResult, error)

	// SynthesizeStream converts text to speech with streaming output.
	SynthesizeStream(ctx context.Context, text string, config SynthesisConfig) (<-chan StreamChunk, error)

	// ListVoices returns available voices from this provider.
	ListVoices(ctx context.Context) ([]Voice, error)

	// GetVoice returns a specific voice by ID.
	GetVoice(ctx context.Context, voiceID string) (*Voice, error)
}

Provider defines the interface for TTS providers.

type ReferenceSynthesizeRequest added in v0.15.0

type ReferenceSynthesizeRequest struct {
	// Text is the text to synthesize.
	Text string

	// ReferenceAudio is the reference speaker's audio (WAV or PCM).
	ReferenceAudio []byte

	// ReferenceText is the transcript of the reference audio.
	ReferenceText string

	// Config contains synthesis configuration (format, speed, etc.).
	Config SynthesisConfig
}

ReferenceSynthesizeRequest contains parameters for reference-based synthesis.

type ReferenceSynthesizer added in v0.15.0

type ReferenceSynthesizer interface {
	// SynthesizeWithReference performs synthesis using reference audio inline.
	// This is useful for one-off synthesis without creating a persistent profile.
	SynthesizeWithReference(ctx context.Context, req ReferenceSynthesizeRequest) (*SynthesisResult, error)
}

ReferenceSynthesizer is implemented by providers that support zero-shot synthesis with reference audio (without pre-creating a profile).

type RuntimeChecker added in v0.15.0

type RuntimeChecker interface {
	// RuntimeInfo returns information about the runtime environment.
	RuntimeInfo(ctx context.Context) (*RuntimeInfo, error)
}

RuntimeChecker is implemented by providers that can report runtime environment information.

type RuntimeInfo added in v0.15.0

type RuntimeInfo struct {
	// DeviceType is the compute device ("mlx", "mps", "cpu", "cuda").
	DeviceType string

	// MemoryUsedMB is the current memory usage.
	MemoryUsedMB int64

	// MemoryAvailableMB is the available memory.
	MemoryAvailableMB int64

	// FrameworkVersion is the ML framework version (e.g., MLX version).
	FrameworkVersion string

	// PythonVersion is the Python version if applicable.
	PythonVersion string

	// ModelInfo contains loaded model information.
	ModelInfo *ModelInfo
}

RuntimeInfo contains details about the runtime environment.

type StreamChunk

type StreamChunk struct {
	// Audio is a chunk of audio data.
	Audio []byte

	// IsFinal indicates if this is the last chunk.
	IsFinal bool

	// Error contains any error that occurred during streaming.
	Error error
}

StreamChunk represents a chunk of streaming audio.

type StreamingProvider

type StreamingProvider interface {
	Provider

	// SynthesizeFromReader reads text from a reader and streams audio output.
	// Useful for streaming LLM output directly to TTS.
	SynthesizeFromReader(ctx context.Context, reader io.Reader, config SynthesisConfig) (<-chan StreamChunk, error)
}

StreamingProvider extends Provider with input streaming support.

type StreamingReferenceSynthesizer added in v0.15.0

type StreamingReferenceSynthesizer interface {
	ReferenceSynthesizer

	// SynthesizeWithReferenceStream performs streaming synthesis with reference audio.
	SynthesizeWithReferenceStream(ctx context.Context, req ReferenceSynthesizeRequest) (<-chan StreamChunk, error)
}

StreamingReferenceSynthesizer extends ReferenceSynthesizer with streaming output.

type SynthesisConfig

type SynthesisConfig struct {
	// VoiceID is the voice to use for synthesis.
	VoiceID string

	// Model is the provider-specific model identifier (optional).
	Model string

	// OutputFormat specifies the audio format ("mp3", "pcm", "wav", "opus").
	OutputFormat string

	// SampleRate is the audio sample rate in Hz (e.g., 22050, 44100).
	SampleRate int

	// Speed is the speech speed multiplier (1.0 = normal).
	Speed float64

	// Pitch adjusts the voice pitch (-1.0 to 1.0, 0 = normal).
	Pitch float64

	// Stability controls voice consistency (0.0 to 1.0, provider-specific).
	Stability float64

	// SimilarityBoost enhances voice similarity (0.0 to 1.0, provider-specific).
	SimilarityBoost float64

	// Extensions holds provider-specific settings.
	// Keys should be namespaced by provider (e.g., "elevenlabs.style", "deepgram.tier").
	// Use provider-specific helper functions for type-safe access.
	Extensions map[string]any

	// Hook provides observability for TTS operations.
	// If nil, no hooks are called.
	Hook observability.TTSHook
}

SynthesisConfig configures a TTS synthesis request.

type SynthesisResult

type SynthesisResult struct {
	// Audio is the synthesized audio data.
	Audio []byte

	// Format is the audio format of the result.
	Format string

	// SampleRate is the sample rate of the audio.
	SampleRate int

	// DurationMs is the duration of the audio in milliseconds.
	DurationMs int

	// CharacterCount is the number of characters processed.
	CharacterCount int
}

SynthesisResult contains the result of a TTS synthesis.

type UnloadModelResult added in v0.15.0

type UnloadModelResult struct {
	// Success indicates whether unloading succeeded.
	Success bool

	// MemoryFreedMB is the memory freed by unloading.
	MemoryFreedMB int64
}

UnloadModelResult contains the result of unloading a model.

type Voice

type Voice struct {
	// ID is the provider-specific voice identifier.
	ID string

	// Name is a human-readable name for the voice.
	Name string

	// Language is the BCP-47 language code (e.g., "en-US").
	Language string

	// Gender is the voice gender ("male", "female", "neutral").
	Gender string

	// Provider is the name of the TTS provider.
	Provider string

	// Metadata contains provider-specific additional information.
	Metadata map[string]any
}

Voice represents a voice configuration for TTS.

type VoiceCloner added in v0.15.0

type VoiceCloner interface {
	// CloneVoice creates a voice profile from reference audio.
	// The reference audio should be 5-15 seconds of clear speech.
	// The reference text must accurately transcribe the audio.
	CloneVoice(ctx context.Context, req CloneVoiceRequest) (*VoiceProfile, error)
}

VoiceCloner is implemented by providers that support voice cloning. Voice cloning creates a voice profile from reference audio that can be used for subsequent synthesis.

type VoiceProfile added in v0.15.0

type VoiceProfile struct {
	// ID is the unique identifier for this profile.
	ID string

	// Name is the human-readable name.
	Name string

	// Language is the BCP-47 language code.
	Language string

	// CreatedAt is when the profile was created.
	CreatedAt time.Time

	// Provider is the provider that created this profile.
	Provider string

	// Cached indicates whether the embedding is cached.
	Cached bool

	// EmbeddingSize is the size of the cached embedding in bytes.
	EmbeddingSize int64
}

VoiceProfile represents a cloned voice that can be used for synthesis.

Directories

Path Synopsis
Package providertest provides conformance tests for TTS provider implementations.
Package providertest provides conformance tests for TTS provider implementations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL