tts

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 30, 2026 License: Apache-2.0 Imports: 17 Imported by: 0

Documentation

Index

Examples

Constants

View Source 
const (
	ProviderCartesia  = "cartesia"
	ProviderInworld   = "inworld"
	ProviderXAI       = "xai"
	ProviderFishAudio = "fishaudio"
)
View Source 
const DefaultMood = MoodCalm

Variables

View Source 
var DefaultSpeechSteeringOptions = SpeechSteeringOptions{Disfluencies: ToggleEnabled}

DefaultSpeechSteeringOptions enables light fillers and the full sound vocabulary. It is a value, so callers can copy it without shared mutation.

View Source 
var ErrFallbackBufferLimit = errors.New("TTS fallback replay buffer limit exceeded")

ErrFallbackBufferLimit means streamed input could not be retained safely for a possible failover. The stream terminates instead of growing without bound.

Functions

func ConvertExpressionTags 

func ConvertExpressionTags(text string) string

ConvertExpressionTags converts framework-standard expression and sound XML tags to square-bracket provider cues.

func ConvertMarkup 

func ConvertMarkup(provider, text string) string

ConvertMarkup lowers expr markers and framework-standard markup to a provider's native dialect.

Example 
package main

import (
	"fmt"

	"github.com/infinityscroll/livekit-agents-go/tts"
)

func main() {
	text := `<expr type="sound" label="laugh"/> That was a good one.`
	fmt.Println(tts.ConvertMarkup(tts.ProviderXAI, text))
}

Output:
[laugh] That was a good one.

func DropBracketCues 

func DropBracketCues(tokens []agents.TimedString, held *[]agents.TimedString, final bool) []agents.TimedString

DropBracketCues removes provider-native square-bracket cues while preserving all timing metadata. held carries an unfinished cue between calls.

func ExpressionAttribute 

func ExpressionAttribute(tags []ExpressiveTag) map[string]string

ExpressionAttribute builds the lk.expression transcription attribute.

func HasMarkupDialect 

func HasMarkupDialect(provider string) bool

HasMarkupDialect reports whether provider supports the shared expr dialect. It performs no prompt rendering or allocation.

func LLMInstructions 

func LLMInstructions(provider string, steering *SpeechSteeringOptions) (instructions string, ok bool)

LLMInstructions returns the provider's expr guide. ok is false for providers without a markup dialect. A nil steering pointer selects the provider default.

func LLMInstructionsText 

func LLMInstructionsText(provider string, steering *SpeechSteeringOptions) string

LLMInstructionsText is a convenience for call sites where an unsupported provider naturally maps to an empty instruction string.

func MaxInputLen 

func MaxInputLen(provider string) (int, bool)

MaxInputLen returns the provider hard chunk limit and whether one is known.

func NormalizeMarkup 

func NormalizeMarkup(provider, text string) string

NormalizeMarkup closes provider self-closing tags commonly left open by LLMs.

func SentenceTokenizer 

func SentenceTokenizer(provider string, expressive bool) *tokenize.SentenceTokenizer

SentenceTokenizer returns the default streamed-input tokenizer for provider.

func SteeringInstructions 

func SteeringInstructions(provider string, steering SpeechSteeringOptions) string

SteeringInstructions renders only overrides that differ from the default.

func StripAllMarkup 

func StripAllMarkup(text string) string

StripAllMarkup is SplitAllMarkup with tags discarded.

func StripExprMarkup 

func StripExprMarkup(text string) string

StripExprMarkup removes only the framework expr dialect.

func SupportedNonverbals 

func SupportedNonverbals(provider string) map[NonverbalField][]string

SupportedNonverbals returns a defensive copy of the provider capability map.

Types

type AgentMood 

type AgentMood string

AgentMood is the stable, provider-independent delivery mood. 

const (
	MoodExcited    AgentMood = "excited"
	MoodHappy      AgentMood = "happy"
	MoodPlayful    AgentMood = "playful"
	MoodCurious    AgentMood = "curious"
	MoodSurprised  AgentMood = "surprised"
	MoodHopeful    AgentMood = "hopeful"
	MoodEmpathetic AgentMood = "empathetic"
	MoodSad        AgentMood = "sad"
	MoodAngry      AgentMood = "angry"
	MoodAnxious    AgentMood = "anxious"
	MoodCalm       AgentMood = "calm"
)

func MatchMood 

func MatchMood(label string) AgentMood

MatchMood normalizes an open-ended provider delivery label.

func MatchMoodOK 

func MatchMoodOK(label string) (AgentMood, bool)

MatchMoodOK reports whether a non-fallback mood was recognized.

func MatchMoodOr 

func MatchMoodOr(label string, fallback AgentMood) AgentMood

MatchMoodOr returns fallback when no keyword matches.

func MoodPriority 

func MoodPriority() []AgentMood

MoodPriority returns the tie-break order, most specific first.

type AvailabilityChangedEvent 

type AvailabilityChangedEvent struct {
	TTS       TTS
	Available bool
}

AvailabilityChangedEvent reports a TTS provider entering or leaving the fallback rotation.

type Base 

type Base struct {
	// contains filtered or unexported fields
}

func NewBase 

func NewBase(label, provider, model string, sampleRate, channels int, capabilities Capabilities) *Base

func (*Base) Capabilities 

func (b *Base) Capabilities() Capabilities

func (*Base) Channels 

func (b *Base) Channels() int

func (*Base) EmitError 

func (b *Base) EmitError(event ErrorEvent)

func (*Base) EmitMetrics 

func (b *Base) EmitMetrics(metric metrics.TTS)

func (*Base) Expressive 

func (b *Base) Expressive() bool

func (*Base) Label 

func (b *Base) Label() string

func (*Base) MarkupProviderKey 

func (b *Base) MarkupProviderKey() string

func (*Base) Model 

func (b *Base) Model() string

func (*Base) OnError 

func (b *Base) OnError(fn func(ErrorEvent)) func()

func (*Base) OnMetrics 

func (b *Base) OnMetrics(fn func(metrics.TTS)) func()

func (*Base) Provider 

func (b *Base) Provider() string

func (*Base) SampleRate 

func (b *Base) SampleRate() int

func (*Base) SetExpressive 

func (b *Base) SetExpressive(enabled bool)

func (*Base) SetMarkupProviderKey 

func (b *Base) SetMarkupProviderKey(key string)

func (*Base) SetModel 

func (b *Base) SetModel(model string)

type BaseChunkedStream 

type BaseChunkedStream struct {
	// contains filtered or unexported fields
}

func NewBaseChunkedStream 

func NewBaseChunkedStream(parent context.Context, capacity int) *BaseChunkedStream

func (*BaseChunkedStream) Close 

func (s *BaseChunkedStream) Close() error

func (*BaseChunkedStream) Context 

func (s *BaseChunkedStream) Context() context.Context

func (*BaseChunkedStream) Emit 

func (s *BaseChunkedStream) Emit(ctx context.Context, audio SynthesizedAudio) error

func (*BaseChunkedStream) Finish 

func (s *BaseChunkedStream) Finish(err error)

func (*BaseChunkedStream) Recv 

func (s *BaseChunkedStream) Recv(ctx context.Context) (SynthesizedAudio, error)

type BaseSynthesizeStream 

type BaseSynthesizeStream struct {
	// contains filtered or unexported fields
}

func NewBaseSynthesizeStream 

func NewBaseSynthesizeStream(parent context.Context, capacity int) *BaseSynthesizeStream

func (*BaseSynthesizeStream) Close 

func (s *BaseSynthesizeStream) Close() error

func (*BaseSynthesizeStream) Context 

func (s *BaseSynthesizeStream) Context() context.Context

func (*BaseSynthesizeStream) Emit 

func (s *BaseSynthesizeStream) Emit(ctx context.Context, audio SynthesizedAudio) error

func (*BaseSynthesizeStream) EndInput 

func (s *BaseSynthesizeStream) EndInput() error

func (*BaseSynthesizeStream) Finish 

func (s *BaseSynthesizeStream) Finish(err error)

func (*BaseSynthesizeStream) Flush 

func (s *BaseSynthesizeStream) Flush(ctx context.Context) error

func (*BaseSynthesizeStream) Inputs 

func (s *BaseSynthesizeStream) Inputs() stream.Reader[StreamInput]

func (*BaseSynthesizeStream) PushText 

func (s *BaseSynthesizeStream) PushText(ctx context.Context, text string) error

func (*BaseSynthesizeStream) Recv 

func (s *BaseSynthesizeStream) Recv(ctx context.Context) (SynthesizedAudio, error)

type BracketCueStripper 

type BracketCueStripper struct {
	// contains filtered or unexported fields
}

BracketCueStripper owns the held state needed by DropBracketCues.

func (*BracketCueStripper) Flush 

func (s *BracketCueStripper) Flush() []agents.TimedString

func (*BracketCueStripper) Push 

func (s *BracketCueStripper) Push(tokens []agents.TimedString) []agents.TimedString

type Capabilities 

type Capabilities struct {
	Streaming         bool
	AlignedTranscript bool
}

type ChunkedStream 

type ChunkedStream interface {
	stream.Reader[SynthesizedAudio]
	Close() error
}

type ErrorEvent 

type ErrorEvent struct {
	Timestamp   time.Time
	Label       string
	Err         error
	Recoverable bool
}

type ExpressiveTag 

type ExpressiveTag struct {
	Type  string
	Value string
}

ExpressiveTag is one expressive XML tag removed from a transcript.

func ExtractAndStrip 

func ExtractAndStrip(text string, xmlTags []string, attributeTags ...[]string) (string, []ExpressiveTag)

ExtractAndStrip removes selected XML tags and returns their semantic values. Attribute-tag names prefer their first quoted attribute over wrapped text.

func SplitAllMarkup 

func SplitAllMarkup(text string) (string, []ExpressiveTag)

SplitAllMarkup strips expr and every supported provider-native XML tag.

type FallbackAdapter 

type FallbackAdapter struct {
	// contains filtered or unexported fields
}

FallbackAdapter provides ordered TTS failover, background recovery, automatic streaming wrappers, sample-rate normalization, and dynamic model attribution.

func NewFallbackAdapter 

func NewFallbackAdapter(options FallbackOptions) (*FallbackAdapter, error)

func (*FallbackAdapter) Availability 

func (a *FallbackAdapter) Availability() []bool

func (*FallbackAdapter) Capabilities 

func (a *FallbackAdapter) Capabilities() Capabilities

func (*FallbackAdapter) Channels 

func (a *FallbackAdapter) Channels() int

func (*FallbackAdapter) Close 

func (a *FallbackAdapter) Close(ctx context.Context) error

func (*FallbackAdapter) Instances 

func (a *FallbackAdapter) Instances() []TTS

func (*FallbackAdapter) Label 

func (a *FallbackAdapter) Label() string

func (*FallbackAdapter) MaxRetriesPerTTS 

func (a *FallbackAdapter) MaxRetriesPerTTS() int

func (*FallbackAdapter) Model 

func (a *FallbackAdapter) Model() string

func (*FallbackAdapter) OnAvailabilityChanged 

func (a *FallbackAdapter) OnAvailabilityChanged(fn func(AvailabilityChangedEvent)) func()

func (*FallbackAdapter) OnError 

func (a *FallbackAdapter) OnError(fn func(ErrorEvent)) func()

func (*FallbackAdapter) OnMetrics 

func (a *FallbackAdapter) OnMetrics(fn func(metrics.TTS)) func()

func (*FallbackAdapter) Provider 

func (a *FallbackAdapter) Provider() string

func (*FallbackAdapter) RecoveryDelay 

func (a *FallbackAdapter) RecoveryDelay() time.Duration

func (*FallbackAdapter) RetryOnChunkSent 

func (a *FallbackAdapter) RetryOnChunkSent() bool

func (*FallbackAdapter) SampleRate 

func (a *FallbackAdapter) SampleRate() int

func (*FallbackAdapter) Status 

func (a *FallbackAdapter) Status() []FallbackStatus

func (*FallbackAdapter) Stream 

func (a *FallbackAdapter) Stream(parent context.Context, options StreamOptions) (SynthesizeStream, error)

func (*FallbackAdapter) Synthesize 

func (a *FallbackAdapter) Synthesize(parent context.Context, text string, options SynthesizeOptions) (ChunkedStream, error)

func (*FallbackAdapter) TTSs 

func (a *FallbackAdapter) TTSs() []TTS

type FallbackOptions 

type FallbackOptions struct {
	TTSs []TTS
	// TTSInstances is the agents-js-compatible spelling. Set only one of
	// TTSs or TTSInstances.
	TTSInstances     []TTS
	MaxRetriesPerTTS int
	// MaxRetryPerTTS is the agents-js-compatible spelling. Non-zero values
	// override MaxRetriesPerTTS.
	MaxRetryPerTTS   int
	RecoveryDelay    time.Duration
	RetryOnChunkSent bool
	// RetryOnChunk is a concise compatibility alias for RetryOnChunkSent.
	RetryOnChunk         bool
	StreamCapacity       int
	MaxBufferedTextBytes int
	KeepProvidersOpen    bool
}

FallbackOptions configures ordered TTS failover. MaxRetriesPerTTS defaults to two; use a negative value to disable child retries explicitly.

type FallbackStatus 

type FallbackStatus struct {
	Available  bool
	Recovering bool
}

FallbackStatus is an immutable provider-health snapshot.

type MarkupInfo 

type MarkupInfo struct {
	Nonverbals map[NonverbalField][]string
}

MarkupInfo describes the expressive features supported by a provider.

func ProviderMarkupInfo 

func ProviderMarkupInfo(provider string) MarkupInfo

ProviderMarkupInfo returns the queryable markup matrix for a provider.

type NonverbalField 

type NonverbalField string

NonverbalField names a queryable sound category. 

const (
	NonverbalLaughing     NonverbalField = "laughing"
	NonverbalBreathing    NonverbalField = "breathing"
	NonverbalSighing      NonverbalField = "sighing"
	NonverbalCrying       NonverbalField = "crying"
	NonverbalVocalizing   NonverbalField = "vocalizing"
	NonverbalMouthSounds  NonverbalField = "mouthSounds"
	NonverbalReflexSounds NonverbalField = "reflexSounds"
)

type NonverbalOptions 

type NonverbalOptions struct {
	All          Toggle
	Laughing     Toggle
	Breathing    Toggle
	Sighing      Toggle
	Crying       Toggle
	Vocalizing   Toggle
	MouthSounds  Toggle
	ReflexSounds Toggle
}

NonverbalOptions is a sparse opt-out. Unspecified categories remain enabled. All corresponds to the JS boolean form of nonverbalSounds.

type Pace 

type Pace string

Pace controls the requested overall delivery pace. 

const (
	PaceDefault Pace = ""
	PaceSlow    Pace = "slow"
	PaceNormal  Pace = "normal"
	PaceFast    Pace = "fast"
)

type RequestMetrics 

type RequestMetrics struct {
	// contains filtered or unexported fields
}

RequestMetrics tracks one synthesis attempt using monotonic time.

func BeginRequest 

func BeginRequest(base *Base, requestID, segmentID, text string, streamed bool) *RequestMetrics

func (*RequestMetrics) AddAudio 

func (r *RequestMetrics) AddAudio(frame agents.AudioFrame)

func (*RequestMetrics) Finish 

func (r *RequestMetrics) Finish(cancelled bool, inputTokens, outputTokens int64)

type SpeechSteeringOptions 

type SpeechSteeringOptions struct {
	Disfluencies    Toggle
	NonverbalSounds NonverbalOptions
	Pace            Pace
}

SpeechSteeringOptions steers verbal delivery and non-verbal sounds.

type StreamAdapter 

type StreamAdapter struct {
	// contains filtered or unexported fields
}

StreamAdapter converts a batch TTS into a streaming TTS by incrementally tokenizing complete sentences. The wrapped TTS remains caller-owned.

func NewStreamAdapter 

func NewStreamAdapter(wrapped TTS, sentenceTokenizer *tokenize.SentenceTokenizer, options ...StreamAdapterOptions) (*StreamAdapter, error)

NewStreamAdapter constructs a sentence-streaming wrapper. Passing a nil tokenizer selects the built-in tokenizer with formatting retained.

func (*StreamAdapter) Capabilities 

func (a *StreamAdapter) Capabilities() Capabilities

func (*StreamAdapter) Channels 

func (a *StreamAdapter) Channels() int

func (*StreamAdapter) Close 

func (a *StreamAdapter) Close(ctx context.Context) error

Close stops active wrapper streams and detaches child listeners. It does not close the caller-owned wrapped TTS.

func (*StreamAdapter) Label 

func (a *StreamAdapter) Label() string

func (*StreamAdapter) Model 

func (a *StreamAdapter) Model() string

func (*StreamAdapter) OnError 

func (a *StreamAdapter) OnError(fn func(ErrorEvent)) func()

func (*StreamAdapter) OnMetrics 

func (a *StreamAdapter) OnMetrics(fn func(metrics.TTS)) func()

func (*StreamAdapter) Provider 

func (a *StreamAdapter) Provider() string

func (*StreamAdapter) SampleRate 

func (a *StreamAdapter) SampleRate() int

func (*StreamAdapter) Stream 

func (a *StreamAdapter) Stream(parent context.Context, options StreamOptions) (SynthesizeStream, error)

func (*StreamAdapter) Synthesize 

func (a *StreamAdapter) Synthesize(ctx context.Context, text string, options SynthesizeOptions) (ChunkedStream, error)

func (*StreamAdapter) WrappedTTS 

func (a *StreamAdapter) WrappedTTS() TTS

type StreamAdapterOptions 

type StreamAdapterOptions struct {
	StreamCapacity int
}

StreamAdapterOptions controls the bounded queues used by StreamAdapter.

type StreamAdapterWrapper 

type StreamAdapterWrapper struct {
	*BaseSynthesizeStream
	// contains filtered or unexported fields
}

StreamAdapterWrapper is the concrete sentence-backed synthesis stream.

func (*StreamAdapterWrapper) Close 

func (s *StreamAdapterWrapper) Close() error

func (*StreamAdapterWrapper) Wait 

func (s *StreamAdapterWrapper) Wait(ctx context.Context) error

type StreamInput 

type StreamInput struct {
	Text  string
	Flush bool
}

type StreamOptions 

type StreamOptions struct {
	ConnectOptions agents.APIConnectOptions
}

type SynthesizeOptions 

type SynthesizeOptions struct {
	ConnectOptions agents.APIConnectOptions
}

type SynthesizeStream 

type SynthesizeStream interface {
	stream.Reader[SynthesizedAudio]
	PushText(context.Context, string) error
	Flush(context.Context) error
	EndInput() error
	Close() error
}

type SynthesizedAudio 

type SynthesizedAudio struct {
	RequestID        string
	SegmentID        string
	Frame            agents.AudioFrame
	DeltaText        string
	Final            bool
	TimedTranscripts []agents.TimedString
}

type TTS 

type TTS interface {
	Label() string
	Provider() string
	Model() string
	SampleRate() int
	Channels() int
	Capabilities() Capabilities
	Synthesize(context.Context, string, SynthesizeOptions) (ChunkedStream, error)
	Stream(context.Context, StreamOptions) (SynthesizeStream, error)
	Close(context.Context) error
	OnMetrics(func(metrics.TTS)) func()
	OnError(func(ErrorEvent)) func()
}

type Toggle 

type Toggle uint8

Toggle is a tri-state sparse override used by speech steering. 

const (
	ToggleDefault Toggle = iota
	ToggleEnabled
	ToggleDisabled
)

type TranscriptMarkupStripper 

type TranscriptMarkupStripper struct {
	// contains filtered or unexported fields
}

TranscriptMarkupStripper incrementally strips provider-agnostic markup. Its retained tail is bounded so malformed input cannot grow memory indefinitely.

func (*TranscriptMarkupStripper) ExpressionAttribute 

func (s *TranscriptMarkupStripper) ExpressionAttribute() map[string]string

ExpressionAttribute returns the current lk.expression attribute.

func (*TranscriptMarkupStripper) Flush 

func (s *TranscriptMarkupStripper) Flush() string

Flush drains any held fragment at segment end.

func (*TranscriptMarkupStripper) Push 

func (s *TranscriptMarkupStripper) Push(text string) string

Push consumes one streamed text chunk.

func (*TranscriptMarkupStripper) Tags 

func (s *TranscriptMarkupStripper) Tags() []ExpressiveTag

Tags returns a defensive copy of stripped tags in observed order.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.