README
¶
audio
What - the audio capability: recording and transcribing speech with Whisper, and reading the length of a WAV.
Why - recording and transcribing run local binaries and model files with their own downloads and timeouts, none of which belong in the agent loop.
How - the container wires the recorder and transcriber into the voice shortcut, and TextToSpeech reports the length of the WAV the gateway returned.
How it plugs in
speech_to_textenables the voice shortcut and the transcription of Telegram voice messages. The Whisper model is downloaded on first use.- Speech synthesis is not here:
TextToSpeechgoes through the agent's speech service to the gateway. - Music, sound effects and video are not here: those tools call the gateway through the agent's media ports.
Related
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func WAVDurationSeconds ¶ added in v0.184.0
WAVDurationSeconds returns the duration of a RIFF/WAVE file in seconds by reading its fmt and data chunk headers. It lets callers report the duration of synthesized speech without external tools.
Types ¶
type Converter ¶
type Converter struct {
// contains filtered or unexported fields
}
Converter converts arbitrary audio files into Whisper-ready 16kHz mono WAV. It is used to turn Telegram voice notes (OGG/Opus) into a format whisper.cpp can read.
func NewConverter ¶
func NewConverter(cfg config.SpeechToTextConfig) *Converter
NewConverter creates a Converter from the speech-to-text config.
func (*Converter) SetBinaryEnsurer ¶ added in v0.148.0
func (c *Converter) SetBinaryEnsurer(f func(ctx context.Context, name binariesdomain.Name) (string, error))
SetBinaryEnsurer installs a fallback that resolves (downloading if needed) a named binary when it is not found on PATH.
type FileTranscriber ¶ added in v0.178.0
type FileTranscriber struct {
// contains filtered or unexported fields
}
FileTranscriber transcribes an arbitrary audio file by first converting it to 16kHz mono WAV with ffmpeg, then running Whisper. It is used for Telegram voice messages (OGG/Opus), which must be decoded before transcription.
func NewFileTranscriber ¶ added in v0.178.0
func NewFileTranscriber(cfg config.SpeechToTextConfig) *FileTranscriber
NewFileTranscriber creates a FileTranscriber from the speech-to-text config.
func (*FileTranscriber) TranscribeFile ¶ added in v0.178.0
TranscribeFile converts audioPath to WAV and returns its transcription. The intermediate WAV file is removed before returning.
type ModelStore ¶ added in v0.207.0
type ModelStore struct {
// contains filtered or unexported fields
}
ModelStore resolves and (optionally) downloads the GGML model file.
func NewModelStore ¶ added in v0.207.0
func NewModelStore(cfg config.SpeechToTextConfig) *ModelStore
NewModelStore creates a ModelStore from the speech-to-text config.
func (*ModelStore) EnsureModel ¶ added in v0.207.0
func (m *ModelStore) EnsureModel(ctx context.Context) (string, error)
EnsureModel returns the local path to the model file, downloading it on first use when AutoDownload is enabled. Concurrent callers are serialized so a cold cache triggers one download.
type Recorder ¶
type Recorder struct {
// contains filtered or unexported fields
}
Recorder captures microphone audio into a 16kHz mono WAV file by shelling out to ffmpeg (with arecord/sox fallbacks on Linux), mirroring the candidate-list pattern used by the clipboard text writer. It adds no CGO.
func NewRecorder ¶
func NewRecorder(cfg config.SpeechToTextConfig) *Recorder
NewRecorder creates a Recorder from the speech-to-text config.
func (*Recorder) EnsureAvailable ¶
EnsureAvailable reports whether a microphone capture tool can be resolved (possibly by downloading ffmpeg), without recording. It lets callers fail fast (with an actionable error) before prompting the user to speak.
func (*Recorder) Record ¶
Record captures up to maxSeconds of microphone audio and returns the path to a 16kHz mono WAV file. When speech_to_text.silence_timeout is set, ffmpeg recordings stop shortly after the speaker goes quiet instead of always running for the full cap. The caller owns the returned file and should remove it.
type WhisperTranscriber ¶ added in v0.178.0
type WhisperTranscriber struct {
// contains filtered or unexported fields
}
WhisperTranscriber transcribes a 16kHz mono WAV file using whisper.cpp.
func NewWhisperTranscriber ¶ added in v0.178.0
func NewWhisperTranscriber(cfg config.SpeechToTextConfig) *WhisperTranscriber
NewWhisperTranscriber creates a transcriber from the speech-to-text config.
func (*WhisperTranscriber) EnsureAvailable ¶ added in v0.178.0
func (w *WhisperTranscriber) EnsureAvailable() error
EnsureAvailable reports whether the whisper binary can be resolved (possibly by downloading it), without transcribing or downloading a model. It lets callers fail fast (with an actionable install hint) before recording audio.
func (*WhisperTranscriber) Transcribe ¶ added in v0.178.0
Transcribe converts the audio at wavPath (16kHz mono WAV) into text.
Source Files