Documentation
¶
Overview ¶
Package audiobuffer records a conversation's audio. A Processor taps the audio flowing through the pipeline — the user's InputAudioRawFrames and the bot's Output/TTS audio — keeps the two tracks time-aligned by padding silence for wall-clock gaps, and delivers the recording through callbacks: a merged mix (mono) or interleaved stereo (user left, bot right), and the separate tracks.
Recording is off until started, either automatically at pipeline start (AutoStart) or by StartRecording / an AudioBufferStartRecordingFrame. It is stopped by StopRecording, an AudioBufferStopRecordingFrame, or pipeline shutdown, which flushes the final audio and fires OnRecordingStopped.
Index ¶
- type Config
- type Processor
- func (p *Processor) Cleanup(ctx context.Context) error
- func (p *Processor) ProcessFrame(ctx context.Context, f frames.Frame, dir processor.Direction) error
- func (p *Processor) Recording() bool
- func (p *Processor) Setup(ctx context.Context, s processor.Setup) error
- func (p *Processor) StartRecording()
- func (p *Processor) StopRecording()
- type TurnAudioData
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// SampleRate overrides the recording sample rate in Hz; 0 uses the pipeline
// output rate from the StartFrame.
SampleRate int
// NumChannels is 1 for a mono mix of user and bot, or 2 for stereo with the
// user on the left and the bot on the right; 0 uses 1.
NumChannels int
// BufferSize is the byte threshold at which OnAudioData and OnTrackAudioData
// fire with the audio accumulated so far; 0 emits only when recording stops.
BufferSize int
// AutoStart begins recording as soon as the pipeline starts.
AutoStart bool
// OnRecordingStarted is called when recording transitions to active.
OnRecordingStarted func()
// OnRecordingStopped is called after recording stops and the final audio has
// been delivered.
OnRecordingStopped func()
// OnAudioData receives the merged recording (mono mix or interleaved stereo)
// at each BufferSize boundary and once more on stop.
OnAudioData func(audio []byte, sampleRate, numChannels int)
// OnTrackAudioData receives the separate, time-aligned user and bot tracks at
// each BufferSize boundary and once more on stop.
OnTrackAudioData func(user, bot []byte, sampleRate, numChannels int)
// EnableTurnAudio delivers each turn's audio on its own, as that turn ends,
// through OnUserTurnAudio and OnBotTurnAudio. It is off by default: the
// session tracks are what a recording wants, and a turn's audio is for
// something that works turn by turn, scoring one utterance or handing it to a
// classifier. The session tracks cannot be cut into turns afterwards, since
// nothing in them marks where a turn began.
//
// It needs the worker to be tracking turns, which it does by default, since
// the turn tracker is what says where a turn ended.
EnableTurnAudio bool
// OnUserTurnAudio receives everything the user said during one turn, mono,
// once that turn ends.
OnUserTurnAudio func(d TurnAudioData)
// OnBotTurnAudio receives everything the bot said during one turn, mono, once
// that turn ends.
OnBotTurnAudio func(d TurnAudioData)
// OnUserTurnAudioData receives one run of the user's speech, mono, each time
// they stop speaking. A turn can hold several runs, so it fires more than
// once per turn and says nothing about which turn the audio belongs to; use
// OnUserTurnAudio for that.
OnUserTurnAudioData func(audio []byte, sampleRate, numChannels int)
// OnBotTurnAudioData receives one run of the bot's speech, mono, each time it
// stops speaking. See OnUserTurnAudioData.
OnBotTurnAudioData func(audio []byte, sampleRate, numChannels int)
}
Config configures a recording Processor.
type Processor ¶
Processor records the pipeline's audio. It is placed downstream, typically after the output transport, so it sees both the user's input audio and the bot's output audio.
func (*Processor) ProcessFrame ¶
func (p *Processor) ProcessFrame(ctx context.Context, f frames.Frame, dir processor.Direction) error
ProcessFrame taps audio for recording and forwards every frame untouched.
func (*Processor) Setup ¶ added in v0.1.0
Setup resolves the rate the recording is kept at. A rate configured on the processor wins; otherwise it takes the pipeline's output rate, which it knows from the moment it is set up.
func (*Processor) StartRecording ¶
func (p *Processor) StartRecording()
StartRecording begins recording. It does nothing if already recording.
func (*Processor) StopRecording ¶
func (p *Processor) StopRecording()
StopRecording flushes the buffered audio and stops recording. It does nothing if not recording.
type TurnAudioData ¶ added in v0.1.0
type TurnAudioData struct {
// TurnNumber is the turn this audio belongs to.
TurnNumber int
// Audio is everything the speaker said during the turn, as raw PCM, with the
// pauses between their runs of speech left out.
Audio []byte
// SampleRate is the rate of the audio, in Hz.
SampleRate int
// NumChannels is how many channels the audio has.
NumChannels int
}
TurnAudioData is one speaker's audio for one conversation turn.