audiobuffer

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 30, 2026 License: BSD-2-Clause Imports: 8 Imported by: 0

Documentation

Overview

Package audiobuffer records a conversation's audio. A Processor taps the audio flowing through the pipeline — the user's InputAudioRawFrames and the bot's Output/TTS audio — keeps the two tracks time-aligned by padding silence for wall-clock gaps, and delivers the recording through callbacks: a merged mix (mono) or interleaved stereo (user left, bot right), and the separate tracks.

Recording is off until started, either automatically at pipeline start (AutoStart) or by StartRecording / an AudioBufferStartRecordingFrame. It is stopped by StopRecording, an AudioBufferStopRecordingFrame, or pipeline shutdown, which flushes the final audio and fires OnRecordingStopped.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config

type Config struct {
	// SampleRate overrides the recording sample rate in Hz; 0 uses the pipeline
	// output rate from the StartFrame.
	SampleRate int
	// NumChannels is 1 for a mono mix of user and bot, or 2 for stereo with the
	// user on the left and the bot on the right; 0 uses 1.
	NumChannels int
	// BufferSize is the byte threshold at which OnAudioData and OnTrackAudioData
	// fire with the audio accumulated so far; 0 emits only when recording stops.
	BufferSize int
	// AutoStart begins recording as soon as the pipeline starts.
	AutoStart bool

	// OnRecordingStarted is called when recording transitions to active.
	OnRecordingStarted func()
	// OnRecordingStopped is called after recording stops and the final audio has
	// been delivered.
	OnRecordingStopped func()
	// OnAudioData receives the merged recording (mono mix or interleaved stereo)
	// at each BufferSize boundary and once more on stop.
	OnAudioData func(audio []byte, sampleRate, numChannels int)
	// OnTrackAudioData receives the separate, time-aligned user and bot tracks at
	// each BufferSize boundary and once more on stop.
	OnTrackAudioData func(user, bot []byte, sampleRate, numChannels int)

	// EnableTurnAudio delivers each turn's audio on its own, as that turn ends,
	// through OnUserTurnAudio and OnBotTurnAudio. It is off by default: the
	// session tracks are what a recording wants, and a turn's audio is for
	// something that works turn by turn, scoring one utterance or handing it to a
	// classifier. The session tracks cannot be cut into turns afterwards, since
	// nothing in them marks where a turn began.
	//
	// It needs the worker to be tracking turns, which it does by default, since
	// the turn tracker is what says where a turn ended.
	EnableTurnAudio bool
	// OnUserTurnAudio receives everything the user said during one turn, mono,
	// once that turn ends.
	OnUserTurnAudio func(d TurnAudioData)
	// OnBotTurnAudio receives everything the bot said during one turn, mono, once
	// that turn ends.
	OnBotTurnAudio func(d TurnAudioData)

	// OnUserTurnAudioData receives one run of the user's speech, mono, each time
	// they stop speaking. A turn can hold several runs, so it fires more than
	// once per turn and says nothing about which turn the audio belongs to; use
	// OnUserTurnAudio for that.
	OnUserTurnAudioData func(audio []byte, sampleRate, numChannels int)
	// OnBotTurnAudioData receives one run of the bot's speech, mono, each time it
	// stops speaking. See OnUserTurnAudioData.
	OnBotTurnAudioData func(audio []byte, sampleRate, numChannels int)
}

Config configures a recording Processor.

type Processor

type Processor struct {
	*processor.Base
	// contains filtered or unexported fields
}

Processor records the pipeline's audio. It is placed downstream, typically after the output transport, so it sees both the user's input audio and the bot's output audio.

func New

func New(cfg Config) *Processor

New builds a recording Processor.

func (*Processor) Cleanup

func (p *Processor) Cleanup(ctx context.Context) error

Cleanup releases the resamplers after the processor stops.

func (*Processor) ProcessFrame

func (p *Processor) ProcessFrame(ctx context.Context, f frames.Frame, dir processor.Direction) error

ProcessFrame taps audio for recording and forwards every frame untouched.

func (*Processor) Recording

func (p *Processor) Recording() bool

Recording reports whether recording is active.

func (*Processor) Setup added in v0.1.0

func (p *Processor) Setup(ctx context.Context, s processor.Setup) error

Setup resolves the rate the recording is kept at. A rate configured on the processor wins; otherwise it takes the pipeline's output rate, which it knows from the moment it is set up.

func (*Processor) StartRecording

func (p *Processor) StartRecording()

StartRecording begins recording. It does nothing if already recording.

func (*Processor) StopRecording

func (p *Processor) StopRecording()

StopRecording flushes the buffered audio and stops recording. It does nothing if not recording.

type TurnAudioData added in v0.1.0

type TurnAudioData struct {
	// TurnNumber is the turn this audio belongs to.
	TurnNumber int
	// Audio is everything the speaker said during the turn, as raw PCM, with the
	// pauses between their runs of speech left out.
	Audio []byte
	// SampleRate is the rate of the audio, in Hz.
	SampleRate int
	// NumChannels is how many channels the audio has.
	NumChannels int
}

TurnAudioData is one speaker's audio for one conversation turn.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL