Documentation
¶
Overview ¶
Package audio defines the interfaces for pluggable audio processing a transport applies to the media it sends and receives. A Filter transforms incoming audio (noise reduction, say) before it enters the pipeline; a Mixer blends auxiliary audio (background music, say) into the outgoing audio. Concrete implementations live in the audio subpackages (rnnoise, noise, mixer).
Index ¶
- func ALawToPCM(alaw []byte, r *resample.Resampler) []byte
- func InterleaveStereo(left, right []byte) []byte
- func IsSilence(pcm []byte) bool
- func MixAudio(a, b []byte) []byte
- func PCMToALaw(pcm []byte, r *resample.Resampler) []byte
- func PCMToULaw(pcm []byte, r *resample.Resampler) []byte
- func PCMToWAV(pcm []byte, sampleRate, numChannels int) []byte
- func ResamplePCM(pcm []byte, r *resample.Resampler) []byte
- func ULawToPCM(ulaw []byte, r *resample.Resampler) []byte
- type Chain
- type Filter
- type Mixer
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ALawToPCM ¶ added in v0.1.0
ALawToPCM decodes G.711 A-law to 16-bit PCM and converts it to the rate r runs at. A nil r leaves the rate alone. See ULawToPCM.
func InterleaveStereo ¶ added in v0.1.0
InterleaveStereo weaves two mono streams of 16-bit signed PCM into one stereo stream, left channel first (L, R, L, R and so on). The channels are truncated to the shorter of the two, since a frame is only complete when both channels have a sample in it.
func IsSilence ¶ added in v0.1.0
IsSilence reports whether a chunk of 16-bit signed PCM is silence, by comparing the largest absolute sample amplitude against speakingThreshold.
It expects clean speech or true silence: a stream with audible background noise keeps the amplitude above the threshold and never reads as silent. An empty buffer carries no audible sample and so counts as silence.
func MixAudio ¶ added in v0.1.0
MixAudio sums two streams of 16-bit signed PCM sample by sample, clipping the result to the 16-bit range. The streams need not be the same length: the shorter one is treated as though it were padded with silence, so the result is as long as the longer input.
func PCMToALaw ¶ added in v0.1.0
PCMToALaw converts 16-bit PCM to the rate r runs at and encodes it as G.711 A-law. A nil r leaves the rate alone. See ULawToPCM.
func PCMToULaw ¶ added in v0.1.0
PCMToULaw converts 16-bit PCM to the rate r runs at and encodes it as G.711 μ-law. A nil r leaves the rate alone. Resampling comes first, for the reason given on ULawToPCM.
func PCMToWAV ¶ added in v0.1.0
PCMToWAV wraps raw PCM in a WAV container. The samples are expected to be signed 16-bit little-endian, which is what pipelines carry, interleaved by channel.
Trailing bytes that do not complete a frame are dropped. Keeping them would leave the header reporting a frame count that excludes them while the data chunk still carries them, so readers would disagree about the length, and a stereo stream cut mid-frame would come back with its channels swapped.
func ResamplePCM ¶ added in v0.1.0
ResamplePCM converts 16-bit PCM to the rate r runs at, leaving the samples otherwise untouched. A nil r returns the audio unchanged. It is the companding-free member of the family above, for a stream that is already linear PCM and only needs its rate changed.
func ULawToPCM ¶ added in v0.1.0
ULawToPCM decodes G.711 μ-law to 16-bit PCM and converts it to the rate r runs at. A nil r leaves the rate alone.
Decoding comes first: companded bytes are not samples, so resampling them would interpolate between values on a logarithmic scale as though they were linear.
Types ¶
type Chain ¶
type Chain struct {
// contains filtered or unexported fields
}
Chain applies several Filters in sequence, composing multiple input-audio stages — noise reduction, gating, conditioning — into one Filter that a transport can use through Params.AudioInFilter.
func (*Chain) ProcessFrame ¶ added in v0.1.0
ProcessFrame applies a control frame to every filter, returning the last error if any. Each filter decides for itself which controls it recognizes.
type Filter ¶
type Filter interface {
Start(ctx context.Context, sampleRate int) error
Stop(ctx context.Context) error
Filter(ctx context.Context, pcm []byte) ([]byte, error)
ProcessFrame(ctx context.Context, f frames.FilterControlFrame) error
}
Filter transforms input audio before it enters the pipeline. Start is called with the input sample rate when the transport starts, Filter is called for each received chunk of 16-bit mono PCM, and Stop is called when the transport stops. Filter may buffer internally and return fewer (or more) samples than it was given; it returns an empty slice when it has nothing to emit yet. ProcessFrame applies a runtime control frame, so the filter can be retuned or switched off without being torn down.
type Mixer ¶
type Mixer interface {
Start(ctx context.Context, sampleRate int) error
Stop(ctx context.Context) error
Mix(ctx context.Context, pcm []byte) ([]byte, error)
ProcessFrame(ctx context.Context, f frames.MixerControlFrame) error
}
Mixer mixes auxiliary audio (background music or sound effects, say) into a transport's outgoing audio. Start is called with the output sample rate when the transport starts; Mix blends the mixer's current audio into an outgoing chunk of bot audio and returns the result; ProcessFrame applies a runtime control frame; Stop is called when the transport stops.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package dtmf generates the dual-tone multi-frequency signals a telephone keypad produces, as 16-bit mono PCM.
|
Package dtmf generates the dual-tone multi-frequency signals a telephone keypad produces, as 16-bit mono PCM. |
|
Package g711 implements the ITU-T G.711 companding codecs used by telephony media streams (Twilio, Telnyx, Plivo and the wider PSTN): μ-law (PCMU), the North American variant, and A-law (PCMA), used elsewhere.
|
Package g711 implements the ITU-T G.711 companding codecs used by telephony media streams (Twilio, Telnyx, Plivo and the wider PSTN): μ-law (PCMU), the North American variant, and A-law (PCMA), used elsewhere. |
|
Package loudness measures how loud audio is, to ITU-R BS.1770 (the EBU R128 standard), and normalizes the result to a 0..1 scale.
|
Package loudness measures how loud audio is, to ITU-R BS.1770 (the EBU R128 standard), and normalizes the result to a 0..1 scale. |
|
Package mixer mixes background audio into a transport's outgoing audio.
|
Package mixer mixes background audio into a transport's outgoing audio. |
|
noise
|
|
|
gate
Package gate provides a noise gate: an audio.Filter that silences a chunk of audio whose RMS energy falls below a threshold, suppressing low-level background hum and hiss between speech.
|
Package gate provides a noise gate: an audio.Filter that silences a chunk of audio whose RMS energy falls below a threshold, suppressing low-level background hum and hiss between speech. |
|
rnnoise
Package rnnoise is an input audio filter that suppresses background noise with RNNoise, Xiph's recurrent-network denoiser.
|
Package rnnoise is an input audio filter that suppresses background noise with RNNoise, Xiph's recurrent-network denoiser. |
|
Package onset detects the first audible sample in a stream of PCM audio.
|
Package onset detects the first audible sample in a stream of PCM audio. |
|
Package opus wraps the Opus codec for jargo's audio path: it decodes Opus packets to PCM on the way in and encodes PCM to Opus packets on the way out.
|
Package opus wraps the Opus codec for jargo's audio path: it decodes Opus packets to PCM on the way in and encodes PCM to Opus packets on the way out. |
|
Package resample converts interleaved S16LE PCM audio between sample rates, preserving the channel count.
|
Package resample converts interleaved S16LE PCM audio between sample rates, preserving the channel count. |
|
Package turn provides end-of-turn detection: it decides when a user has actually finished speaking, as opposed to merely pausing.
|
Package turn provides end-of-turn detection: it decides when a user has actually finished speaking, as opposed to merely pausing. |
|
Package vad provides voice activity detection: it tells the pipeline when a user is speaking.
|
Package vad provides voice activity detection: it tells the pipeline when a user is speaking. |
|
controller
Package controller drives a voice-activity detector and reports what it hears.
|
Package controller drives a voice-activity detector and reports what it hears. |