Documentation
¶
Overview ¶
Package audiosample reads stretches of a book's audio from an Audiobookshelf server: seconds or minutes from anywhere in the book, as mono PCM at the rate asked for, across its files in play order. It is the groundwork for anything that listens to a book rather than reading its metadata - comparing two copies, transcribing a passage - and it holds the loudness envelope and the matching that compare two stretches.
ffmpeg does the decoding and reads the server's file route itself, seeking with byte ranges, so only the stretch asked for crosses the network. The API key never goes on ffmpeg's command line, where ps would show it: ffmpeg is pointed at a proxy on 127.0.0.1, inside this process, which adds the key.
Index ¶
- Variables
- func Envelope(pcm []int16, window int) []float64
- func SpectralMatch(needle, haystack []int16, rate int, found Found, window int) float64
- func SpectralScore(a, b [][]float64) float64
- func Spectrogram(pcm []int16, rate, frame, hop, bands int) [][]float64
- func Speeds(lo, hi float64, n int) []float64
- type Book
- type Enveloper
- type Found
- type Health
- type Sampler
- func (s *Sampler) BytesRead() int64
- func (s *Sampler) Check(ctx context.Context, itemID, fileID string, off, length float64) (Health, error)
- func (s *Sampler) Close() error
- func (s *Sampler) Read(ctx context.Context, b *Book, start, length float64, rate int) ([]int16, error)
- func (s *Sampler) Stream(ctx context.Context, b *Book, start, length float64, rate int, ...) error
- type Track
Constants ¶
This section is empty.
Variables ¶
var ErrNoFFmpeg = errors.New("ffmpeg is not installed or not on the PATH: reading a book's audio needs it")
ErrNoFFmpeg is returned by New when ffmpeg is not on the PATH.
Functions ¶
func Envelope ¶
Envelope is a stretch's loudness over time: the natural log of the RMS of each window of samples (200 samples at 4 kHz is 50 ms). Speech rises and falls with every syllable and pause, and that pattern survives a new encoding, a new bitrate and a small change of speed, where the samples themselves do not; two narrators reading the same words do not share it.
func SpectralMatch ¶ added in v0.6.0
SpectralMatch is the spectral score of needle against the stretch of haystack that Match found for it, both PCM at rate: found.At is in envelope windows of window samples, and found.Speed is the speed the needle was played at to fit. The needle is taken a chunk at a time, each chunk lined up on its own within a window either side of where the envelope put it, as the envelope's window is coarser than a frame and a speed found to a quarter of a percent drifts half a frame over a chunk. The score is the mean over the chunks.
func SpectralScore ¶ added in v0.6.0
SpectralScore says how alike two spectrograms of the same length are: in each band the energy over time is scaled to mean 0 and deviation 1, which takes out loudness and the encoder's or the microphone's colouring, and the score is the mean of the bands' correlations. 1 is the same shape in every band, 0 no relation. A band flat in either is left out; when all are, the score is 0.
func Spectrogram ¶ added in v0.6.0
Spectrogram is the log energy of pcm in each of bands frequency bands, a row per frame: frames of frame samples every hop samples, Hann windowed, the bands spaced evenly in pitch from spectrumLowHz up to spectrumShare of the rate. Loudness is not taken out here; SpectralScore does that per band.
Types ¶
type Book ¶
Book is an item's audio in play order.
func BookFromFiles ¶ added in v0.6.0
BookFromFiles is a book made of audio files on this machine, in the order given, for a sampler from NewFiles: each file's length comes from ffprobe.
type Enveloper ¶
type Enveloper struct {
// contains filtered or unexported fields
}
Enveloper builds an Envelope a chunk of samples at a time, for a stretch read with Stream.
func NewEnveloper ¶
NewEnveloper starts an envelope of the given window.
type Found ¶
type Found struct {
// Score is the correlation of the needle with the stretch of haystack it
// fits: 1 is the same shape, 0 no relation.
Score float64
// At is where that stretch starts, in haystack samples.
At int
// Speed is the speed the needle was played at to fit: 1.02 means the
// haystack runs 2% faster than the needle's recording.
Speed float64
}
Found is where a needle fits a haystack best.
func Match ¶
Match finds where needle best fits in haystack, both envelopes, trying the needle played at each speed. The score at each offset is the normalised cross-correlation (Pearson's r of the needle against the stretch under it), so a copy recorded louder or quieter scores as high as the original. The correlation at every offset comes from one FFT of the haystack and two per speed, which is what makes searching an hour of envelope for 30 seconds of it, at 25 speeds, take a fraction of a second rather than minutes.
type Health ¶ added in v0.6.0
type Health struct {
Seconds float64 // audio decoded
Complaints int // lines ffmpeg wrote at its error level
First []string // the first few of them
Failed string // why ffmpeg gave up on the file, when it did
}
Health is how a stretch of one file decoded: how much audio came back, and what ffmpeg complained of on the way.
type Sampler ¶
type Sampler struct {
// contains filtered or unexported fields
}
Sampler decodes stretches of books through ffmpeg. It serves ffmpeg the server's file route through a proxy on 127.0.0.1 while it is open, so it must be closed. The proxy answers only under a random path, and only for the items this sampler was asked to read.
func New ¶
New starts a sampler reading through client. It fails with ErrNoFFmpeg when ffmpeg is not installed.
func NewFiles ¶ added in v0.6.0
NewFiles is a sampler over files on this machine rather than a server's: each Track's FileID is a path. It is what the calibration reads a corpus through, and what reads a book before it is on a server. It fails with ErrNoFFmpeg when ffmpeg is not installed.
func (*Sampler) Check ¶ added in v0.6.0
func (s *Sampler) Check(ctx context.Context, itemID, fileID string, off, length float64) (Health, error)
Check decodes length seconds of one file of an item from off seconds in and says how it went. What is wrong with the file - ffmpeg failing on it, complaining of it, or running out of it early - is in the Health; an error is the server or the proxy failing, which says nothing of the file.
func (*Sampler) Read ¶
func (s *Sampler) Read(ctx context.Context, b *Book, start, length float64, rate int) ([]int16, error)
Read decodes length seconds of the book from start seconds in, as mono 16-bit PCM at rate samples a second, crossing from file to file in play order. ffmpeg seeks each file itself (-ss before -i), which over http is a ranged request, so only the stretch and the file's header are fetched; in an mp3 the seek lands where the file's table of contents or its bitrate says, which can be a second or so off. A stretch running past the end of the book comes back short.
func (*Sampler) Stream ¶
func (s *Sampler) Stream(ctx context.Context, b *Book, start, length float64, rate int, fn func(pcm []int16) error) error
Stream is Read handing the samples to fn as ffmpeg decodes them, a chunk at a time, rather than holding the whole stretch: an hour at 4 kHz is 29 MB. fn must not keep the slice past the call. An error from fn stops the read and is returned.