capture

package
v0.67.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 3, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package capture is the public microphone / system-audio capture layer of the SpeechKit framework: backend registry (RegisterBackend, Open), capture Session contract, device enumeration (ListCaptureDevices, ListOutputDevices), and the pooled PCM frame buffers (FramePool).

Backends today: Windows/WASAPI via malgo, compiled in behind the `windows && cgo` build tags. On every other build (non-Windows, or CGO_ENABLED=0) Open and NewCapturer return an error wrapping ErrBackendUnavailable. RegisterBackend is the extension point for additional platform backends.

Captured PCM uses the canonical SpeechKit format (16 kHz, 16-bit signed, mono) declared in pkg/speechkit/audio; a Session structurally satisfies pkg/speechkit's AudioRecorder (and, via SetPooledPCMHandler, its PooledPCMRecorder optimisation).

Playback (TTS/voice-agent output) is intentionally not part of this package; the reference app keeps it in internal/audio.

Index

Constants

View Source
const DefaultFrameCapacity = 4096

DefaultFrameCapacity is the per-buffer capacity returned by the package-level frame pool. It is sized for the worst-case malgo callback chunk we have observed in production (16 kHz mono S16 at 32 ms FrameSizeMs → 1024 bytes; doubled for a generous safety margin).

Variables

View Source
var (
	ErrUnsupportedBackend      = errors.New("unsupported audio backend")
	ErrBackendUnavailable      = errors.New("audio backend unavailable in this build")
	ErrUnsupportedSource       = errors.New("unsupported audio input source")
	ErrOutputDeviceUnavailable = errors.New("audio output device unavailable")
)
View Source
var OnCaptureDeviceRebound func(oldID, newID, name string)

OnCaptureDeviceRebound, when set, is called after a configured capture device ID was not found but the device was recovered via its persisted name (USB/UAC re-enumeration). The host app can persist the new ID.

Functions

func Get

func Get() []byte

Get returns a recyclable buffer from the default pool.

func Put

func Put(buf []byte)

Put returns a buffer to the default pool.

func RegisterBackend

func RegisterBackend(name Backend, factory Factory) error

Types

type Backend

type Backend string
const (
	BackendAuto                Backend = "auto"
	BackendWindowsWASAPIMalgo  Backend = "windows-wasapi-malgo"
	BackendWindowsWASAPINative Backend = "windows-wasapi-native"
)

type Capturer

type Capturer = Session

Capturer is kept as an alias while the app migrates to the session terminology.

func NewCapturer

func NewCapturer() (Capturer, error)

func NewCapturerWithConfig

func NewCapturerWithConfig(cfg Config) (Capturer, error)

type Config

type Config struct {
	Backend     Backend
	InputSource InputSource
	DeviceID    string
	// DeviceName is the human-readable name of the capture device. USB/UAC
	// devices can re-enumerate with a new WASAPI endpoint ID after a replug
	// or firmware update; when DeviceID no longer matches, the capturer
	// falls back to matching by this name.
	DeviceName     string
	OutputDeviceID string
	SampleRate     int
	Channels       int
	FrameSizeMs    int
	LatencyHint    string
	// CaptureThreadPriority selects the OS priority of the backend's
	// audio worker thread: "realtime" (default, TIME_CRITICAL — safe
	// because the callback only memcpys and enqueues) or "highest"
	// (malgo's own default). Ignored by backends without native thread
	// control.
	CaptureThreadPriority string
}

type DeviceInfo

type DeviceInfo struct {
	ID        string `json:"deviceId"`
	Name      string `json:"label"`
	IsDefault bool   `json:"isDefault"`
}

DeviceInfo describes a capture device that can be presented to the user.

func ListCaptureDevices

func ListCaptureDevices(cfg Config) ([]DeviceInfo, error)

ListCaptureDevices returns the available microphone devices for the selected backend.

func ListOutputDevices

func ListOutputDevices(cfg Config) ([]DeviceInfo, error)

ListOutputDevices returns the available speaker devices for the selected backend.

type Event

type Event struct {
	Type    EventType
	Backend Backend
	Message string
	Err     error
}

type EventType

type EventType string
const (
	EventStarted EventType = "started"
	EventStopped EventType = "stopped"
	EventWarning EventType = "warning"
	EventError   EventType = "error"
	// EventOverrun signals that the frame dispatcher dropped captured
	// frames because the consumer (level/VAD handlers) could not keep
	// up. The authoritative full-capture buffer is unaffected — only
	// live level/segmentation frames were lost.
	EventOverrun EventType = "overrun"
	// EventStalled signals that the capture device stopped delivering
	// frames while it claims to be running (driver stall, device
	// starvation). Emitted once per stall episode.
	EventStalled EventType = "stalled"
)

type Factory

type Factory func(Config) (Session, error)

type FramePool

type FramePool struct {
	// Capacity is the cap() of fresh buffers returned by Get. Zero
	// falls back to DefaultFrameCapacity at first use.
	Capacity int
	// contains filtered or unexported fields
}

FramePool returns recyclable byte slices for short-lived PCM frame buffers. The hot path it addresses is the malgo capture callback (capture_windows_cgo.go) which used to allocate a fresh slice per callback (~33×/sec per active capture, ~3300×/sec at 100 concurrent server sessions).

Contract

  • Get returns a []byte with len(buf) == 0 and cap(buf) >= pool.Capacity (DefaultFrameCapacity if unset). Callers append to it; reslicing past cap will reallocate the way the language normally would.
  • Put returns the buffer to the pool. After Put the caller MUST NOT read from or write to the slice — another goroutine may pick it up immediately.
  • Put is idempotent on the nil slice (no-op).
  • Buffers whose grown capacity exceeds 4× the configured Capacity are dropped on Put rather than retained, so pathological growth does not pin large allocations in the pool.

Concurrency

FramePool wraps sync.Pool — safe for concurrent Get / Put from many goroutines. The pool may discard entries at any GC cycle; callers MUST treat Get as may-allocate-fresh.

Observability

HitRatio() returns a coarse hit/miss ratio over the pool's lifetime. It is intended for occasional sanity checks and metric exports; the underlying counters use atomic adds so HitRatio is safe to call concurrently with Get and Put.

var DefaultFramePool FramePool

DefaultFramePool is the package-level pool. Most callers should use the package functions Get / Put rather than constructing their own pool. Tests that need isolation construct their own FramePool.

func (*FramePool) Get

func (p *FramePool) Get() []byte

Get returns a recyclable buffer with len 0 and capacity at least p.Capacity (or DefaultFrameCapacity when unset).

func (*FramePool) HitRatio

func (p *FramePool) HitRatio() float64

HitRatio returns the fraction of Get calls served from the cache (1.0 = always hit, 0.0 = always allocated fresh). Returns 0 when no operations have happened yet. The return is a snapshot — the counters may advance immediately after the call.

Concretely: HitRatio = (gets - misses) / gets, where misses is incremented inside sync.Pool.New (i.e. every fresh allocation counts as a miss).

func (*FramePool) Put

func (p *FramePool) Put(buf []byte)

Put returns the buffer to the pool. nil buffers are silently dropped. Buffers whose capacity exceeds 4× the configured Capacity are dropped to prevent unbounded growth from pinning the pool.

func (*FramePool) Stats

func (p *FramePool) Stats() Stats

Stats returns the lifetime counters. Hits can be derived as stats.Gets - stats.Misses.

type InputSource

type InputSource string
const (
	InputSourceMicrophone     InputSource = "microphone"
	InputSourceSystemLoopback InputSource = "system_loopback"
	InputSourceMicAndSystem   InputSource = "mic_and_system"
)

type PooledPCMHandler

type PooledPCMHandler = func(buf []byte, release func())

PooledPCMHandler receives one captured PCM frame with explicit buffer-ownership semantics. The release closure MUST be invoked exactly once when the handler is done with buf — either before returning, or asynchronously once any retained reference is released. The buffer MUST NOT be read or written after release.

See FramePool for the underlying lifecycle. The optimisation only matters for sustained capture (~33 callbacks/sec per session); short-lived recording paths can stay on the legacy SetPCMHandler API without ceremony.

Declared as a type alias (not a defined type) so implementations of Session structurally satisfy interfaces declared outside this package (e.g. pkg/speechkit's PooledPCMRecorder) without importing this package.

type Session

type Session interface {
	Start() error
	Stop() ([]byte, error)
	IsRunning() bool
	Events() <-chan Event
	SetLevelHandler(func(float64))
	SetPCMHandler(func([]byte))
	// SetPooledPCMHandler installs the pool-aware variant of the PCM
	// callback. When set (non-nil), the capture backend leases the
	// per-frame buffer from this package's package-level FramePool
	// instead of allocating fresh, and invokes the handler with a
	// release closure. The handler MUST call release exactly once
	// before returning OR before retaining any reference to the
	// slice. Forgetting to release leaks one pool slot per frame but
	// does not corrupt data.
	//
	// When both SetPCMHandler and SetPooledPCMHandler are set, the
	// pool-aware variant wins — the legacy handler is not invoked
	// for that frame, so callers that adopt the pooled API should
	// also unset the legacy one to avoid surprise.
	//
	// Backends not yet wired to honour the pool MAY no-op this
	// setter; the legacy SetPCMHandler path remains the canonical
	// contract for all existing callers.
	SetPooledPCMHandler(PooledPCMHandler)
	Close() error
}

Session records microphone PCM and exposes both level and live-audio callbacks.

func Open

func Open(cfg Config) (Session, error)

type Stats

type Stats struct {
	Gets   uint64
	Misses uint64
}

Stats are the raw lifetime counters. Useful for OTel meter callbacks that prefer integer reports over a derived ratio.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL