Documentation
¶
Overview ¶
Package dictation implements speech-to-text capture and transcription for the composer: press a key, talk, and the transcript lands in the input for review — never auto-fired at the agent.
Two pipelines share a keybinding and a composer-insertion target:
- Batch: record to a temp WAV file (Recorder.Start/Stop), then transcribe in one shot (Transcriber.Transcribe) — a local sherpa-onnx-offline exec or a single HTTPS POST to Groq/OpenAI. The only pipeline on Termux.
- Streaming: capture raw PCM from the mic tool's stdout (Recorder.StartStreaming), feed 50ms chunks to a websocket (Transcriber.StreamTranscribe), and render partial transcripts live.
Every OS integration shells out via exec.Command with discrete argv — never a shell string, never CGO — matching internal/imageinput/clipboard.go. The external binaries (arecord/sox/ffmpeg/termux-microphone-record, sherpa-onnx) are looked up on PATH and degrade to a clear setup message when missing, matching internal/lsp's philosophy for language servers.
Index ¶
- Constants
- Variables
- func AutoDownloadSupported() bool
- func BatchProviders() []string
- func ChunkLevel(chunk []byte) float64
- func EngineDownloaded(destRoot, version string) bool
- func ModelDownloaded(destRoot, dirName string) bool
- func RequiredSampleRate(t Transcriber) int
- func StreamProviders() []string
- type AudioFormat
- type AuthError
- type CloudConfig
- type DeepgramConfig
- type DownloadOptions
- type EngineComponents
- type LocalConfig
- type ModelVariant
- type OpenAIRealtimeConfig
- type Platform
- type Recorder
- type RecorderOptions
- type SampleRate
- type ServerConfig
- type ServerManager
- type SetupError
- type Transcriber
- func NewCloudTranscriber(cfg CloudConfig) (Transcriber, error)
- func NewDeepgramTranscriber(cfg DeepgramConfig) (Transcriber, error)
- func NewLocalStreamingTranscriber(manager *ServerManager) Transcriber
- func NewLocalTranscriber(cfg LocalConfig) (Transcriber, error)
- func NewOpenAIRealtimeTranscriber(cfg OpenAIRealtimeConfig) (Transcriber, error)
Constants ¶
const ( ProviderLocal = "local" ProviderGroq = "groq" ProviderOpenAI = "openai" )
Batch transcription providers (config value stt.provider).
const ( StreamProviderLocal = "local" StreamProviderDeepgram = "deepgram" StreamProviderOpenAI = "openai" )
Streaming transcription providers (config value stt.streamProvider).
const ( // DefaultSampleRate is what the local engine and Deepgram consume. // OpenAI Realtime's pcm16 requires 24kHz; its transcriber constructs the // recorder accordingly. DefaultSampleRate = 16000 // DefaultMaxDuration is the hard runaway-recording cap (§12), applied on // every platform, batch and streaming. Overridable via // stt.maxDurationSeconds. DefaultMaxDuration = 5 * time.Minute )
const DefaultServerPort = 6006
DefaultServerPort is the localhost port the sherpa-onnx streaming server binds.
const DefaultSherpaVersion = "v1.13.3"
DefaultSherpaVersion is the pinned, smoke-tested release used unless stt.engineVersion overrides it. "latest" or any tag also works.
Variables ¶
var ErrStreamingUnsupported = errors.New("this transcription provider does not support streaming")
ErrStreamingUnsupported reports that a batch-only Transcriber cannot stream; the caller falls back to the batch pipeline.
Functions ¶
func AutoDownloadSupported ¶
func AutoDownloadSupported() bool
AutoDownloadSupported reports whether the current platform has a known engine asset (so the F9 flow can offer the download). Termux/Android and unusual arches install manually or use a cloud provider.
func BatchProviders ¶
func BatchProviders() []string
BatchProviders lists valid stt.provider values, for config validation.
func ChunkLevel ¶
ChunkLevel returns a perceptual 0..1 loudness for a little-endian PCM16 chunk, driving the live recording waveform from the actual microphone input. It uses an RMS→dBFS→linear mapping over [levelFloorDB, levelCeilDB].
func EngineDownloaded ¶
EngineDownloaded reports whether the shared sherpa-onnx engine is already on disk for this platform (one engine serves every model), so the picker can drop the engine's size from a model's download total once it's been fetched.
func ModelDownloaded ¶
ModelDownloaded reports whether a model variant is already extracted under destRoot (so the picker can mark it as installed). dirName is the variant's DirName.
func RequiredSampleRate ¶
func RequiredSampleRate(t Transcriber) int
RequiredSampleRate returns the capture rate a Transcriber needs.
func StreamProviders ¶
func StreamProviders() []string
StreamProviders lists valid stt.streamProvider values, for config validation.
Types ¶
type AudioFormat ¶
type AudioFormat string
AudioFormat identifies a recorded container format, sniffed from the bytes — desktop recorders produce WAV, Termux's recorder produces M4A/AAC.
const ( FormatWAV AudioFormat = "wav" FormatM4A AudioFormat = "m4a" FormatUnknown AudioFormat = "" )
func SniffFormat ¶
func SniffFormat(data []byte) AudioFormat
SniffFormat detects the audio container from magic bytes. WAV is "RIFF....WAVE"; MP4-family (Termux's .m4a) has "ftyp" at offset 4.
func (AudioFormat) FileName ¶
func (f AudioFormat) FileName() string
FileName returns a transcription-upload filename for the format.
type AuthError ¶
AuthError reports that a cloud provider rejected a transcription request for authentication reasons (HTTP 401/403) — a missing or invalid API key. It is a fixable credential problem, so callers (the TUI) can offer an inline key prompt for Provider instead of a dead-end error. Message is the already- classified, secret-redacted user-facing text.
type CloudConfig ¶
type CloudConfig struct {
Provider string // ProviderGroq or ProviderOpenAI, for error messages
BaseURL string // e.g. https://api.groq.com/openai/v1
APIKey string
Model string // e.g. whisper-large-v3-turbo, whisper-1
// Language optionally constrains recognition (ISO-639-1). Empty = auto.
Language string
// HTTPClient is injectable for tests; nil uses providerio's shared,
// stall-hardened client.
HTTPClient *http.Client
}
CloudConfig configures a batch cloud transcriber. Groq and OpenAI share the OpenAI-compatible /audio/transcriptions multipart endpoint — the design's "each owns its endpoint/model specifics while sharing the request shape" (§6). Key and base URL are resolved by the caller (TUI layer) from config/credstore, keeping this type decoupled and unit-testable.
type DeepgramConfig ¶
type DeepgramConfig struct {
APIKey string
Model string // default "nova-3"
Language string
SampleRate int // capture rate; default DefaultSampleRate (16kHz)
// BaseURL overrides the wss endpoint (tests point it at a fake server).
BaseURL string
}
DeepgramConfig configures the Deepgram streaming transcriber (§6b, default cloud streaming provider). The key is resolved by the caller.
type DownloadOptions ¶
type DownloadOptions struct {
// DestRoot is where extracted engines/models live (e.g. ~/.config/zero/stt).
DestRoot string
// EngineVersion selects the sherpa-onnx release tag ("" → DefaultSherpaVersion;
// "latest" and any tag also work).
EngineVersion string
// HTTPClient is injectable for tests; nil uses a plain client.
HTTPClient *http.Client
// APIBase overrides the GitHub API base (tests). "" → api.github.com.
APIBase string
// Model selection (default: the Moonshine-tiny int8 batch model). Set these
// from a ModelVariant to download a different model.
ModelAssetName string
ModelPinnedDigest string
ModelDirName string
// ModelLabel is the friendly model name shown in progress ("Zipformer 20M").
ModelLabel string
// Progress receives live status strings for the UI, including a percentage
// while downloading (e.g. "Engine 45% · 57/126 MB").
Progress func(status string)
// contains filtered or unexported fields
}
DownloadOptions configures EnsureLocalEngine.
type EngineComponents ¶
type EngineComponents struct {
BinaryPath string // extracted sherpa-onnx-offline
ServerPath string // extracted sherpa-onnx-online-websocket-server
ModelPath string // extracted model directory
}
EngineComponents identifies the resolved local-engine paths.
func EnsureLocalEngine ¶
func EnsureLocalEngine(ctx context.Context, opts DownloadOptions) (EngineComponents, error)
EnsureLocalEngine downloads (if absent) and verifies the sherpa-onnx engine and the default model into DestRoot, returning the resolved paths. Idempotent: an already-extracted, still-present engine/model is reused without re-downloading. Every archive is SHA256-verified against the GitHub API digest before extraction.
type LocalConfig ¶
type LocalConfig struct {
// Binary is the sherpa-onnx-offline executable; looked up on PATH when a
// bare name. Empty defaults to "sherpa-onnx-offline".
Binary string
// ModelPath is the directory holding the model's .onnx + tokens.txt files
// (stt.localModelPath).
ModelPath string
NumThreads int
// contains filtered or unexported fields
}
LocalConfig configures the offline sherpa-onnx batch transcriber. Every tier (default/standard/high-end, §7) is just a different ModelPath — there are no tier-specific code paths; the model family is detected from the files present.
type ModelVariant ¶
type ModelVariant struct {
ID string
Label string
Description string
Bytes int64
AssetName string
Digest string
DirName string
Streaming bool
// Recommended marks a curated, checksum-pinned, validated model (shown first).
Recommended bool
}
ModelVariant is a downloadable model the /stt-model picker offers, with a pinned SHA256 (the model release predates GitHub's per-asset digests, so these must be pinned). Batch variants use the offline binary; the streaming variant (a transducer) drives the websocket server for a live transcript.
func ListModels ¶
ListModels fetches every transcription model in the sherpa-onnx asr-models release and returns them as variants (the curated ones flagged Recommended and carrying a pinned digest; the rest with an empty digest, downloaded with TLS-only verification since that release predates GitHub's per-asset digests). Non-ASR assets (VAD, TTS, punctuation, speaker/keyword/…): filtered out.
func ModelVariants ¶
func ModelVariants() []ModelVariant
ModelVariants returns the curated English models the download picker offers. This is a hand-picked shortlist (each pinned by SHA256 since the model release predates GitHub's per-asset digests), NOT the whole sherpa-onnx model zoo — many more models (other languages, sizes, and streaming/batch families) work via a manual stt.localModelPath. Streaming (live) options come first.
type OpenAIRealtimeConfig ¶
type OpenAIRealtimeConfig struct {
APIKey string
Model string // default "gpt-4o-transcribe"
// BaseURL overrides the wss endpoint (tests point it at a fake server).
BaseURL string
}
OpenAIRealtimeConfig configures the OpenAI Realtime transcription transcriber (§6b, the credential-reuse streaming alternative). Reuses the OpenAI key.
type Platform ¶
type Platform string
Platform selects the audio-capture toolchain (§9 of the design doc).
func DetectPlatform ¶
func DetectPlatform() Platform
DetectPlatform picks the capture toolchain for the current host. Termux is detected by its environment (it reports GOOS=linux/android): TERMUX_VERSION is exported to every Termux shell, and PREFIX carries the app package path.
func (Platform) StreamingSupported ¶
StreamingSupported reports whether the platform's capture tool can write raw PCM to stdout. Termux's termux-microphone-record requires a regular output file — a hard tool limitation, so Termux is batch-only permanently.
type Recorder ¶
type Recorder interface {
Start() error
// Stop ends a batch recording and returns the recorded audio bytes (WAV on
// desktop, M4A on Termux — SniffFormat distinguishes them).
Stop() ([]byte, error)
// StartStreaming begins continuous capture of raw little-endian PCM16 mono
// at the configured sample rate. Chunks arrive on the channel in ~50ms
// units; the channel closes when capture ends (stop called, max duration
// hit, or trailing-silence auto-stop). stop is idempotent.
StartStreaming() (chunks <-chan []byte, stop func() error, err error)
}
Recorder captures microphone audio. Start/Stop is the batch path (used on Termux, and as the fallback everywhere); StartStreaming is the desktop-only continuous-PCM path.
func NewRecorder ¶
func NewRecorder(opts RecorderOptions) Recorder
NewRecorder builds the platform recorder. Construction is cheap and side-effect free; tools are only exec'd on Start/StartStreaming.
type RecorderOptions ¶
type RecorderOptions struct {
Platform Platform
SampleRate int
MaxDuration time.Duration
SilenceAutoStop bool
// WindowsAudioDevice is the dshow capture device name; auto-detected from
// `ffmpeg -list_devices` when empty.
WindowsAudioDevice string
// contains filtered or unexported fields
}
RecorderOptions configures a Recorder. Zero values pick sensible defaults.
type SampleRate ¶
type SampleRate interface {
SampleRate() int
}
SampleRate lets a Transcriber declare the capture rate its wire format expects, so the recorder can be built to match (OpenAI Realtime wants 24kHz; everything else uses 16kHz). A Transcriber that does not implement this uses DefaultSampleRate.
type ServerConfig ¶
type ServerConfig struct {
// Binary is the streaming server executable (default
// "sherpa-onnx-online-websocket-server"); looked up on PATH.
Binary string
// ModelPath is the sherpa-onnx streaming model directory (transducer:
// encoder/decoder/joiner + tokens).
ModelPath string
Port int
NumThreads int
// contains filtered or unexported fields
}
ServerConfig configures the warm sherpa-onnx streaming server (§6a).
type ServerManager ¶
type ServerManager struct {
// contains filtered or unexported fields
}
ServerManager owns one long-lived sherpa-onnx streaming server, spawned lazily on first streaming use and kept warm for the session — a websocket server with ~1-2s startup latency can't be respawned per utterance (§6a). Its lifecycle mirrors internal/lsp's Manager: lazy start, health-checked reuse, one restart on a crashed process, torn down on exit. Safe for concurrent use.
func NewServerManager ¶
func NewServerManager(cfg ServerConfig) *ServerManager
NewServerManager builds a manager. Construction is side-effect free; the server is only spawned on the first EnsureRunning.
func (*ServerManager) EnsureRunning ¶
func (m *ServerManager) EnsureRunning(ctx context.Context) (string, error)
EnsureRunning returns the websocket URL of a live server, starting one if needed. A previously-started server whose process has since died is reaped and replaced (mirroring the LSP manager's crashed-session eviction).
func (*ServerManager) SetModelPath ¶
func (m *ServerManager) SetModelPath(path string)
SetModelPath updates the model directory the server launches with, for a mid-session config change (e.g. an F9 auto-download). It takes effect the next time the server (re)starts; a running server is left as-is.
func (*ServerManager) Shutdown ¶
func (m *ServerManager) Shutdown(ctx context.Context) error
Shutdown stops the running server. Called alongside the LSP manager's shutdown on exit — a leaked warm server holds a port and CPU.
func (*ServerManager) URL ¶
func (m *ServerManager) URL() string
URL returns the current server URL without starting one ("" when not running).
type SetupError ¶
SetupError reports a missing external tool with install guidance. Missing binaries are an expected condition (the user hasn't set dictation up yet), not a bug — callers show the hint instead of a raw exec error.
func (*SetupError) Error ¶
func (e *SetupError) Error() string
type Transcriber ¶
type Transcriber interface {
// Transcribe returns the final text for one recorded clip.
Transcribe(ctx context.Context, audio []byte) (string, error)
// StreamTranscribe consumes PCM16 chunks and reports incremental results
// via onPartial(text, final) as decoding proceeds, returning the complete
// final transcript when the chunk channel closes.
//
// onPartial's text is the CURRENT best transcript of the utterance so far
// (cumulative, not a delta) so the composer can replace the live region
// wholesale; final marks a segment the provider considers settled.
//
// On a mid-stream failure (network drop for cloud, subprocess crash for
// local) StreamTranscribe returns the best-effort text accumulated from
// onPartial calls made so far PLUS a non-nil error — already-streamed text
// is never discarded, only the untranscribed tail after the failure is lost.
StreamTranscribe(ctx context.Context, chunks <-chan []byte, onPartial func(text string, final bool)) (string, error)
}
Transcriber turns captured audio into text. Batch (Transcribe) and streaming (StreamTranscribe) are one interface but two independent code paths — a batch-only provider (Groq/OpenAI, sherpa-onnx-offline) leaves StreamTranscribe returning ErrStreamingUnsupported, and a streaming provider still implements Transcribe for the batch/Termux fallback.
func NewCloudTranscriber ¶
func NewCloudTranscriber(cfg CloudConfig) (Transcriber, error)
NewCloudTranscriber builds a batch transcriber for Groq or OpenAI.
func NewDeepgramTranscriber ¶
func NewDeepgramTranscriber(cfg DeepgramConfig) (Transcriber, error)
NewDeepgramTranscriber builds the Deepgram streaming transcriber.
func NewLocalStreamingTranscriber ¶
func NewLocalStreamingTranscriber(manager *ServerManager) Transcriber
NewLocalStreamingTranscriber builds a streaming transcriber backed by the shared, session-long sherpa-onnx server manager.
func NewLocalTranscriber ¶
func NewLocalTranscriber(cfg LocalConfig) (Transcriber, error)
NewLocalTranscriber builds the offline transcriber. It does not touch the filesystem or PATH until Transcribe runs, so a missing binary/model degrades to a clear setup message at use time (matching internal/lsp).
func NewOpenAIRealtimeTranscriber ¶
func NewOpenAIRealtimeTranscriber(cfg OpenAIRealtimeConfig) (Transcriber, error)
NewOpenAIRealtimeTranscriber builds the OpenAI Realtime streaming transcriber.