Documentation
¶
Overview ¶
Package stt defines the SpeechKit speech-to-text provider interface and houses the concrete provider implementations: whisper.cpp (local built-in), HuggingFace, OpenAI, Groq, Google, an OpenAI-compatible adapter (covers Ollama and other compatible servers), and the self-hosted VPS adapter.
All providers must go through github.com/kombifyio/SpeechKit/pkg/speechkit/netsec for outbound HTTP. Routing — which provider to pick for a given request — is the host's responsibility (a routing/fallback layer above these providers).
Index ¶
- Constants
- func FindWhisperBinary() (string, error)
- func ParseDeepgramKeyterms(raw string) []string
- func SetSecretResolver(fn func(string) string)
- func ValidateModelPath(path string) error
- type AssemblyAIProvider
- func (*AssemblyAIProvider) Capabilities() []speechkit.Capability
- func (p *AssemblyAIProvider) Health(ctx context.Context) error
- func (p *AssemblyAIProvider) Name() string
- func (p *AssemblyAIProvider) StartSpeakerStream(ctx context.Context, opts speaker.Options, format speaker.AudioFormat) (speaker.SpeakerStream, error)
- func (p *AssemblyAIProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)
- type CapabilityReporter
- type DeepgramOptions
- type DeepgramProvider
- func (p *DeepgramProvider) ApplyOptions(opts DeepgramOptions)
- func (*DeepgramProvider) Capabilities() []speechkit.Capability
- func (p *DeepgramProvider) Health(ctx context.Context) error
- func (p *DeepgramProvider) Name() string
- func (p *DeepgramProvider) StartDictationStream(ctx context.Context, opts speechkit.DictationStreamOptions, ...) (speechkit.DictationStream, error)
- func (p *DeepgramProvider) StartSpeakerStream(ctx context.Context, opts speaker.Options, format speaker.AudioFormat) (speaker.SpeakerStream, error)
- func (p *DeepgramProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)
- type GoogleSTTProvider
- func (*GoogleSTTProvider) Capabilities() []speechkit.Capability
- func (p *GoogleSTTProvider) Health(ctx context.Context) error
- func (p *GoogleSTTProvider) Name() string
- func (p *GoogleSTTProvider) SetStreamingCredentialEnvs(credentialsJSONEnv, applicationCredentialsEnv string)
- func (p *GoogleSTTProvider) StartSpeakerStream(ctx context.Context, opts speaker.Options, format speaker.AudioFormat) (speaker.SpeakerStream, error)
- func (p *GoogleSTTProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)
- type HuggingFaceProvider
- type InstallStatus
- type LocalProvider
- func (*LocalProvider) Capabilities() []speechkit.Capability
- func (p *LocalProvider) Health(ctx context.Context) error
- func (p *LocalProvider) IsReady() bool
- func (p *LocalProvider) Name() string
- func (p *LocalProvider) StartServer(ctx context.Context) error
- func (p *LocalProvider) StopServer()
- func (p *LocalProvider) Transcribe(ctx context.Context, audioData []byte, opts TranscribeOpts) (*Result, error)
- func (p *LocalProvider) VerifyInstallation() InstallStatus
- type OpenAICompatibleProvider
- func NewGroqSTTProvider(apiKey string) *OpenAICompatibleProvider
- func NewOllamaSTTProvider(baseURL, model string) *OpenAICompatibleProvider
- func NewOpenAICompatibleProvider(name, baseURL, apiKey, model string) *OpenAICompatibleProvider
- func NewOpenAISTTProvider(apiKey string) *OpenAICompatibleProvider
- func NewVPSProvider(baseURL, apiKey string) *OpenAICompatibleProvider
- func NewVPSProviderWithModel(baseURL, apiKey, model string) *OpenAICompatibleProvider
- func (*OpenAICompatibleProvider) Capabilities() []speechkit.Capability
- func (p *OpenAICompatibleProvider) Health(ctx context.Context) error
- func (p *OpenAICompatibleProvider) Name() string
- func (p *OpenAICompatibleProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)
- type OpenRouterSTTProvider
- type ResolvedTranscribeOptions
- type Result
- type STTProvider
- type TranscribeOpts
- type VPSProvider
- type WordConfidence
Constants ¶
const MinWhisperModelBytes = 50_000_000
MinWhisperModelBytes is the minimum file size we expect for a valid ggml model. ggml-base.bin is ~150 MB; anything under 50 MB is clearly corrupt/truncated.
Variables ¶
This section is empty.
Functions ¶
func FindWhisperBinary ¶
FindWhisperBinary exposes the local whisper runtime lookup for callers that need to reflect runtime readiness without starting the subprocess.
func ParseDeepgramKeyterms ¶
func SetSecretResolver ¶
SetSecretResolver overrides how this package resolves named secrets (see resolveSecret). A nil resolver is ignored. Call once at startup, before constructing or using providers; the resolver is read when a provider opens a credentialed connection.
func ValidateModelPath ¶
ValidateModelPath verifies that path points at a whisper.cpp ggml model file with a safe filename. It rejects path traversal, non-absolute paths, and filenames that don't match the ggml-*.bin pattern.
Types ¶
type AssemblyAIProvider ¶
type AssemblyAIProvider struct {
APIKey string
Models []string
StreamingModel string
BaseURL string
StreamingBaseURL string
Validation netsec.ValidationOptions
PollInterval time.Duration
PollTimeout time.Duration
// contains filtered or unexported fields
}
func NewAssemblyAIProvider ¶
func NewAssemblyAIProvider(apiKey, models string) *AssemblyAIProvider
func (*AssemblyAIProvider) Capabilities ¶
func (*AssemblyAIProvider) Capabilities() []speechkit.Capability
AssemblyAI additionally attributes/identifies speakers against caller-supplied names or roles.
func (*AssemblyAIProvider) Name ¶
func (p *AssemblyAIProvider) Name() string
func (*AssemblyAIProvider) StartSpeakerStream ¶
func (p *AssemblyAIProvider) StartSpeakerStream(ctx context.Context, opts speaker.Options, format speaker.AudioFormat) (speaker.SpeakerStream, error)
func (*AssemblyAIProvider) Transcribe ¶
func (p *AssemblyAIProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)
type CapabilityReporter ¶
type CapabilityReporter interface {
Capabilities() []speechkit.Capability
}
CapabilityReporter is an optional interface a provider can implement to self-describe its capabilities. Routing and catalogs can type-assert it to query capabilities directly instead of string-matching profile IDs.
It is intentionally NOT part of STTProvider, so existing providers, server clients, and test doubles continue to satisfy the core interface without change; only the production providers below opt in. Capabilities are reported using the public speechkit.Capability vocabulary so the adapters carry no internal/ dependency.
type DeepgramOptions ¶
type DeepgramOptions struct {
Configured bool
SmartFormat bool
Dictation bool
FillerWords bool
Numerals bool
DetectLanguage bool
LanguageOverride string
UseVocabularyKeyterms bool
Keyterms []string
EndpointingMs int
}
DeepgramOptions holds provider-specific Deepgram STT controls. These map to Deepgram Listen query parameters while keeping SpeechKit's public STT router interface provider-neutral.
type DeepgramProvider ¶
type DeepgramProvider struct {
APIKey string
Model string
DiarizationModel string
BaseURL string
Validation netsec.ValidationOptions
SmartFormat bool
Dictation bool
FillerWords bool
Numerals bool
DetectLanguage bool
LanguageOverride string
UseVocabularyKeyterms bool
Keyterms []string
EndpointingMs int
// contains filtered or unexported fields
}
func NewDeepgramProvider ¶
func NewDeepgramProvider(apiKey, model string) *DeepgramProvider
func (*DeepgramProvider) ApplyOptions ¶
func (p *DeepgramProvider) ApplyOptions(opts DeepgramOptions)
func (*DeepgramProvider) Capabilities ¶
func (*DeepgramProvider) Capabilities() []speechkit.Capability
Deepgram and Google add batch speaker diarization (live-verified in the speaker layer).
func (*DeepgramProvider) Name ¶
func (p *DeepgramProvider) Name() string
func (*DeepgramProvider) StartDictationStream ¶ added in v0.48.0
func (p *DeepgramProvider) StartDictationStream(ctx context.Context, opts speechkit.DictationStreamOptions, format speaker.AudioFormat) (speechkit.DictationStream, error)
func (*DeepgramProvider) StartSpeakerStream ¶
func (p *DeepgramProvider) StartSpeakerStream(ctx context.Context, opts speaker.Options, format speaker.AudioFormat) (speaker.SpeakerStream, error)
func (*DeepgramProvider) Transcribe ¶
func (p *DeepgramProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)
type GoogleSTTProvider ¶
type GoogleSTTProvider struct {
APIKey string
Model string // "latest_long", "latest_short", or another Google STT v1 model tag
STTCredentialsJSONEnv string
ApplicationCredentialsEnv string
BaseURL string // Override for testing; defaults to googleSTTBaseURL
Validation netsec.ValidationOptions
// contains filtered or unexported fields
}
GoogleSTTProvider implements STTProvider for Google Cloud Speech-to-Text v1 REST API.
BaseURL is user-configurable (for testing or regional endpoints). It is validated against Validation on every request. Default Validation is strict (public https only).
func NewGoogleSTTProvider ¶
func NewGoogleSTTProvider(apiKey, model string) *GoogleSTTProvider
NewGoogleSTTProvider creates a provider for Google Cloud Speech-to-Text. Model defaults to "latest_long" if empty.
func (*GoogleSTTProvider) Capabilities ¶
func (*GoogleSTTProvider) Capabilities() []speechkit.Capability
func (*GoogleSTTProvider) Health ¶
func (p *GoogleSTTProvider) Health(ctx context.Context) error
Health checks if the Google Speech API is reachable.
func (*GoogleSTTProvider) Name ¶
func (p *GoogleSTTProvider) Name() string
Name returns the provider identifier.
func (*GoogleSTTProvider) SetStreamingCredentialEnvs ¶
func (p *GoogleSTTProvider) SetStreamingCredentialEnvs(credentialsJSONEnv, applicationCredentialsEnv string)
func (*GoogleSTTProvider) StartSpeakerStream ¶
func (p *GoogleSTTProvider) StartSpeakerStream(ctx context.Context, opts speaker.Options, format speaker.AudioFormat) (speaker.SpeakerStream, error)
StartSpeakerStream opens a Google Cloud Speech-to-Text v2 StreamingRecognize session.
IMPORTANT: Google STT v2 does NOT support speaker diarization in streaming mode — diarization is available only in BatchRecognize/Recognize (see the Chirp-3 docs). This adapter therefore provides realtime TRANSCRIPTION only: frames carry Text without speaker labels (no Segment/Speakers). Streaming speaker diarization in the Voice Agent comes from Deepgram/AssemblyAI.
func (*GoogleSTTProvider) Transcribe ¶
func (p *GoogleSTTProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)
Transcribe sends audio to Google Cloud Speech-to-Text v1 REST API.
type HuggingFaceProvider ¶
type HuggingFaceProvider struct {
Model string
Token string
BaseURL string // Override for testing; defaults to hfBaseURL
Validation netsec.ValidationOptions
// contains filtered or unexported fields
}
HuggingFaceProvider implements STTProvider for Tier 3: HuggingFace Inference API.
BaseURL is user-configurable. It is validated against Validation on every request. Default Validation is strict (public https only).
func NewHuggingFaceProvider ¶
func NewHuggingFaceProvider(model, token string) *HuggingFaceProvider
func (*HuggingFaceProvider) Capabilities ¶
func (*HuggingFaceProvider) Capabilities() []speechkit.Capability
func (*HuggingFaceProvider) Health ¶
func (p *HuggingFaceProvider) Health(ctx context.Context) error
func (*HuggingFaceProvider) Name ¶
func (p *HuggingFaceProvider) Name() string
func (*HuggingFaceProvider) Transcribe ¶
func (p *HuggingFaceProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)
type InstallStatus ¶
type InstallStatus struct {
BinaryFound bool `json:"binaryFound"`
BinaryPath string `json:"binaryPath"`
ModelFound bool `json:"modelFound"`
ModelPath string `json:"modelPath"`
ModelBytes int64 `json:"modelBytes"`
ServerReady bool `json:"serverReady"`
Problems []string `json:"problems,omitempty"`
}
InstallStatus describes what's present and what's missing for local STT.
type LocalProvider ¶
type LocalProvider struct {
BaseURL string // e.g. "http://127.0.0.1:8080"
Port int
ModelPath string
GPU string
Validation netsec.ValidationOptions
// contains filtered or unexported fields
}
LocalProvider implements STTProvider for Tier 1: localhost whisper.cpp server.
func NewLocalProvider ¶
func NewLocalProvider(port int, modelPath, gpu string) *LocalProvider
func (*LocalProvider) Capabilities ¶
func (*LocalProvider) Capabilities() []speechkit.Capability
func (*LocalProvider) IsReady ¶
func (p *LocalProvider) IsReady() bool
IsReady returns true if the whisper-server subprocess is running and responding.
func (*LocalProvider) Name ¶
func (p *LocalProvider) Name() string
func (*LocalProvider) StartServer ¶
func (p *LocalProvider) StartServer(ctx context.Context) error
StartServer starts the whisper.cpp server subprocess. Blocks until ready or context cancelled.
func (*LocalProvider) StopServer ¶
func (p *LocalProvider) StopServer()
StopServer terminates the whisper-server subprocess.
func (*LocalProvider) Transcribe ¶
func (p *LocalProvider) Transcribe(ctx context.Context, audioData []byte, opts TranscribeOpts) (*Result, error)
func (*LocalProvider) VerifyInstallation ¶
func (p *LocalProvider) VerifyInstallation() InstallStatus
VerifyInstallation checks binary and model availability without starting the server.
type OpenAICompatibleProvider ¶
type OpenAICompatibleProvider struct {
BaseURL string
APIKey string
Model string
Validation netsec.ValidationOptions
// contains filtered or unexported fields
}
OpenAICompatibleProvider implements STTProvider for any endpoint speaking the OpenAI /v1/audio/transcriptions API (OpenAI, Groq, VPS whisper-server, etc.).
BaseURL is user-supplied configuration. It is validated against Validation on every request (Transcribe, Health). The default Validation is strict: only public https:// endpoints are accepted. Self-hosted VPS and local whisper-server require relaxing Validation — see NewVPSProvider.
func NewGroqSTTProvider ¶
func NewGroqSTTProvider(apiKey string) *OpenAICompatibleProvider
NewGroqSTTProvider creates a provider for the Groq Whisper API.
func NewOllamaSTTProvider ¶
func NewOllamaSTTProvider(baseURL, model string) *OpenAICompatibleProvider
NewOllamaSTTProvider creates a provider for Ollama-compatible local transcription endpoints. Ollama runs on loopback by default and can be pointed at a user-managed self-hosted URL.
func NewOpenAICompatibleProvider ¶
func NewOpenAICompatibleProvider(name, baseURL, apiKey, model string) *OpenAICompatibleProvider
NewOpenAICompatibleProvider creates a provider for any OpenAI-compatible STT endpoint. Default Validation is strict (public https only). Callers with a non-public endpoint (loopback, RFC1918) must set Validation explicitly.
func NewOpenAISTTProvider ¶
func NewOpenAISTTProvider(apiKey string) *OpenAICompatibleProvider
NewOpenAISTTProvider creates a provider for the OpenAI Whisper API.
func NewVPSProvider ¶
func NewVPSProvider(baseURL, apiKey string) *OpenAICompatibleProvider
NewVPSProvider creates a provider for a self-hosted whisper-server. Allows loopback, private IP ranges and plain http:// because self-hosted deployments frequently run inside a VPN, on a home LAN, or on localhost.
func NewVPSProviderWithModel ¶
func NewVPSProviderWithModel(baseURL, apiKey, model string) *OpenAICompatibleProvider
func (*OpenAICompatibleProvider) Capabilities ¶
func (*OpenAICompatibleProvider) Capabilities() []speechkit.Capability
func (*OpenAICompatibleProvider) Health ¶
func (p *OpenAICompatibleProvider) Health(ctx context.Context) error
Health checks provider reachability. Tries GET /health first (whisper-server), then falls back to GET /v1/models (OpenAI, Groq).
func (*OpenAICompatibleProvider) Name ¶
func (p *OpenAICompatibleProvider) Name() string
Name returns the provider identifier.
func (*OpenAICompatibleProvider) Transcribe ¶
func (p *OpenAICompatibleProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)
Transcribe sends audio to the OpenAI-compatible /v1/audio/transcriptions endpoint.
type OpenRouterSTTProvider ¶
type OpenRouterSTTProvider struct {
BaseURL string
APIKey string
Model string
Validation netsec.ValidationOptions
// contains filtered or unexported fields
}
OpenRouterSTTProvider implements OpenRouter's JSON speech-to-text endpoint. OpenRouter is a cloud gateway, not a direct model provider, and its STT API accepts base64 audio rather than OpenAI's multipart Whisper shape.
func NewOpenRouterSTTProvider ¶
func NewOpenRouterSTTProvider(apiKey, model string) *OpenRouterSTTProvider
func (*OpenRouterSTTProvider) Capabilities ¶
func (*OpenRouterSTTProvider) Capabilities() []speechkit.Capability
func (*OpenRouterSTTProvider) Health ¶
func (p *OpenRouterSTTProvider) Health(ctx context.Context) error
func (*OpenRouterSTTProvider) Name ¶
func (p *OpenRouterSTTProvider) Name() string
func (*OpenRouterSTTProvider) Transcribe ¶
func (p *OpenRouterSTTProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)
type ResolvedTranscribeOptions ¶
type ResolvedTranscribeOptions struct {
Language string
Model string
Prompt string
ContextPrompt string
LanguageHints []string
Keyterms []string
Speaker speaker.Options
DetectLanguage bool
Punctuation bool
SmartFormat bool
Dictation bool
FillerWords bool
Numerals bool
UseVocabularyKeyterms bool
Timestamps bool
EndpointingMs int
PrivacyRedaction bool
VoiceFocus bool
MedicalDomain bool
Effective provideropts.EffectiveOptions
}
func ResolveTranscribeOptions ¶
func ResolveTranscribeOptions(provider, profileID string, opts TranscribeOpts, providerDefaults, providerOverrides provideropts.Values) ResolvedTranscribeOptions
func (ResolvedTranscribeOptions) APILanguage ¶
func (r ResolvedTranscribeOptions) APILanguage() string
type Result ¶
type Result struct {
Text string
Language string
Duration time.Duration
Provider string
Model string
Confidence float64 // If available from the provider
// Words carries per-word acoustic confidence when the provider exposes it.
// nil for providers without word-level confidence. Offsets are NOT tracked
// because downstream vocabulary/punctuation rewriting invalidates them;
// consumers match on the word text instead.
Words []WordConfidence
Speakers *speaker.DiarizationResult
}
Result holds the output of a transcription.
type STTProvider ¶
type STTProvider interface {
// Transcribe sends audio data to the STT backend and returns the transcription.
Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)
// Name returns the provider identifier (e.g. "local", "vps", "huggingface").
Name() string
// Health checks if the provider is reachable and ready.
Health(ctx context.Context) error
}
STTProvider defines the interface for all speech-to-text backends.
type TranscribeOpts ¶
type TranscribeOpts struct {
Language string // "de", "en", "auto"; request override only
Model string // Optional: model override
Prompt string // Optional: provider-specific hint prompt for better recognition
Keyterms []string // Optional: provider-native vocabulary bias terms
Speaker speaker.Options // Optional speaker diarization / attribution request
Options provideropts.Values // Optional normalized global/default voice options
ProviderOptions provideropts.Values // Optional normalized overrides for the selected provider
ProviderOptionsByProvider map[string]provideropts.Values // Optional provider-keyed overrides used by routers
}
TranscribeOpts configures a single transcription request.
func (TranscribeOpts) ForProvider ¶
func (o TranscribeOpts) ForProvider(provider string) TranscribeOpts
type VPSProvider ¶
type VPSProvider = OpenAICompatibleProvider
VPSProvider is an alias for backward compatibility. Use OpenAICompatibleProvider directly for new code.
type WordConfidence ¶
WordConfidence is a single recognized word with the provider's per-word acoustic confidence in [0,1]. It is populated only by providers that expose word-level confidence (Deepgram, AssemblyAI); it stays nil for providers that do not (Google v1, OpenAI/Groq/HuggingFace Whisper, local whisper.cpp). Confidence here is acoustic (how sure the model is it heard this word), not semantic — a low value flags a likely mis-recognition or dropped word.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package sttcontract provides a reusable conformance suite that every stt.STTProvider implementation is expected to satisfy.
|
Package sttcontract provides a reusable conformance suite that every stt.STTProvider implementation is expected to satisfy. |