Documentation
¶
Overview ¶
Package stt defines the SpeechKit speech-to-text provider interface and houses the concrete provider implementations: whisper.cpp (local built-in), HuggingFace, OpenAI, Groq, Google, an OpenAI-compatible adapter (covers Ollama and other compatible servers), and the self-hosted VPS adapter.
All providers must go through github.com/kombifyio/SpeechKit/pkg/speechkit/netsec for outbound HTTP. Routing — which provider to pick for a given request — is the host's responsibility (a routing/fallback layer above these providers).
router.go implements the STT routing layer. It picks the right provider (local, cloud, or direct) for a transcription request based on the configured Strategy (Dynamic, LocalOnly, CloudOnly) and current connectivity / readiness.
The router is platform-neutral and is consumed by both the Device-Target (Wails client) and the Server-Target. TTS has a parallel routing layer in pkg/speechkit/tts; this file is STT-only. It was promoted from internal/router, which remains as a compatibility alias shim.
Index ¶
- Constants
- func AsTranscriber(p STTProvider, opts ...TranscriberOption) speechkit.Transcriber
- func BaseCapabilities() []speechkit.Capability
- func EnsureTranscriptionWAV(raw []byte) []byte
- func EnvSecretResolver(name string) string
- func FirstNonEmptyTrimmed(values ...string) string
- func IsMultilanguage(language string) bool
- func IsWAV(raw []byte) bool
- func IsWebSocketClose(err error) bool
- func MaxSpeakers(opts speaker.Options) int
- func MinSpeakers(opts speaker.Options) int
- func PCM16FromWAV(audio []byte) (pcm []byte, sampleRate, channels int, ok bool)
- func ProviderNameFromProfileID(profileID string) string
- func ToTranscript(r *Result, durationSecs float64) speechkit.Transcript
- type CapabilityReporter
- type ProviderSelectedObserver
- type ResolvedTranscribeOptions
- type Result
- type Router
- func (r *Router) AddCloud(p STTProvider)
- func (r *Router) AvailableProviders() []string
- func (r *Router) Cloud(name string) STTProvider
- func (r *Router) HasDictationStreaming() bool
- func (r *Router) HuggingFace() STTProvider
- func (r *Router) Local() STTProvider
- func (r *Router) PreferCloud(name string, p STTProvider)
- func (r *Router) Providers() []STTProvider
- func (r *Router) Route(ctx context.Context, audio []byte, audioDurationSecs float64, ...) (res *Result, err error)
- func (r *Router) SetCloud(name string, p STTProvider)
- func (r *Router) SetCloudProviders(providers []STTProvider)
- func (r *Router) SetHuggingFace(p STTProvider)
- func (r *Router) SetLocal(p STTProvider)
- func (r *Router) SetVPS(p STTProvider)
- func (r *Router) StartDictationStream(ctx context.Context, opts speechkit.DictationStreamOptions, ...) (speechkit.DictationStream, error)
- func (r *Router) StartSpeakerStream(ctx context.Context, opts speaker.Options, format speaker.AudioFormat) (speaker.SpeakerStream, error)
- func (r *Router) VPS() STTProvider
- type STTProvider
- type SecretResolver
- type Strategy
- type TranscribeOpts
- type TranscriberOption
- type WordConfidence
Examples ¶
Constants ¶
const LanguageMulti = "multi"
LanguageMulti is the value SpeechKit carries when no language is pinned.
SpeechKit does not narrow speech to one language: a pinned language breaks switching language mid-conversation and speaking different languages with different people in one session, and it is a silent data-loss bug — the same English audio returns a zero-length transcript when pinned to German, with HTTP 200 and no error.
It is deliberately NOT forwarded verbatim. Each provider expresses multilanguage in its own dialect: Deepgram takes the literal "multi", AssemblyAI sets language_detection, and the OpenAI-compatible, local and OpenRouter adapters express it by omitting the field so the model auto-detects. Sending "multi" to any of the latter is an invalid value.
const MaxResponseBytes = 1 << 20
MaxResponseBytes bounds how much of a provider's HTTP response an adapter reads. Transcription responses are text; anything larger is a misconfiguration or a hostile endpoint, not a longer transcript.
Variables ¶
This section is empty.
Functions ¶
func AsTranscriber ¶ added in v0.61.28
func AsTranscriber(p STTProvider, opts ...TranscriberOption) speechkit.Transcriber
AsTranscriber adapts an STTProvider to the kernel's speechkit.Transcriber interface, so hosts can hand any provider straight to dictation.NewRuntime or a TranscriptionWorker without hand-writing the mechanical Result-to-Transcript adapter. The per-call language overrides the base option language when non-empty; when both are empty the request carries no language at all — the adapter never invents a default, because language selection (including multilanguage) is an explicit caller decision.
The device app keeps its own richer internal adapter (vocabulary bias, customization, routing); this bridge covers the plain provider-to-kernel path for library consumers.
Example ¶
ExampleAsTranscriber bridges an STTProvider to the kernel's speechkit.Transcriber so it can be handed straight to dictation.NewRuntime or a TranscriptionWorker. The per-call language wins over the base options.
package main
import (
"context"
"fmt"
"log"
"github.com/kombifyio/SpeechKit/pkg/speechkit"
"github.com/kombifyio/SpeechKit/pkg/speechkit/stt"
)
// echoProvider is a minimal stt.STTProvider. Real hosts use the providers in
// the stt subpackages (local, deepgram, openai, ...) or their own backend.
type echoProvider struct{ name string }
func (p echoProvider) Name() string { return p.name }
func (p echoProvider) Health(context.Context) error { return nil }
func (p echoProvider) Transcribe(_ context.Context, _ []byte, opts stt.TranscribeOpts) (*stt.Result, error) {
return &stt.Result{Text: "hello from " + p.name, Language: opts.Language, Provider: p.name}, nil
}
func main() {
var transcriber speechkit.Transcriber = stt.AsTranscriber(
echoProvider{name: "echo"},
stt.WithTranscribeOpts(stt.TranscribeOpts{Language: "de"}),
)
wav := speechkit.PCMToWAV(make([]byte, 16000*2))
transcript, err := transcriber.Transcribe(context.Background(), wav, 1.0, "")
if err != nil {
log.Fatal(err)
}
fmt.Println(transcript.Text, transcript.Language, transcript.Provider)
transcript, _ = transcriber.Transcribe(context.Background(), wav, 1.0, "en")
fmt.Println(transcript.Language)
}
Output: hello from echo de echo en
func BaseCapabilities ¶ added in v0.64.3
func BaseCapabilities() []speechkit.Capability
BaseCapabilities are reported by every speech-to-text provider. A fresh slice is returned each call so callers may append without aliasing.
func EnsureTranscriptionWAV ¶ added in v0.64.3
EnsureTranscriptionWAV wraps raw PCM in a WAV header when it does not already carry one, because every HTTP transcription endpoint expects a container rather than bare samples.
func EnvSecretResolver ¶ added in v0.67.14
EnvSecretResolver reads secrets from the process environment. It is the default every provider falls back to when no resolver is configured.
func FirstNonEmptyTrimmed ¶ added in v0.64.3
FirstNonEmptyTrimmed returns the first value that is not blank after trimming, or "". Provider adapters use it to layer a request override over a provider default over a package default.
func IsMultilanguage ¶ added in v0.53.0
IsMultilanguage reports whether language asks for multilanguage rather than a specific locale. Empty, "auto" and "multi" all mean the same thing: do not pin. The settings UI offers "auto" and "multi" as separate labels, so both have to resolve here or the literal token reaches providers that reject it.
func IsWebSocketClose ¶ added in v0.64.3
IsWebSocketClose reports whether err is a WebSocket close, as opposed to a transport failure. Streaming providers treat the two differently: a close is the end of a session, a transport failure is worth surfacing.
func MaxSpeakers ¶ added in v0.64.3
MaxSpeakers is the upper-bound counterpart of MinSpeakers.
func MinSpeakers ¶ added in v0.64.3
MinSpeakers and MaxSpeakers collapse the three speaker-count knobs into the single lower and upper bound a provider request carries. An exact expected count wins over the range.
func PCM16FromWAV ¶ added in v0.64.3
PCM16FromWAV extracts raw PCM16 sample data plus the sample rate and channel count from a LINEAR16 RIFF/WAVE byte slice. It returns ok=false when the input is not a 16-bit PCM WAV, in which case callers should treat the bytes as already-raw PCM and fall back to a default rate.
Providers that must declare the sample rate explicitly (e.g. Google STT v1 speech:recognize) rely on this to avoid sending a wrong hard-coded rate, which the provider rejects with HTTP 400 on a mismatch.
func ProviderNameFromProfileID ¶ added in v0.64.3
ProviderNameFromProfileID extracts the bare provider name from a model_selection profile id, or "" when the id names a specific model.
func ToTranscript ¶ added in v0.61.28
func ToTranscript(r *Result, durationSecs float64) speechkit.Transcript
ToTranscript maps a provider Result to a speechkit.Transcript. The mapping is mechanical: text, language, provider, model, confidence, per-word confidences, and speaker diarization pass through 1:1. Duration comes from the Result when the provider reported one; otherwise durationSecs (the caller-measured audio length) fills in. A nil Result yields a zero Transcript.
Types ¶
type CapabilityReporter ¶
type CapabilityReporter interface {
Capabilities() []speechkit.Capability
}
CapabilityReporter is an optional interface a provider can implement to self-describe its capabilities. Routing and catalogs can type-assert it to query capabilities directly instead of string-matching profile IDs.
It is intentionally NOT part of STTProvider, so existing providers, server clients, and test doubles continue to satisfy the core interface without change; only the production providers below opt in. Capabilities are reported using the public speechkit.Capability vocabulary so the adapters carry no internal/ dependency.
type ProviderSelectedObserver ¶ added in v0.67.14
ProviderSelectedObserver is invoked after every successful routed transcription with the winning provider's name and the active strategy. Hosts use it to record audit events; the framework itself stays free of host logging dependencies. Observer failures must never abort a user-facing transcription, so the callback has no error return.
type ResolvedTranscribeOptions ¶
type ResolvedTranscribeOptions struct {
Language string
Model string
Prompt string
ContextPrompt string
LanguageHints []string
Keyterms []string
Speaker speaker.Options
DetectLanguage bool
Punctuation bool
SmartFormat bool
Dictation bool
FillerWords bool
Numerals bool
UseVocabularyKeyterms bool
Timestamps bool
EndpointingMs int
PrivacyRedaction bool
VoiceFocus bool
MedicalDomain bool
Effective provideropts.EffectiveOptions
}
func ResolveTranscribeOptions ¶
func ResolveTranscribeOptions(provider, profileID string, opts TranscribeOpts, providerDefaults, providerOverrides provideropts.Values) ResolvedTranscribeOptions
func (ResolvedTranscribeOptions) APILanguage ¶
func (r ResolvedTranscribeOptions) APILanguage() string
type Result ¶
type Result struct {
Text string
Language string
Duration time.Duration
Provider string
Model string
Confidence float64 // If available from the provider
// Words carries per-word acoustic confidence when the provider exposes it.
// nil for providers without word-level confidence. Offsets are NOT tracked
// because downstream vocabulary/punctuation rewriting invalidates them;
// consumers match on the word text instead.
Words []WordConfidence
Speakers *speaker.DiarizationResult
}
Result holds the output of a transcription.
type Router ¶ added in v0.61.32
type Router struct {
Strategy Strategy
PreferLocalUnderSecs float64
ParallelCloud bool
ReplaceOnBetter bool
// ConnectivityProbe is the TCP address used to test internet connectivity.
// Defaults to "1.1.1.1:443" when empty.
ConnectivityProbe string
// OnProviderSelected, when set, is called after every successful routed
// transcription. It scopes audit reporting to this router instance.
OnProviderSelected ProviderSelectedObserver
// contains filtered or unexported fields
}
Router selects the best STTProvider based on audio length, availability, and config.
Example ¶
ExampleRouter wires one local provider into a Router and pins the strategy to local-only, which is the fresh-install path with no cloud keys. The per-instance OnProviderSelected hook replaces the deprecated process-wide observer, so two routers in one process never share audit state.
package main
import (
"context"
"fmt"
"log"
"github.com/kombifyio/SpeechKit/pkg/speechkit"
"github.com/kombifyio/SpeechKit/pkg/speechkit/stt"
)
// echoProvider is a minimal stt.STTProvider. Real hosts use the providers in
// the stt subpackages (local, deepgram, openai, ...) or their own backend.
type echoProvider struct{ name string }
func (p echoProvider) Name() string { return p.name }
func (p echoProvider) Health(context.Context) error { return nil }
func (p echoProvider) Transcribe(_ context.Context, _ []byte, opts stt.TranscribeOpts) (*stt.Result, error) {
return &stt.Result{Text: "hello from " + p.name, Language: opts.Language, Provider: p.name}, nil
}
func main() {
router := &stt.Router{
Strategy: stt.StrategyLocalOnly,
OnProviderSelected: func(_ context.Context, provider string, strategy stt.Strategy) {
fmt.Println("selected", provider, "via", strategy)
},
}
router.SetLocal(echoProvider{name: "local"})
result, err := router.Route(context.Background(), speechkit.PCMToWAV(make([]byte, 16000*2)), 1.0, stt.TranscribeOpts{Language: "en"})
if err != nil {
log.Fatal(err)
}
fmt.Println(result.Text)
}
Output: selected local via local-only hello from local
func (*Router) AddCloud ¶ added in v0.61.32
func (r *Router) AddCloud(p STTProvider)
AddCloud appends a cloud provider to the ordered list (thread-safe).
func (*Router) AvailableProviders ¶ added in v0.61.32
AvailableProviders returns the names of configured providers.
func (*Router) Cloud ¶ added in v0.61.32
func (r *Router) Cloud(name string) STTProvider
Cloud returns a cloud provider by name, or nil if not found.
func (*Router) HasDictationStreaming ¶ added in v0.61.32
HasDictationStreaming reports whether at least one configured provider (honoring the routing strategy) can serve provider-native realtime dictation. Server surfaces use this for capability discovery so clients can fall back to batch transcription without a doomed stream attempt.
func (*Router) HuggingFace ¶ added in v0.61.32
func (r *Router) HuggingFace() STTProvider
HuggingFace returns the HuggingFace cloud provider (backward-compatible convenience).
func (*Router) Local ¶ added in v0.61.32
func (r *Router) Local() STTProvider
Local returns the current local provider.
func (*Router) PreferCloud ¶ added in v0.61.32
func (r *Router) PreferCloud(name string, p STTProvider)
PreferCloud sets/replaces a cloud provider and moves it to the front so it becomes the next cloud provider used by routing, while keeping remaining providers as fallbacks.
func (*Router) Providers ¶ added in v0.61.32
func (r *Router) Providers() []STTProvider
Providers returns a snapshot of every configured provider: the local provider first (when set), then the cloud providers in routing order. Hosts use it to register health probes without re-deriving the provider set from config.
func (*Router) Route ¶ added in v0.61.32
func (r *Router) Route(ctx context.Context, audio []byte, audioDurationSecs float64, opts TranscribeOpts) (res *Result, err error)
Route selects the appropriate provider(s) and returns the transcription result.
func (*Router) SetCloud ¶ added in v0.61.32
func (r *Router) SetCloud(name string, p STTProvider)
SetCloud replaces a cloud provider by name, or appends if not found. Pass nil to remove the provider with that name.
func (*Router) SetCloudProviders ¶ added in v0.61.32
func (r *Router) SetCloudProviders(providers []STTProvider)
SetCloudProviders replaces the ordered cloud provider list.
func (*Router) SetHuggingFace ¶ added in v0.61.32
func (r *Router) SetHuggingFace(p STTProvider)
SetHuggingFace sets/replaces the HuggingFace cloud provider (backward-compatible convenience).
func (*Router) SetLocal ¶ added in v0.61.32
func (r *Router) SetLocal(p STTProvider)
SetLocal sets the local provider (thread-safe).
func (*Router) SetVPS ¶ added in v0.61.32
func (r *Router) SetVPS(p STTProvider)
SetVPS sets/replaces the VPS cloud provider (backward-compatible convenience).
func (*Router) StartDictationStream ¶ added in v0.61.32
func (r *Router) StartDictationStream(ctx context.Context, opts speechkit.DictationStreamOptions, format speaker.AudioFormat) (speechkit.DictationStream, error)
StartDictationStream selects the first configured provider that can perform provider-native realtime dictation. It never changes batch routing; callers choose this path explicitly per recording session.
func (*Router) StartSpeakerStream ¶ added in v0.61.32
func (r *Router) StartSpeakerStream(ctx context.Context, opts speaker.Options, format speaker.AudioFormat) (speaker.SpeakerStream, error)
StartSpeakerStream selects the first configured provider that can perform realtime speaker attribution. The stream is an add-on path and never changes the normal STT routing decision for Dictation/Assist.
func (*Router) VPS ¶ added in v0.61.32
func (r *Router) VPS() STTProvider
VPS returns the VPS cloud provider (backward-compatible convenience).
type STTProvider ¶
type STTProvider interface {
// Transcribe sends audio data to the STT backend and returns the transcription.
Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)
// Name returns the provider identifier (e.g. "local", "vps", "huggingface").
Name() string
// Health checks if the provider is reachable and ready.
Health(ctx context.Context) error
}
STTProvider defines the interface for all speech-to-text backends.
It is the provider-facing contract (the SPI): implement it to add a backend, hand it to a Router for fallback and routing, and bridge it to the kernel's host-facing speechkit.Transcriber with AsTranscriber. Hosts consume Transcriber; providers implement STTProvider.
func PrioritizeProviderProfile ¶ added in v0.64.3
func PrioritizeProviderProfile(candidates []STTProvider, profileID string) []STTProvider
PrioritizeProviderProfile moves the providers matching profileID to the front of the candidate list, keeping the rest as fallbacks in their original order.
type SecretResolver ¶ added in v0.67.14
SecretResolver resolves a named secret (typically an environment-variable name) to its value. An empty string means "not set".
Providers that read credentials lazily (for example the Google streaming adapter, which loads service-account JSON on first stream) expose a SecretResolver field so each provider instance can be bound to its own secret backend. That keeps two SpeechKit runtimes in one process — a test harness, a multi-tenant server — independent of each other. There is no process-wide resolver: hosts bind secrets per provider (or through allproviders.EnabledProviders.Secrets).
func (SecretResolver) Resolve ¶ added in v0.67.14
func (r SecretResolver) Resolve(name string) string
Resolve returns r(name). A nil resolver reads the process environment, so provider packages can call it on a zero-value field.
type TranscribeOpts ¶
type TranscribeOpts struct {
Language string // "de", "en", "auto"; request override only
Model string // Optional: model override
Prompt string // Optional: provider-specific hint prompt for better recognition
Keyterms []string // Optional: provider-native vocabulary bias terms
// ConversationContext carries the preceding dialogue turns (oldest
// first, no speaker labels) for providers whose models condition on
// conversational context — e.g. AssemblyAI Universal-3.5 Pro sync
// accepts up to 100 turns / 4096 chars via conversation_context.
// Distinct from Prompt (which describes the domain/scenario) and
// Keyterms (explicit vocabulary). Providers without native support
// ignore it.
ConversationContext []string
// ProviderProfileID prioritizes a provider for this request. Accepts a
// full provider-profile ID (e.g. "stt.deepgram.nova-3") or a bare
// provider ID (e.g. "deepgram"); routers move matching providers to the
// front of their candidate list while keeping the remaining providers as
// fallbacks. An unknown or unconfigured value changes nothing — a
// request never hard-fails on an unsatisfiable preference. Mirrors
// speechkit.DictationStreamOptions.ProviderProfileID for the batch path.
ProviderProfileID string
Speaker speaker.Options // Optional speaker diarization / attribution request
Options provideropts.Values // Optional normalized global/default voice options
ProviderOptions provideropts.Values // Optional normalized overrides for the selected provider
ProviderOptionsByProvider map[string]provideropts.Values // Optional provider-keyed overrides used by routers
}
TranscribeOpts configures a single transcription request.
func (TranscribeOpts) ForProvider ¶
func (o TranscribeOpts) ForProvider(provider string) TranscribeOpts
type TranscriberOption ¶ added in v0.61.28
type TranscriberOption func(*transcriberOptions)
TranscriberOption configures the adapter returned by AsTranscriber.
func WithTranscribeOpts ¶ added in v0.61.28
func WithTranscribeOpts(base TranscribeOpts) TranscriberOption
WithTranscribeOpts sets the base TranscribeOpts applied to every request the adapter makes (prompt, keyterms, provider options, and so on). The Language field of the base is used only when the per-call language passed to Transcribe is empty; a non-empty per-call language always wins.
type WordConfidence ¶
type WordConfidence = speechkit.WordConfidence
WordConfidence is a single recognized word with the provider's per-word acoustic confidence in [0,1]. It is populated only by providers that expose word-level confidence (Deepgram, AssemblyAI); it stays nil for providers that do not (Google v1, OpenAI/Groq/HuggingFace Whisper, local whisper.cpp). Confidence here is acoustic (how sure the model is it heard this word), not semantic — a low value flags a likely mis-recognition or dropped word.
It is an alias of the kernel type so provider results and Transcripts share one word type; hosts never convert between the two.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package allproviders is the batteries-included STT assembly layer: it knows every provider SpeechKit ships and turns a host's resolved credentials into a ready stt.Router.
|
Package allproviders is the batteries-included STT assembly layer: it knows every provider SpeechKit ships and turns a host's resolved credentials into a ready stt.Router. |
|
Package sttcontract provides a reusable conformance suite that every stt.STTProvider implementation is expected to satisfy.
|
Package sttcontract provides a reusable conformance suite that every stt.STTProvider implementation is expected to satisfy. |
|
Package vps is the self-hosted whisper-server provider for SpeechKit: an OpenAI-compatible endpoint a user runs themselves, so audio never reaches a commercial provider.
|
Package vps is the self-hosted whisper-server provider for SpeechKit: an OpenAI-compatible endpoint a user runs themselves, so audio never reaches a commercial provider. |