stt

package
v0.68.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 4, 2026 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Overview

Package stt defines the SpeechKit speech-to-text provider interface and houses the concrete provider implementations: whisper.cpp (local built-in), HuggingFace, OpenAI, Groq, Google, an OpenAI-compatible adapter (covers Ollama and other compatible servers), and the self-hosted VPS adapter.

All providers must go through github.com/kombifyio/SpeechKit/pkg/speechkit/netsec for outbound HTTP. Routing — which provider to pick for a given request — is the host's responsibility (a routing/fallback layer above these providers).

router.go implements the STT routing layer. It picks the right provider (local, cloud, or direct) for a transcription request based on the configured Strategy (Dynamic, LocalOnly, CloudOnly) and current connectivity / readiness.

The router is platform-neutral and is consumed by both the Device-Target (Wails client) and the Server-Target. TTS has a parallel routing layer in pkg/speechkit/tts; this file is STT-only. It was promoted from internal/router, which remains as a compatibility alias shim.

Index

Examples

Constants

View Source
const LanguageMulti = "multi"

LanguageMulti is the value SpeechKit carries when no language is pinned.

SpeechKit does not narrow speech to one language: a pinned language breaks switching language mid-conversation and speaking different languages with different people in one session, and it is a silent data-loss bug — the same English audio returns a zero-length transcript when pinned to German, with HTTP 200 and no error.

It is deliberately NOT forwarded verbatim. Each provider expresses multilanguage in its own dialect: Deepgram takes the literal "multi", AssemblyAI sets language_detection, and the OpenAI-compatible, local and OpenRouter adapters express it by omitting the field so the model auto-detects. Sending "multi" to any of the latter is an invalid value.

View Source
const MaxResponseBytes = 1 << 20

MaxResponseBytes bounds how much of a provider's HTTP response an adapter reads. Transcription responses are text; anything larger is a misconfiguration or a hostile endpoint, not a longer transcript.

Variables

This section is empty.

Functions

func AsTranscriber added in v0.61.28

func AsTranscriber(p STTProvider, opts ...TranscriberOption) speechkit.Transcriber

AsTranscriber adapts an STTProvider to the kernel's speechkit.Transcriber interface, so hosts can hand any provider straight to dictation.NewRuntime or a TranscriptionWorker without hand-writing the mechanical Result-to-Transcript adapter. The per-call language overrides the base option language when non-empty; when both are empty the request carries no language at all — the adapter never invents a default, because language selection (including multilanguage) is an explicit caller decision.

The device app keeps its own richer internal adapter (vocabulary bias, customization, routing); this bridge covers the plain provider-to-kernel path for library consumers.

Example

ExampleAsTranscriber bridges an STTProvider to the kernel's speechkit.Transcriber so it can be handed straight to dictation.NewRuntime or a TranscriptionWorker. The per-call language wins over the base options.

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/kombifyio/SpeechKit/pkg/speechkit"
	"github.com/kombifyio/SpeechKit/pkg/speechkit/stt"
)

// echoProvider is a minimal stt.STTProvider. Real hosts use the providers in
// the stt subpackages (local, deepgram, openai, ...) or their own backend.
type echoProvider struct{ name string }

func (p echoProvider) Name() string                 { return p.name }
func (p echoProvider) Health(context.Context) error { return nil }
func (p echoProvider) Transcribe(_ context.Context, _ []byte, opts stt.TranscribeOpts) (*stt.Result, error) {
	return &stt.Result{Text: "hello from " + p.name, Language: opts.Language, Provider: p.name}, nil
}

func main() {
	var transcriber speechkit.Transcriber = stt.AsTranscriber(
		echoProvider{name: "echo"},
		stt.WithTranscribeOpts(stt.TranscribeOpts{Language: "de"}),
	)

	wav := speechkit.PCMToWAV(make([]byte, 16000*2))
	transcript, err := transcriber.Transcribe(context.Background(), wav, 1.0, "")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(transcript.Text, transcript.Language, transcript.Provider)

	transcript, _ = transcriber.Transcribe(context.Background(), wav, 1.0, "en")
	fmt.Println(transcript.Language)
}
Output:
hello from echo de echo
en

func BaseCapabilities added in v0.64.3

func BaseCapabilities() []speechkit.Capability

BaseCapabilities are reported by every speech-to-text provider. A fresh slice is returned each call so callers may append without aliasing.

func EnsureTranscriptionWAV added in v0.64.3

func EnsureTranscriptionWAV(raw []byte) []byte

EnsureTranscriptionWAV wraps raw PCM in a WAV header when it does not already carry one, because every HTTP transcription endpoint expects a container rather than bare samples.

func EnvSecretResolver added in v0.67.14

func EnvSecretResolver(name string) string

EnvSecretResolver reads secrets from the process environment. It is the default every provider falls back to when no resolver is configured.

func FirstNonEmptyTrimmed added in v0.64.3

func FirstNonEmptyTrimmed(values ...string) string

FirstNonEmptyTrimmed returns the first value that is not blank after trimming, or "". Provider adapters use it to layer a request override over a provider default over a package default.

func IsMultilanguage added in v0.53.0

func IsMultilanguage(language string) bool

IsMultilanguage reports whether language asks for multilanguage rather than a specific locale. Empty, "auto" and "multi" all mean the same thing: do not pin. The settings UI offers "auto" and "multi" as separate labels, so both have to resolve here or the literal token reaches providers that reject it.

func IsWAV added in v0.64.3

func IsWAV(raw []byte) bool

IsWAV reports whether raw starts with a RIFF/WAVE header.

func IsWebSocketClose added in v0.64.3

func IsWebSocketClose(err error) bool

IsWebSocketClose reports whether err is a WebSocket close, as opposed to a transport failure. Streaming providers treat the two differently: a close is the end of a session, a transport failure is worth surfacing.

func MaxSpeakers added in v0.64.3

func MaxSpeakers(opts speaker.Options) int

MaxSpeakers is the upper-bound counterpart of MinSpeakers.

func MinSpeakers added in v0.64.3

func MinSpeakers(opts speaker.Options) int

MinSpeakers and MaxSpeakers collapse the three speaker-count knobs into the single lower and upper bound a provider request carries. An exact expected count wins over the range.

func PCM16FromWAV added in v0.64.3

func PCM16FromWAV(audio []byte) (pcm []byte, sampleRate, channels int, ok bool)

PCM16FromWAV extracts raw PCM16 sample data plus the sample rate and channel count from a LINEAR16 RIFF/WAVE byte slice. It returns ok=false when the input is not a 16-bit PCM WAV, in which case callers should treat the bytes as already-raw PCM and fall back to a default rate.

Providers that must declare the sample rate explicitly (e.g. Google STT v1 speech:recognize) rely on this to avoid sending a wrong hard-coded rate, which the provider rejects with HTTP 400 on a mismatch.

func ProviderNameFromProfileID added in v0.64.3

func ProviderNameFromProfileID(profileID string) string

ProviderNameFromProfileID extracts the bare provider name from a model_selection profile id, or "" when the id names a specific model.

func ToTranscript added in v0.61.28

func ToTranscript(r *Result, durationSecs float64) speechkit.Transcript

ToTranscript maps a provider Result to a speechkit.Transcript. The mapping is mechanical: text, language, provider, model, confidence, per-word confidences, and speaker diarization pass through 1:1. Duration comes from the Result when the provider reported one; otherwise durationSecs (the caller-measured audio length) fills in. A nil Result yields a zero Transcript.

Types

type CapabilityReporter

type CapabilityReporter interface {
	Capabilities() []speechkit.Capability
}

CapabilityReporter is an optional interface a provider can implement to self-describe its capabilities. Routing and catalogs can type-assert it to query capabilities directly instead of string-matching profile IDs.

It is intentionally NOT part of STTProvider, so existing providers, server clients, and test doubles continue to satisfy the core interface without change; only the production providers below opt in. Capabilities are reported using the public speechkit.Capability vocabulary so the adapters carry no internal/ dependency.

type ProviderSelectedObserver added in v0.67.14

type ProviderSelectedObserver func(ctx context.Context, providerName string, strategy Strategy)

ProviderSelectedObserver is invoked after every successful routed transcription with the winning provider's name and the active strategy. Hosts use it to record audit events; the framework itself stays free of host logging dependencies. Observer failures must never abort a user-facing transcription, so the callback has no error return.

type ResolvedTranscribeOptions

type ResolvedTranscribeOptions struct {
	Language              string
	Model                 string
	Prompt                string
	ContextPrompt         string
	LanguageHints         []string
	Keyterms              []string
	Speaker               speaker.Options
	DetectLanguage        bool
	Punctuation           bool
	SmartFormat           bool
	Dictation             bool
	FillerWords           bool
	Numerals              bool
	UseVocabularyKeyterms bool
	Timestamps            bool
	EndpointingMs         int
	PrivacyRedaction      bool
	VoiceFocus            bool
	MedicalDomain         bool
	Effective             provideropts.EffectiveOptions
}

func ResolveTranscribeOptions

func ResolveTranscribeOptions(provider, profileID string, opts TranscribeOpts, providerDefaults, providerOverrides provideropts.Values) ResolvedTranscribeOptions

func (ResolvedTranscribeOptions) APILanguage

func (r ResolvedTranscribeOptions) APILanguage() string

type Result

type Result struct {
	Text       string
	Language   string
	Duration   time.Duration
	Provider   string
	Model      string
	Confidence float64 // If available from the provider
	// Words carries per-word acoustic confidence when the provider exposes it.
	// nil for providers without word-level confidence. Offsets are NOT tracked
	// because downstream vocabulary/punctuation rewriting invalidates them;
	// consumers match on the word text instead.
	Words    []WordConfidence
	Speakers *speaker.DiarizationResult
}

Result holds the output of a transcription.

type Router added in v0.61.32

type Router struct {
	Strategy             Strategy
	PreferLocalUnderSecs float64
	ParallelCloud        bool
	ReplaceOnBetter      bool
	// ConnectivityProbe is the TCP address used to test internet connectivity.
	// Defaults to "1.1.1.1:443" when empty.
	ConnectivityProbe string

	// OnProviderSelected, when set, is called after every successful routed
	// transcription. It scopes audit reporting to this router instance.
	OnProviderSelected ProviderSelectedObserver
	// contains filtered or unexported fields
}

Router selects the best STTProvider based on audio length, availability, and config.

Example

ExampleRouter wires one local provider into a Router and pins the strategy to local-only, which is the fresh-install path with no cloud keys. The per-instance OnProviderSelected hook replaces the deprecated process-wide observer, so two routers in one process never share audit state.

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/kombifyio/SpeechKit/pkg/speechkit"
	"github.com/kombifyio/SpeechKit/pkg/speechkit/stt"
)

// echoProvider is a minimal stt.STTProvider. Real hosts use the providers in
// the stt subpackages (local, deepgram, openai, ...) or their own backend.
type echoProvider struct{ name string }

func (p echoProvider) Name() string                 { return p.name }
func (p echoProvider) Health(context.Context) error { return nil }
func (p echoProvider) Transcribe(_ context.Context, _ []byte, opts stt.TranscribeOpts) (*stt.Result, error) {
	return &stt.Result{Text: "hello from " + p.name, Language: opts.Language, Provider: p.name}, nil
}

func main() {
	router := &stt.Router{
		Strategy: stt.StrategyLocalOnly,
		OnProviderSelected: func(_ context.Context, provider string, strategy stt.Strategy) {
			fmt.Println("selected", provider, "via", strategy)
		},
	}
	router.SetLocal(echoProvider{name: "local"})

	result, err := router.Route(context.Background(), speechkit.PCMToWAV(make([]byte, 16000*2)), 1.0, stt.TranscribeOpts{Language: "en"})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(result.Text)
}
Output:
selected local via local-only
hello from local

func (*Router) AddCloud added in v0.61.32

func (r *Router) AddCloud(p STTProvider)

AddCloud appends a cloud provider to the ordered list (thread-safe).

func (*Router) AvailableProviders added in v0.61.32

func (r *Router) AvailableProviders() []string

AvailableProviders returns the names of configured providers.

func (*Router) Cloud added in v0.61.32

func (r *Router) Cloud(name string) STTProvider

Cloud returns a cloud provider by name, or nil if not found.

func (*Router) HasDictationStreaming added in v0.61.32

func (r *Router) HasDictationStreaming() bool

HasDictationStreaming reports whether at least one configured provider (honoring the routing strategy) can serve provider-native realtime dictation. Server surfaces use this for capability discovery so clients can fall back to batch transcription without a doomed stream attempt.

func (*Router) HuggingFace added in v0.61.32

func (r *Router) HuggingFace() STTProvider

HuggingFace returns the HuggingFace cloud provider (backward-compatible convenience).

func (*Router) Local added in v0.61.32

func (r *Router) Local() STTProvider

Local returns the current local provider.

func (*Router) PreferCloud added in v0.61.32

func (r *Router) PreferCloud(name string, p STTProvider)

PreferCloud sets/replaces a cloud provider and moves it to the front so it becomes the next cloud provider used by routing, while keeping remaining providers as fallbacks.

func (*Router) Providers added in v0.61.32

func (r *Router) Providers() []STTProvider

Providers returns a snapshot of every configured provider: the local provider first (when set), then the cloud providers in routing order. Hosts use it to register health probes without re-deriving the provider set from config.

func (*Router) Route added in v0.61.32

func (r *Router) Route(ctx context.Context, audio []byte, audioDurationSecs float64, opts TranscribeOpts) (res *Result, err error)

Route selects the appropriate provider(s) and returns the transcription result.

func (*Router) SetCloud added in v0.61.32

func (r *Router) SetCloud(name string, p STTProvider)

SetCloud replaces a cloud provider by name, or appends if not found. Pass nil to remove the provider with that name.

func (*Router) SetCloudProviders added in v0.61.32

func (r *Router) SetCloudProviders(providers []STTProvider)

SetCloudProviders replaces the ordered cloud provider list.

func (*Router) SetHuggingFace added in v0.61.32

func (r *Router) SetHuggingFace(p STTProvider)

SetHuggingFace sets/replaces the HuggingFace cloud provider (backward-compatible convenience).

func (*Router) SetLocal added in v0.61.32

func (r *Router) SetLocal(p STTProvider)

SetLocal sets the local provider (thread-safe).

func (*Router) SetVPS added in v0.61.32

func (r *Router) SetVPS(p STTProvider)

SetVPS sets/replaces the VPS cloud provider (backward-compatible convenience).

func (*Router) StartDictationStream added in v0.61.32

StartDictationStream selects the first configured provider that can perform provider-native realtime dictation. It never changes batch routing; callers choose this path explicitly per recording session.

func (*Router) StartSpeakerStream added in v0.61.32

func (r *Router) StartSpeakerStream(ctx context.Context, opts speaker.Options, format speaker.AudioFormat) (speaker.SpeakerStream, error)

StartSpeakerStream selects the first configured provider that can perform realtime speaker attribution. The stream is an add-on path and never changes the normal STT routing decision for Dictation/Assist.

func (*Router) VPS added in v0.61.32

func (r *Router) VPS() STTProvider

VPS returns the VPS cloud provider (backward-compatible convenience).

type STTProvider

type STTProvider interface {
	// Transcribe sends audio data to the STT backend and returns the transcription.
	Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)

	// Name returns the provider identifier (e.g. "local", "vps", "huggingface").
	Name() string

	// Health checks if the provider is reachable and ready.
	Health(ctx context.Context) error
}

STTProvider defines the interface for all speech-to-text backends.

It is the provider-facing contract (the SPI): implement it to add a backend, hand it to a Router for fallback and routing, and bridge it to the kernel's host-facing speechkit.Transcriber with AsTranscriber. Hosts consume Transcriber; providers implement STTProvider.

func PrioritizeProviderProfile added in v0.64.3

func PrioritizeProviderProfile(candidates []STTProvider, profileID string) []STTProvider

PrioritizeProviderProfile moves the providers matching profileID to the front of the candidate list, keeping the rest as fallbacks in their original order.

type SecretResolver added in v0.67.14

type SecretResolver func(name string) string

SecretResolver resolves a named secret (typically an environment-variable name) to its value. An empty string means "not set".

Providers that read credentials lazily (for example the Google streaming adapter, which loads service-account JSON on first stream) expose a SecretResolver field so each provider instance can be bound to its own secret backend. That keeps two SpeechKit runtimes in one process — a test harness, a multi-tenant server — independent of each other. There is no process-wide resolver: hosts bind secrets per provider (or through allproviders.EnabledProviders.Secrets).

func (SecretResolver) Resolve added in v0.67.14

func (r SecretResolver) Resolve(name string) string

Resolve returns r(name). A nil resolver reads the process environment, so provider packages can call it on a zero-value field.

type Strategy added in v0.61.32

type Strategy string

Strategy defines the routing strategy.

const (
	StrategyDynamic   Strategy = "dynamic"
	StrategyLocalOnly Strategy = "local-only"
	StrategyCloudOnly Strategy = "cloud-only"
)

type TranscribeOpts

type TranscribeOpts struct {
	Language string   // "de", "en", "auto"; request override only
	Model    string   // Optional: model override
	Prompt   string   // Optional: provider-specific hint prompt for better recognition
	Keyterms []string // Optional: provider-native vocabulary bias terms
	// ConversationContext carries the preceding dialogue turns (oldest
	// first, no speaker labels) for providers whose models condition on
	// conversational context — e.g. AssemblyAI Universal-3.5 Pro sync
	// accepts up to 100 turns / 4096 chars via conversation_context.
	// Distinct from Prompt (which describes the domain/scenario) and
	// Keyterms (explicit vocabulary). Providers without native support
	// ignore it.
	ConversationContext []string
	// ProviderProfileID prioritizes a provider for this request. Accepts a
	// full provider-profile ID (e.g. "stt.deepgram.nova-3") or a bare
	// provider ID (e.g. "deepgram"); routers move matching providers to the
	// front of their candidate list while keeping the remaining providers as
	// fallbacks. An unknown or unconfigured value changes nothing — a
	// request never hard-fails on an unsatisfiable preference. Mirrors
	// speechkit.DictationStreamOptions.ProviderProfileID for the batch path.
	ProviderProfileID         string
	Speaker                   speaker.Options                // Optional speaker diarization / attribution request
	Options                   provideropts.Values            // Optional normalized global/default voice options
	ProviderOptions           provideropts.Values            // Optional normalized overrides for the selected provider
	ProviderOptionsByProvider map[string]provideropts.Values // Optional provider-keyed overrides used by routers
}

TranscribeOpts configures a single transcription request.

func (TranscribeOpts) ForProvider

func (o TranscribeOpts) ForProvider(provider string) TranscribeOpts

type TranscriberOption added in v0.61.28

type TranscriberOption func(*transcriberOptions)

TranscriberOption configures the adapter returned by AsTranscriber.

func WithTranscribeOpts added in v0.61.28

func WithTranscribeOpts(base TranscribeOpts) TranscriberOption

WithTranscribeOpts sets the base TranscribeOpts applied to every request the adapter makes (prompt, keyterms, provider options, and so on). The Language field of the base is used only when the per-call language passed to Transcribe is empty; a non-empty per-call language always wins.

type WordConfidence

type WordConfidence = speechkit.WordConfidence

WordConfidence is a single recognized word with the provider's per-word acoustic confidence in [0,1]. It is populated only by providers that expose word-level confidence (Deepgram, AssemblyAI); it stays nil for providers that do not (Google v1, OpenAI/Groq/HuggingFace Whisper, local whisper.cpp). Confidence here is acoustic (how sure the model is it heard this word), not semantic — a low value flags a likely mis-recognition or dropped word.

It is an alias of the kernel type so provider results and Transcripts share one word type; hosts never convert between the two.

Directories

Path Synopsis
Package allproviders is the batteries-included STT assembly layer: it knows every provider SpeechKit ships and turns a host's resolved credentials into a ready stt.Router.
Package allproviders is the batteries-included STT assembly layer: it knows every provider SpeechKit ships and turns a host's resolved credentials into a ready stt.Router.
Package sttcontract provides a reusable conformance suite that every stt.STTProvider implementation is expected to satisfy.
Package sttcontract provides a reusable conformance suite that every stt.STTProvider implementation is expected to satisfy.
Package vps is the self-hosted whisper-server provider for SpeechKit: an OpenAI-compatible endpoint a user runs themselves, so audio never reaches a commercial provider.
Package vps is the self-hosted whisper-server provider for SpeechKit: an OpenAI-compatible endpoint a user runs themselves, so audio never reaches a commercial provider.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL