stt

package
v0.48.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 28, 2026 License: Apache-2.0 Imports: 35 Imported by: 0

Documentation

Overview

Package stt defines the SpeechKit speech-to-text provider interface and houses the concrete provider implementations: whisper.cpp (local built-in), HuggingFace, OpenAI, Groq, Google, an OpenAI-compatible adapter (covers Ollama and other compatible servers), and the self-hosted VPS adapter.

All providers must go through github.com/kombifyio/SpeechKit/pkg/speechkit/netsec for outbound HTTP. Routing — which provider to pick for a given request — is the host's responsibility (a routing/fallback layer above these providers).

Index

Constants

View Source
const MinWhisperModelBytes = 50_000_000

MinWhisperModelBytes is the minimum file size we expect for a valid ggml model. ggml-base.bin is ~150 MB; anything under 50 MB is clearly corrupt/truncated.

Variables

This section is empty.

Functions

func FindWhisperBinary

func FindWhisperBinary() (string, error)

FindWhisperBinary exposes the local whisper runtime lookup for callers that need to reflect runtime readiness without starting the subprocess.

func ParseDeepgramKeyterms

func ParseDeepgramKeyterms(raw string) []string

func SetSecretResolver

func SetSecretResolver(fn func(string) string)

SetSecretResolver overrides how this package resolves named secrets (see resolveSecret). A nil resolver is ignored. Call once at startup, before constructing or using providers; the resolver is read when a provider opens a credentialed connection.

func ValidateModelPath

func ValidateModelPath(path string) error

ValidateModelPath verifies that path points at a whisper.cpp ggml model file with a safe filename. It rejects path traversal, non-absolute paths, and filenames that don't match the ggml-*.bin pattern.

Types

type AssemblyAIProvider

type AssemblyAIProvider struct {
	APIKey           string
	Models           []string
	StreamingModel   string
	BaseURL          string
	StreamingBaseURL string
	Validation       netsec.ValidationOptions
	PollInterval     time.Duration
	PollTimeout      time.Duration
	// contains filtered or unexported fields
}

func NewAssemblyAIProvider

func NewAssemblyAIProvider(apiKey, models string) *AssemblyAIProvider

func (*AssemblyAIProvider) Capabilities

func (*AssemblyAIProvider) Capabilities() []speechkit.Capability

AssemblyAI additionally attributes/identifies speakers against caller-supplied names or roles.

func (*AssemblyAIProvider) Health

func (p *AssemblyAIProvider) Health(ctx context.Context) error

func (*AssemblyAIProvider) Name

func (p *AssemblyAIProvider) Name() string

func (*AssemblyAIProvider) StartSpeakerStream

func (p *AssemblyAIProvider) StartSpeakerStream(ctx context.Context, opts speaker.Options, format speaker.AudioFormat) (speaker.SpeakerStream, error)

func (*AssemblyAIProvider) Transcribe

func (p *AssemblyAIProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)

type CapabilityReporter

type CapabilityReporter interface {
	Capabilities() []speechkit.Capability
}

CapabilityReporter is an optional interface a provider can implement to self-describe its capabilities. Routing and catalogs can type-assert it to query capabilities directly instead of string-matching profile IDs.

It is intentionally NOT part of STTProvider, so existing providers, server clients, and test doubles continue to satisfy the core interface without change; only the production providers below opt in. Capabilities are reported using the public speechkit.Capability vocabulary so the adapters carry no internal/ dependency.

type DeepgramOptions

type DeepgramOptions struct {
	Configured            bool
	SmartFormat           bool
	Dictation             bool
	FillerWords           bool
	Numerals              bool
	DetectLanguage        bool
	LanguageOverride      string
	UseVocabularyKeyterms bool
	Keyterms              []string
	EndpointingMs         int
}

DeepgramOptions holds provider-specific Deepgram STT controls. These map to Deepgram Listen query parameters while keeping SpeechKit's public STT router interface provider-neutral.

type DeepgramProvider

type DeepgramProvider struct {
	APIKey                string
	Model                 string
	DiarizationModel      string
	BaseURL               string
	Validation            netsec.ValidationOptions
	SmartFormat           bool
	Dictation             bool
	FillerWords           bool
	Numerals              bool
	DetectLanguage        bool
	LanguageOverride      string
	UseVocabularyKeyterms bool
	Keyterms              []string
	EndpointingMs         int
	// contains filtered or unexported fields
}

func NewDeepgramProvider

func NewDeepgramProvider(apiKey, model string) *DeepgramProvider

func (*DeepgramProvider) ApplyOptions

func (p *DeepgramProvider) ApplyOptions(opts DeepgramOptions)

func (*DeepgramProvider) Capabilities

func (*DeepgramProvider) Capabilities() []speechkit.Capability

Deepgram and Google add batch speaker diarization (live-verified in the speaker layer).

func (*DeepgramProvider) Health

func (p *DeepgramProvider) Health(ctx context.Context) error

func (*DeepgramProvider) Name

func (p *DeepgramProvider) Name() string

func (*DeepgramProvider) StartDictationStream added in v0.48.0

func (*DeepgramProvider) StartSpeakerStream

func (p *DeepgramProvider) StartSpeakerStream(ctx context.Context, opts speaker.Options, format speaker.AudioFormat) (speaker.SpeakerStream, error)

func (*DeepgramProvider) Transcribe

func (p *DeepgramProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)

type GoogleSTTProvider

type GoogleSTTProvider struct {
	APIKey                    string
	Model                     string // "latest_long", "latest_short", or another Google STT v1 model tag
	STTCredentialsJSONEnv     string
	ApplicationCredentialsEnv string
	BaseURL                   string // Override for testing; defaults to googleSTTBaseURL
	Validation                netsec.ValidationOptions
	// contains filtered or unexported fields
}

GoogleSTTProvider implements STTProvider for Google Cloud Speech-to-Text v1 REST API.

BaseURL is user-configurable (for testing or regional endpoints). It is validated against Validation on every request. Default Validation is strict (public https only).

func NewGoogleSTTProvider

func NewGoogleSTTProvider(apiKey, model string) *GoogleSTTProvider

NewGoogleSTTProvider creates a provider for Google Cloud Speech-to-Text. Model defaults to "latest_long" if empty.

func (*GoogleSTTProvider) Capabilities

func (*GoogleSTTProvider) Capabilities() []speechkit.Capability

func (*GoogleSTTProvider) Health

func (p *GoogleSTTProvider) Health(ctx context.Context) error

Health checks if the Google Speech API is reachable.

func (*GoogleSTTProvider) Name

func (p *GoogleSTTProvider) Name() string

Name returns the provider identifier.

func (*GoogleSTTProvider) SetStreamingCredentialEnvs

func (p *GoogleSTTProvider) SetStreamingCredentialEnvs(credentialsJSONEnv, applicationCredentialsEnv string)

func (*GoogleSTTProvider) StartSpeakerStream

func (p *GoogleSTTProvider) StartSpeakerStream(ctx context.Context, opts speaker.Options, format speaker.AudioFormat) (speaker.SpeakerStream, error)

StartSpeakerStream opens a Google Cloud Speech-to-Text v2 StreamingRecognize session.

IMPORTANT: Google STT v2 does NOT support speaker diarization in streaming mode — diarization is available only in BatchRecognize/Recognize (see the Chirp-3 docs). This adapter therefore provides realtime TRANSCRIPTION only: frames carry Text without speaker labels (no Segment/Speakers). Streaming speaker diarization in the Voice Agent comes from Deepgram/AssemblyAI.

func (*GoogleSTTProvider) Transcribe

func (p *GoogleSTTProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)

Transcribe sends audio to Google Cloud Speech-to-Text v1 REST API.

type HuggingFaceProvider

type HuggingFaceProvider struct {
	Model      string
	Token      string
	BaseURL    string // Override for testing; defaults to hfBaseURL
	Validation netsec.ValidationOptions
	// contains filtered or unexported fields
}

HuggingFaceProvider implements STTProvider for Tier 3: HuggingFace Inference API.

BaseURL is user-configurable. It is validated against Validation on every request. Default Validation is strict (public https only).

func NewHuggingFaceProvider

func NewHuggingFaceProvider(model, token string) *HuggingFaceProvider

func (*HuggingFaceProvider) Capabilities

func (*HuggingFaceProvider) Capabilities() []speechkit.Capability

func (*HuggingFaceProvider) Health

func (p *HuggingFaceProvider) Health(ctx context.Context) error

func (*HuggingFaceProvider) Name

func (p *HuggingFaceProvider) Name() string

func (*HuggingFaceProvider) Transcribe

func (p *HuggingFaceProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)

type InstallStatus

type InstallStatus struct {
	BinaryFound bool     `json:"binaryFound"`
	BinaryPath  string   `json:"binaryPath"`
	ModelFound  bool     `json:"modelFound"`
	ModelPath   string   `json:"modelPath"`
	ModelBytes  int64    `json:"modelBytes"`
	ServerReady bool     `json:"serverReady"`
	Problems    []string `json:"problems,omitempty"`
}

InstallStatus describes what's present and what's missing for local STT.

type LocalProvider

type LocalProvider struct {
	BaseURL    string // e.g. "http://127.0.0.1:8080"
	Port       int
	ModelPath  string
	GPU        string
	Validation netsec.ValidationOptions
	// contains filtered or unexported fields
}

LocalProvider implements STTProvider for Tier 1: localhost whisper.cpp server.

func NewLocalProvider

func NewLocalProvider(port int, modelPath, gpu string) *LocalProvider

func (*LocalProvider) Capabilities

func (*LocalProvider) Capabilities() []speechkit.Capability

func (*LocalProvider) Health

func (p *LocalProvider) Health(ctx context.Context) error

func (*LocalProvider) IsReady

func (p *LocalProvider) IsReady() bool

IsReady returns true if the whisper-server subprocess is running and responding.

func (*LocalProvider) Name

func (p *LocalProvider) Name() string

func (*LocalProvider) StartServer

func (p *LocalProvider) StartServer(ctx context.Context) error

StartServer starts the whisper.cpp server subprocess. Blocks until ready or context cancelled.

func (*LocalProvider) StopServer

func (p *LocalProvider) StopServer()

StopServer terminates the whisper-server subprocess.

func (*LocalProvider) Transcribe

func (p *LocalProvider) Transcribe(ctx context.Context, audioData []byte, opts TranscribeOpts) (*Result, error)

func (*LocalProvider) VerifyInstallation

func (p *LocalProvider) VerifyInstallation() InstallStatus

VerifyInstallation checks binary and model availability without starting the server.

type OpenAICompatibleProvider

type OpenAICompatibleProvider struct {
	BaseURL    string
	APIKey     string
	Model      string
	Validation netsec.ValidationOptions
	// contains filtered or unexported fields
}

OpenAICompatibleProvider implements STTProvider for any endpoint speaking the OpenAI /v1/audio/transcriptions API (OpenAI, Groq, VPS whisper-server, etc.).

BaseURL is user-supplied configuration. It is validated against Validation on every request (Transcribe, Health). The default Validation is strict: only public https:// endpoints are accepted. Self-hosted VPS and local whisper-server require relaxing Validation — see NewVPSProvider.

func NewGroqSTTProvider

func NewGroqSTTProvider(apiKey string) *OpenAICompatibleProvider

NewGroqSTTProvider creates a provider for the Groq Whisper API.

func NewOllamaSTTProvider

func NewOllamaSTTProvider(baseURL, model string) *OpenAICompatibleProvider

NewOllamaSTTProvider creates a provider for Ollama-compatible local transcription endpoints. Ollama runs on loopback by default and can be pointed at a user-managed self-hosted URL.

func NewOpenAICompatibleProvider

func NewOpenAICompatibleProvider(name, baseURL, apiKey, model string) *OpenAICompatibleProvider

NewOpenAICompatibleProvider creates a provider for any OpenAI-compatible STT endpoint. Default Validation is strict (public https only). Callers with a non-public endpoint (loopback, RFC1918) must set Validation explicitly.

func NewOpenAISTTProvider

func NewOpenAISTTProvider(apiKey string) *OpenAICompatibleProvider

NewOpenAISTTProvider creates a provider for the OpenAI Whisper API.

func NewVPSProvider

func NewVPSProvider(baseURL, apiKey string) *OpenAICompatibleProvider

NewVPSProvider creates a provider for a self-hosted whisper-server. Allows loopback, private IP ranges and plain http:// because self-hosted deployments frequently run inside a VPN, on a home LAN, or on localhost.

func NewVPSProviderWithModel

func NewVPSProviderWithModel(baseURL, apiKey, model string) *OpenAICompatibleProvider

func (*OpenAICompatibleProvider) Capabilities

func (*OpenAICompatibleProvider) Capabilities() []speechkit.Capability

func (*OpenAICompatibleProvider) Health

Health checks provider reachability. Tries GET /health first (whisper-server), then falls back to GET /v1/models (OpenAI, Groq).

func (*OpenAICompatibleProvider) Name

func (p *OpenAICompatibleProvider) Name() string

Name returns the provider identifier.

func (*OpenAICompatibleProvider) Transcribe

func (p *OpenAICompatibleProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)

Transcribe sends audio to the OpenAI-compatible /v1/audio/transcriptions endpoint.

type OpenRouterSTTProvider

type OpenRouterSTTProvider struct {
	BaseURL    string
	APIKey     string
	Model      string
	Validation netsec.ValidationOptions
	// contains filtered or unexported fields
}

OpenRouterSTTProvider implements OpenRouter's JSON speech-to-text endpoint. OpenRouter is a cloud gateway, not a direct model provider, and its STT API accepts base64 audio rather than OpenAI's multipart Whisper shape.

func NewOpenRouterSTTProvider

func NewOpenRouterSTTProvider(apiKey, model string) *OpenRouterSTTProvider

func (*OpenRouterSTTProvider) Capabilities

func (*OpenRouterSTTProvider) Capabilities() []speechkit.Capability

func (*OpenRouterSTTProvider) Health

func (p *OpenRouterSTTProvider) Health(ctx context.Context) error

func (*OpenRouterSTTProvider) Name

func (p *OpenRouterSTTProvider) Name() string

func (*OpenRouterSTTProvider) Transcribe

func (p *OpenRouterSTTProvider) Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)

type ResolvedTranscribeOptions

type ResolvedTranscribeOptions struct {
	Language              string
	Model                 string
	Prompt                string
	ContextPrompt         string
	LanguageHints         []string
	Keyterms              []string
	Speaker               speaker.Options
	DetectLanguage        bool
	Punctuation           bool
	SmartFormat           bool
	Dictation             bool
	FillerWords           bool
	Numerals              bool
	UseVocabularyKeyterms bool
	Timestamps            bool
	EndpointingMs         int
	PrivacyRedaction      bool
	VoiceFocus            bool
	MedicalDomain         bool
	Effective             provideropts.EffectiveOptions
}

func ResolveTranscribeOptions

func ResolveTranscribeOptions(provider, profileID string, opts TranscribeOpts, providerDefaults, providerOverrides provideropts.Values) ResolvedTranscribeOptions

func (ResolvedTranscribeOptions) APILanguage

func (r ResolvedTranscribeOptions) APILanguage() string

type Result

type Result struct {
	Text       string
	Language   string
	Duration   time.Duration
	Provider   string
	Model      string
	Confidence float64 // If available from the provider
	// Words carries per-word acoustic confidence when the provider exposes it.
	// nil for providers without word-level confidence. Offsets are NOT tracked
	// because downstream vocabulary/punctuation rewriting invalidates them;
	// consumers match on the word text instead.
	Words    []WordConfidence
	Speakers *speaker.DiarizationResult
}

Result holds the output of a transcription.

type STTProvider

type STTProvider interface {
	// Transcribe sends audio data to the STT backend and returns the transcription.
	Transcribe(ctx context.Context, audio []byte, opts TranscribeOpts) (*Result, error)

	// Name returns the provider identifier (e.g. "local", "vps", "huggingface").
	Name() string

	// Health checks if the provider is reachable and ready.
	Health(ctx context.Context) error
}

STTProvider defines the interface for all speech-to-text backends.

type TranscribeOpts

type TranscribeOpts struct {
	Language                  string                         // "de", "en", "auto"; request override only
	Model                     string                         // Optional: model override
	Prompt                    string                         // Optional: provider-specific hint prompt for better recognition
	Keyterms                  []string                       // Optional: provider-native vocabulary bias terms
	Speaker                   speaker.Options                // Optional speaker diarization / attribution request
	Options                   provideropts.Values            // Optional normalized global/default voice options
	ProviderOptions           provideropts.Values            // Optional normalized overrides for the selected provider
	ProviderOptionsByProvider map[string]provideropts.Values // Optional provider-keyed overrides used by routers
}

TranscribeOpts configures a single transcription request.

func (TranscribeOpts) ForProvider

func (o TranscribeOpts) ForProvider(provider string) TranscribeOpts

type VPSProvider

type VPSProvider = OpenAICompatibleProvider

VPSProvider is an alias for backward compatibility. Use OpenAICompatibleProvider directly for new code.

type WordConfidence

type WordConfidence struct {
	Text       string
	Confidence float64
	StartMs    int64
	EndMs      int64
}

WordConfidence is a single recognized word with the provider's per-word acoustic confidence in [0,1]. It is populated only by providers that expose word-level confidence (Deepgram, AssemblyAI); it stays nil for providers that do not (Google v1, OpenAI/Groq/HuggingFace Whisper, local whisper.cpp). Confidence here is acoustic (how sure the model is it heard this word), not semantic — a low value flags a likely mis-recognition or dropped word.

Directories

Path Synopsis
Package sttcontract provides a reusable conformance suite that every stt.STTProvider implementation is expected to satisfy.
Package sttcontract provides a reusable conformance suite that every stt.STTProvider implementation is expected to satisfy.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL