nvidia

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 30, 2026 License: BSD-2-Clause Imports: 20 Imported by: 0

Documentation

Overview

Package nvidia provides NVIDIA services: an NVIDIA NIM OpenAI-compatible LLM (NewLLM), a Riva streaming speech-to-text service (NewSTT), and a Riva streaming text-to-speech service (NewTTS). Both speech services talk to NVIDIA's hosted endpoints or to a locally deployed Riva/NIM model (parakeet for recognition, magpie for synthesis), selected through the server address, the TLS setting, and the auth fields.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewLLM

func NewLLM(cfg chat.LLMConfig) *chat.LLMService

NewLLM builds an NVIDIA NIM LLM service.

func NewSTT added in v0.1.0

func NewSTT(cfg STTConfig) *stt.StreamService

NewSTT builds an NVIDIA Riva streaming speech-to-text service. A finalized result (Riva's is_final, driven by server-side endpointing) marks the end of the user's turn.

func NewSegmentedSTT added in v0.1.0

func NewSegmentedSTT(cfg SegmentedSTTConfig) *stt.SegmentService

NewSegmentedSTT builds an NVIDIA Riva batch speech-to-text service. It transcribes each utterance in one call once a turn detector upstream has delimited it, unlike NewSTT which streams continuously. Use it with Riva's offline models, which are more accurate but produce no interim transcripts.

func NewTTS added in v0.1.0

func NewTTS(cfg TTSConfig) *tts.Base

NewTTS builds an NVIDIA Riva streaming text-to-speech service. Each sentence opens a SynthesizeOnline stream, sends the text (split when the model's request-length limit requires it), and streams the generated audio downstream.

Types

type Endpointing added in v0.1.0

type Endpointing struct {
	StartHistory     *int32
	StartThreshold   *float32
	StopHistory      *int32
	StopThreshold    *float32
	StopHistoryEOU   *int32
	StopThresholdEOU *float32
}

Endpointing tunes Riva's server-side start/end-of-utterance detector. Every field is optional; a nil field keeps the server's built-in default. The history fields are window sizes in milliseconds and the threshold fields are fractions in [0, 1].

type STTConfig added in v0.1.0

type STTConfig struct {
	// Server is the Riva gRPC endpoint; empty uses NVIDIA's hosted endpoint. For
	// a local NIM this is typically "localhost:50051".
	Server string
	// APIKey is the NVIDIA API key, sent as an "authorization: Bearer" header.
	// Required by the hosted endpoint; omit for an unauthenticated local NIM.
	APIKey string
	// FunctionID is the NVIDIA Cloud Function id, sent as a "function-id" header.
	// Required by the hosted endpoint to select the model; omit for a local NIM.
	// It identifies one deployment of a model rather than the model itself, so
	// it changes whenever NVIDIA redeploys; the current one is on the model's
	// page under build.nvidia.com.
	FunctionID string
	// Model selects a served model by name; empty lets the server choose. A local
	// Riva build usually serves one model, so this can stay empty.
	Model string
	// Language of the audio, mapped to a Riva BCP-47 code; the zero value uses US
	// English.
	Language language.Language
	// SampleRate is the input audio sample rate; 0 uses the transport's rate.
	SampleRate int

	// UseSSL wraps the gRPC connection in TLS; nil defaults to true. Set to a
	// pointer to false for a plaintext local NIM.
	UseSSL *bool
	// InterimResults requests tentative partial transcripts; nil defaults to true.
	InterimResults *bool
	// AutomaticPunctuation adds punctuation to results; nil defaults to true.
	AutomaticPunctuation *bool
	// VerbatimTranscripts disables inverse text normalization; nil defaults to
	// true.
	VerbatimTranscripts *bool
	// ProfanityFilter masks profanities in the transcript.
	ProfanityFilter bool
	// MaxAlternatives caps the hypotheses returned per result; 0 uses 1.
	MaxAlternatives int
	// AudioChannelCount is the input channel count; 0 uses 1 (mono).
	AudioChannelCount int
	// CustomConfiguration passes request-level options to the model pipeline
	// (for example "enable_vad_endpointing:true"), keyed by option name.
	CustomConfiguration map[string]string
	// Endpointing tunes the server-side utterance detector; nil uses the server
	// defaults.
	Endpointing *Endpointing

	// TTFSP99 overrides the measured transcript latency the turn strategies
	// size their wait by; 0 uses stt.NvidiaTTFSP99.
	TTFSP99 time.Duration
}

STTConfig configures the NVIDIA Riva streaming speech-to-text service. The same service talks to NVIDIA's hosted ASR endpoint and to a locally deployed Riva/NIM model (such as parakeet); the deployment is selected entirely through Server, UseSSL, and the auth fields.

func (STTConfig) Validate added in v0.1.0

func (c STTConfig) Validate() error

Validate reports whether the configuration is usable.

type SegmentedSTTConfig added in v0.1.0

type SegmentedSTTConfig struct {
	// Server is the Riva gRPC endpoint; empty uses NVIDIA's hosted endpoint. For
	// a local NIM this is typically "localhost:50051".
	Server string
	// APIKey is the NVIDIA API key, sent as an "authorization: Bearer" header.
	// Required by the hosted endpoint; omit for an unauthenticated local NIM.
	APIKey string
	// FunctionID is the NVIDIA Cloud Function id, sent as a "function-id" header.
	// Required by the hosted endpoint to select the model; omit for a local NIM.
	// It identifies one deployment of a model rather than the model itself, so
	// it changes whenever NVIDIA redeploys; the current one is on the model's
	// page under build.nvidia.com.
	FunctionID string
	// Model selects a served model by name; empty lets the server choose.
	Model string
	// Language of the audio, mapped to a Riva BCP-47 code; the zero value uses US
	// English.
	Language language.Language
	// SampleRate is the input audio sample rate; 0 uses the transport's rate.
	SampleRate int
	// UseSSL wraps the gRPC connection in TLS; nil defaults to true. Set to a
	// pointer to false for a plaintext local NIM.
	UseSSL *bool
	// AutomaticPunctuation adds punctuation to results; nil defaults to true.
	AutomaticPunctuation *bool
	// VerbatimTranscripts disables inverse text normalization; nil defaults to
	// true.
	VerbatimTranscripts *bool
	// ProfanityFilter masks profanities in the transcript.
	ProfanityFilter bool
	// MaxAlternatives caps the hypotheses returned per result; 0 uses 1.
	MaxAlternatives int
	// AudioChannelCount is the input channel count; 0 uses 1 (mono).
	AudioChannelCount int
	// CustomConfiguration passes request-level options to the model pipeline,
	// keyed by option name.
	CustomConfiguration map[string]string

	// TTFSP99 overrides the measured transcript latency the turn strategies
	// size their wait by; 0 uses stt.NvidiaTTFSP99.
	TTFSP99 time.Duration
}

SegmentedSTTConfig configures the NVIDIA Riva batch speech-to-text service. It transcribes one complete utterance per call against Riva's offline models, so it needs a turn detector upstream to delimit each segment.

func (SegmentedSTTConfig) Validate added in v0.1.0

func (c SegmentedSTTConfig) Validate() error

Validate reports whether the configuration is usable.

type TTSConfig added in v0.1.0

type TTSConfig struct {
	// Server is the Riva gRPC endpoint; empty uses NVIDIA's hosted endpoint. For
	// a local NIM this is typically "localhost:50051".
	Server string
	// APIKey is the NVIDIA API key, sent as an "authorization: Bearer" header.
	// Required by the hosted endpoint; omit for an unauthenticated local NIM.
	APIKey string
	// FunctionID is the NVIDIA Cloud Function id, sent as a "function-id" header.
	// It selects the model on the hosted endpoint; empty uses the multilingual
	// Magpie function. Set it to "-" to send no function id, as a local NIM
	// expects.
	FunctionID string
	// Model names the served model. It labels the metrics and traces and is what
	// a cost-tracking backend prices against; empty uses the hosted default. It
	// does not select the model, FunctionID does.
	Model string
	// Voice is the voice name; empty uses a multilingual Magpie voice.
	Voice string
	// Language for synthesis, mapped to a Riva BCP-47 code; the zero value uses
	// US English.
	Language language.Language
	// SampleRate is the PCM rate requested from the server and emitted
	// downstream; 0 uses 24 kHz. Rates below 8 kHz do not produce usable audio.
	SampleRate int
	// UseSSL wraps the gRPC connection in TLS; nil defaults to true. Set to a
	// pointer to false for a plaintext local NIM.
	UseSSL *bool
	// CustomDictionary maps a written form to its IPA pronunciation, for example
	// {"NVIDIA": "ɛn.vɪ.diː.ʌ"}; empty sends none.
	CustomDictionary map[string]string
	// CustomConfiguration passes model-specific options the schema does not name
	// (for example "exaggeration_factor"), keyed by option name.
	CustomConfiguration map[string]string
	// ZeroShot supplies the voice-cloning prompt a zero-shot model needs; nil
	// omits it, which is what every other model expects.
	ZeroShot *ZeroShot `validate:"omitempty"`
}

TTSConfig configures the NVIDIA Riva streaming text-to-speech service. The same service talks to NVIDIA's hosted TTS endpoint and to a locally deployed Riva/NIM model; the deployment is selected entirely through Server, UseSSL, and the auth fields.

func (TTSConfig) Validate added in v0.1.0

func (c TTSConfig) Validate() error

Validate reports whether the configuration is usable.

type ZeroShot added in v0.1.0

type ZeroShot struct {
	// AudioPrompt is the prompt audio. NVIDIA recommends 16-bit mono, 22.05 kHz
	// or higher, and 3 to 10 seconds long. Required.
	AudioPrompt []byte `validate:"required,min=1"`
	// Encoding of the prompt, either "pcm" (the default) or "oggopus".
	Encoding string `validate:"omitempty,oneof=pcm oggopus"`
	// SampleRate of the prompt; 0 lets the server assume 22050.
	SampleRate int
	// Quality is how many times the prompt passes through the decoder, 1 to 40;
	// 0 uses 20.
	Quality int `validate:"omitempty,min=1,max=40"`
	// Transcript of the prompt audio; empty sends none.
	Transcript string
}

ZeroShot supplies the audio prompt a zero-shot model clones its voice from. Access to NVIDIA's hosted zero-shot models needs approval.

Directories

Path Synopsis
internal
rivapb
Package rivapb holds the generated gRPC clients for the subset of the NVIDIA Riva speech API the streaming ASR and TTS services use.
Package rivapb holds the generated gRPC clients for the subset of the NVIDIA Riva speech API the streaming ASR and TTS services use.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL