Documentation
¶
Overview ¶
Package nvidia provides NVIDIA services: an NVIDIA NIM OpenAI-compatible LLM (NewLLM), a Riva streaming speech-to-text service (NewSTT), and a Riva streaming text-to-speech service (NewTTS). Both speech services talk to NVIDIA's hosted endpoints or to a locally deployed Riva/NIM model (parakeet for recognition, magpie for synthesis), selected through the server address, the TLS setting, and the auth fields.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewLLM ¶
func NewLLM(cfg chat.LLMConfig) *chat.LLMService
NewLLM builds an NVIDIA NIM LLM service.
func NewSTT ¶ added in v0.1.0
func NewSTT(cfg STTConfig) *stt.StreamService
NewSTT builds an NVIDIA Riva streaming speech-to-text service. A finalized result (Riva's is_final, driven by server-side endpointing) marks the end of the user's turn.
func NewSegmentedSTT ¶ added in v0.1.0
func NewSegmentedSTT(cfg SegmentedSTTConfig) *stt.SegmentService
NewSegmentedSTT builds an NVIDIA Riva batch speech-to-text service. It transcribes each utterance in one call once a turn detector upstream has delimited it, unlike NewSTT which streams continuously. Use it with Riva's offline models, which are more accurate but produce no interim transcripts.
Types ¶
type Endpointing ¶ added in v0.1.0
type Endpointing struct {
StartHistory *int32
StartThreshold *float32
StopHistory *int32
StopThreshold *float32
StopHistoryEOU *int32
StopThresholdEOU *float32
}
Endpointing tunes Riva's server-side start/end-of-utterance detector. Every field is optional; a nil field keeps the server's built-in default. The history fields are window sizes in milliseconds and the threshold fields are fractions in [0, 1].
type STTConfig ¶ added in v0.1.0
type STTConfig struct {
// Server is the Riva gRPC endpoint; empty uses NVIDIA's hosted endpoint. For
// a local NIM this is typically "localhost:50051".
Server string
// APIKey is the NVIDIA API key, sent as an "authorization: Bearer" header.
// Required by the hosted endpoint; omit for an unauthenticated local NIM.
APIKey string
// FunctionID is the NVIDIA Cloud Function id, sent as a "function-id" header.
// Required by the hosted endpoint to select the model; omit for a local NIM.
// It identifies one deployment of a model rather than the model itself, so
// it changes whenever NVIDIA redeploys; the current one is on the model's
// page under build.nvidia.com.
FunctionID string
// Model selects a served model by name; empty lets the server choose. A local
// Riva build usually serves one model, so this can stay empty.
Model string
// Language of the audio, mapped to a Riva BCP-47 code; the zero value uses US
// English.
Language language.Language
// SampleRate is the input audio sample rate; 0 uses the transport's rate.
SampleRate int
// UseSSL wraps the gRPC connection in TLS; nil defaults to true. Set to a
// pointer to false for a plaintext local NIM.
UseSSL *bool
// InterimResults requests tentative partial transcripts; nil defaults to true.
InterimResults *bool
// AutomaticPunctuation adds punctuation to results; nil defaults to true.
AutomaticPunctuation *bool
// VerbatimTranscripts disables inverse text normalization; nil defaults to
// true.
VerbatimTranscripts *bool
// ProfanityFilter masks profanities in the transcript.
ProfanityFilter bool
// MaxAlternatives caps the hypotheses returned per result; 0 uses 1.
MaxAlternatives int
// AudioChannelCount is the input channel count; 0 uses 1 (mono).
AudioChannelCount int
// CustomConfiguration passes request-level options to the model pipeline
// (for example "enable_vad_endpointing:true"), keyed by option name.
CustomConfiguration map[string]string
// Endpointing tunes the server-side utterance detector; nil uses the server
// defaults.
Endpointing *Endpointing
// TTFSP99 overrides the measured transcript latency the turn strategies
// size their wait by; 0 uses stt.NvidiaTTFSP99.
TTFSP99 time.Duration
}
STTConfig configures the NVIDIA Riva streaming speech-to-text service. The same service talks to NVIDIA's hosted ASR endpoint and to a locally deployed Riva/NIM model (such as parakeet); the deployment is selected entirely through Server, UseSSL, and the auth fields.
type SegmentedSTTConfig ¶ added in v0.1.0
type SegmentedSTTConfig struct {
// Server is the Riva gRPC endpoint; empty uses NVIDIA's hosted endpoint. For
// a local NIM this is typically "localhost:50051".
Server string
// APIKey is the NVIDIA API key, sent as an "authorization: Bearer" header.
// Required by the hosted endpoint; omit for an unauthenticated local NIM.
APIKey string
// FunctionID is the NVIDIA Cloud Function id, sent as a "function-id" header.
// Required by the hosted endpoint to select the model; omit for a local NIM.
// It identifies one deployment of a model rather than the model itself, so
// it changes whenever NVIDIA redeploys; the current one is on the model's
// page under build.nvidia.com.
FunctionID string
// Model selects a served model by name; empty lets the server choose.
Model string
// Language of the audio, mapped to a Riva BCP-47 code; the zero value uses US
// English.
Language language.Language
// SampleRate is the input audio sample rate; 0 uses the transport's rate.
SampleRate int
// UseSSL wraps the gRPC connection in TLS; nil defaults to true. Set to a
// pointer to false for a plaintext local NIM.
UseSSL *bool
// AutomaticPunctuation adds punctuation to results; nil defaults to true.
AutomaticPunctuation *bool
// VerbatimTranscripts disables inverse text normalization; nil defaults to
// true.
VerbatimTranscripts *bool
// ProfanityFilter masks profanities in the transcript.
ProfanityFilter bool
// MaxAlternatives caps the hypotheses returned per result; 0 uses 1.
MaxAlternatives int
// AudioChannelCount is the input channel count; 0 uses 1 (mono).
AudioChannelCount int
// CustomConfiguration passes request-level options to the model pipeline,
// keyed by option name.
CustomConfiguration map[string]string
// TTFSP99 overrides the measured transcript latency the turn strategies
// size their wait by; 0 uses stt.NvidiaTTFSP99.
TTFSP99 time.Duration
}
SegmentedSTTConfig configures the NVIDIA Riva batch speech-to-text service. It transcribes one complete utterance per call against Riva's offline models, so it needs a turn detector upstream to delimit each segment.
func (SegmentedSTTConfig) Validate ¶ added in v0.1.0
func (c SegmentedSTTConfig) Validate() error
Validate reports whether the configuration is usable.
type TTSConfig ¶ added in v0.1.0
type TTSConfig struct {
// Server is the Riva gRPC endpoint; empty uses NVIDIA's hosted endpoint. For
// a local NIM this is typically "localhost:50051".
Server string
// APIKey is the NVIDIA API key, sent as an "authorization: Bearer" header.
// Required by the hosted endpoint; omit for an unauthenticated local NIM.
APIKey string
// FunctionID is the NVIDIA Cloud Function id, sent as a "function-id" header.
// It selects the model on the hosted endpoint; empty uses the multilingual
// Magpie function. Set it to "-" to send no function id, as a local NIM
// expects.
FunctionID string
// Model names the served model. It labels the metrics and traces and is what
// a cost-tracking backend prices against; empty uses the hosted default. It
// does not select the model, FunctionID does.
Model string
// Voice is the voice name; empty uses a multilingual Magpie voice.
Voice string
// Language for synthesis, mapped to a Riva BCP-47 code; the zero value uses
// US English.
Language language.Language
// SampleRate is the PCM rate requested from the server and emitted
// downstream; 0 uses 24 kHz. Rates below 8 kHz do not produce usable audio.
SampleRate int
// UseSSL wraps the gRPC connection in TLS; nil defaults to true. Set to a
// pointer to false for a plaintext local NIM.
UseSSL *bool
// CustomDictionary maps a written form to its IPA pronunciation, for example
// {"NVIDIA": "ɛn.vɪ.diː.ʌ"}; empty sends none.
CustomDictionary map[string]string
// CustomConfiguration passes model-specific options the schema does not name
// (for example "exaggeration_factor"), keyed by option name.
CustomConfiguration map[string]string
// ZeroShot supplies the voice-cloning prompt a zero-shot model needs; nil
// omits it, which is what every other model expects.
ZeroShot *ZeroShot `validate:"omitempty"`
}
TTSConfig configures the NVIDIA Riva streaming text-to-speech service. The same service talks to NVIDIA's hosted TTS endpoint and to a locally deployed Riva/NIM model; the deployment is selected entirely through Server, UseSSL, and the auth fields.
type ZeroShot ¶ added in v0.1.0
type ZeroShot struct {
// AudioPrompt is the prompt audio. NVIDIA recommends 16-bit mono, 22.05 kHz
// or higher, and 3 to 10 seconds long. Required.
AudioPrompt []byte `validate:"required,min=1"`
// Encoding of the prompt, either "pcm" (the default) or "oggopus".
Encoding string `validate:"omitempty,oneof=pcm oggopus"`
// SampleRate of the prompt; 0 lets the server assume 22050.
SampleRate int
// Quality is how many times the prompt passes through the decoder, 1 to 40;
// 0 uses 20.
Quality int `validate:"omitempty,min=1,max=40"`
// Transcript of the prompt audio; empty sends none.
Transcript string
}
ZeroShot supplies the audio prompt a zero-shot model clones its voice from. Access to NVIDIA's hosted zero-shot models needs approval.
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
rivapb
Package rivapb holds the generated gRPC clients for the subset of the NVIDIA Riva speech API the streaming ASR and TTS services use.
|
Package rivapb holds the generated gRPC clients for the subset of the NVIDIA Riva speech API the streaming ASR and TTS services use. |