Documentation
¶
Overview ¶
SPDX-License-Identifier: MIT Purpose: NVIDIA NIM-specific helpers. NIM exposes an OpenAI-compatible chat/completions endpoint at https://integrate.api.nvidia.com/v1 and serves Anthropic / Llama / Mistral / etc. models under "vendor/name" IDs. Friendly aliases like "haiku" or "sonnet" resolve to the full ID.
SPDX-License-Identifier: MIT Purpose: generic OpenAI-compatible LLM client. Single-shot chat completion request with bearer auth, JSON marshaling, and typed response decoding.
Index ¶
Constants ¶
View Source
const NIMClaudeModel = "anthropic/claude-3-5-sonnet-20241022"
View Source
const NIMDefaultBaseURL = "https://integrate.api.nvidia.com/v1"
View Source
const NIMDefaultModel = "meta/llama-3.1-70b-instruct"
View Source
const NIMHaikuModel = "anthropic/claude-3-5-haiku-20241022"
Variables ¶
View Source
var NIMModelAliases = map[string]string{ "haiku": NIMHaikuModel, "sonnet": NIMClaudeModel, "llama-70b": NIMDefaultModel, "llama-8b": "meta/llama-3.1-8b-instruct", }
Functions ¶
func ResolveModel ¶
Types ¶
type ChatRequest ¶
type ChatResponse ¶
type ChatResponse struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
Model string `json:"model"`
Choices []struct {
Index int `json:"index"`
Message Message `json:"message"`
FinishReason string `json:"finish_reason"`
} `json:"choices"`
Usage struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
} `json:"usage"`
}
func (*ChatResponse) ExtractText ¶
func (r *ChatResponse) ExtractText() string
type Client ¶
func (*Client) Chat ¶
func (c *Client) Chat(ctx context.Context, req ChatRequest) (*ChatResponse, error)
Click to show internal directories.
Click to hide internal directories.