Documentation
¶
Overview ¶
Package nvidia is a reference backend emulator for NVIDIA NIM's API surface. It serves both POST /v1/chat/completions and POST /v1/responses with JSON or SSE bodies compatible with github.com/openai/openai-go/v3, providing request body/header capture for integration test assertions.
Index ¶
Constants ¶
const ChatStreamWithUsageSSE = "data: {\"id\":\"gen-nvidia-refbackend-stream\",\"object\":\"chat.completion.chunk\",\"created\":1715620000,\"model\":\"nvidia/llama-3.1-nemotron-nano-8b-v1\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"nvidia-stream-ok\"},\"finish_reason\":null}]}\n\n" +
"data: {\"id\":\"gen-nvidia-refbackend-stream\",\"object\":\"chat.completion.chunk\",\"created\":1715620000,\"model\":\"nvidia/llama-3.1-nemotron-nano-8b-v1\",\"choices\":[{\"index\":0,\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n" +
"data: {\"id\":\"gen-nvidia-refbackend-stream\",\"object\":\"chat.completion.chunk\",\"created\":1715620000,\"model\":\"nvidia/llama-3.1-nemotron-nano-8b-v1\",\"choices\":[],\"usage\":{\"prompt_tokens\":3,\"completion_tokens\":7,\"total_tokens\":10}}\n\n" +
"data: [DONE]\n\n"
ChatStreamWithUsageSSE includes a final usage chunk before [DONE].
const DefaultChatNonStreamJSON = `` /* 320-byte string literal not displayed */
DefaultChatNonStreamJSON is a minimal Chat Completions response for NVIDIA NIM.
const DefaultChatStreamSSE = "data: {\"id\":\"gen-nvidia-refbackend-stream\",\"object\":\"chat.completion.chunk\",\"created\":1715620000,\"model\":\"nvidia/llama-3.1-nemotron-nano-8b-v1\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"nvidia-stream-ok\"},\"finish_reason\":null}]}\n\n" +
"data: [DONE]\n\n"
DefaultChatStreamSSE is a minimal streaming Chat Completions response.
const DefaultResponsesNonStreamJSON = `` /* 448-byte string literal not displayed */
DefaultResponsesNonStreamJSON is a minimal Responses API completed response.
const DefaultResponsesStreamSSE = "event: response.completed\n" +
"data: {\"type\":\"response.completed\",\"sequence_number\":1,\"response\":{\"id\":\"resp_nvidia_refbackend_stream\",\"object\":\"response\",\"created_at\":1715620000,\"status\":\"completed\",\"model\":\"nvidia/llama-3.1-nemotron-nano-8b-v1\",\"output\":[{\"type\":\"message\",\"id\":\"m1\",\"status\":\"completed\",\"role\":\"assistant\",\"content\":[{\"type\":\"output_text\",\"text\":\"nvidia-stream-ok\"}]}],\"usage\":{\"input_tokens\":3,\"output_tokens\":7,\"total_tokens\":10}}}\n\n" +
"data: [DONE]\n\n"
DefaultResponsesStreamSSE is a minimal Responses SSE stream.
Variables ¶
This section is empty.
Functions ¶
func NewHandler ¶
NewHandler returns an http.Handler that emulates NVIDIA NIM endpoints: POST /v1/chat/completions and POST /v1/responses.
Types ¶
type Config ¶
type Config struct {
AllowMissingBearer bool
OnAuthorizedCredential func(secret string)
ForcedHTTPStatus int
ForcedRetryAfter string
ForcedErrorJSON string
OnRequestBody func(body []byte)
OnRequestHeaders func(h http.Header)
ChatNonStreamJSON string
ChatStreamSSE string
ResponsesNonStreamJSON string
ResponsesStreamSSE string
}
Config tunes the emulator handler.
Source Files