Documentation
¶
Overview ¶
Package openaichat is the openai-chat dialect: POST {base_url}/chat/completions, the wire shape Vertex AI's partner models, xAI, vLLM, SGLang, Ollama, llama.cpp, NVIDIA NIM and the managed long tail all speak (docs/design.md §7.1).
It is written on net/http and its own wire types rather than an SDK. These servers are OpenAI-shaped, not OpenAI: they add fields (reasoning_content), omit fields (usage details, tool-call ids) and differ on what they accept, and pointer-typed wire structs keep a count the server never sent distinct from a zero it did — the distinction usage.Detail exists for.
Retries happen in the HTTP client (package retry), so every attempt gets a fresh credential and a provider's Retry-After is honored.
Index ¶
Constants ¶
const ( // RefusalKey marks a response whose text is the model's refusal. RefusalKey = "core_models.refusal" // FinishReasonKey holds a finish reason with no genai equivalent. FinishReasonKey = "core_models.finish_reason" )
Metadata keys this dialect writes beside usage.MetadataKey.
const ToolChoiceKey = "core_models.tool_choice_downgraded"
ToolChoiceKey marks a response whose request asked for a forced tool call the model cannot take, and was sent with "auto" instead.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type APIError ¶
APIError is a non-2xx answer, or an error event inside a stream. StatusCode is 0 for an in-stream error.
func (*APIError) HTTPStatus ¶
HTTPStatus reports the status, for retry classifiers that sit above this package.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is a connection to one server. Safe for concurrent use.
type ModelOptions ¶ added in v0.2.0
type ModelOptions struct {
ResponseSchema *bool
ReasoningEcho *bool
NoForcedToolChoice *bool
// ExtraBody, when non-nil, replaces the Client's.
ExtraBody map[string]any
}
ModelOptions are the per-model refinements of Options: one server serves many models, and what it honors can differ between them.
type Options ¶
type Options struct {
// BaseURL is the API root; requests go to BaseURL + "/chat/completions".
BaseURL string
// Credential authenticates every request as a bearer token. Nil
// sends none.
Credential *auth.Credential
// HTTPClient supplies the base transport and timeout. Nil means a
// client with http.DefaultTransport.
HTTPClient *http.Client
// Retry is the HTTP-layer retry policy. Nil means retry.Default().
Retry *retry.Policy
// Backend and Region are stamped into every usage.Detail.
Backend string
Region string
// ResponseSchema: the server enforces response_format json_schema.
// A request that needs one is refused when false.
ResponseSchema bool
// ReasoningEcho: send prior reasoning back as reasoning_content.
ReasoningEcho bool
// CachedTokensUnreliable: record cached tokens as not reported even
// when the server sends them.
CachedTokensUnreliable bool
// ThinkTags: the server puts reasoning inline in content as a leading
// <think>…</think> block; split it into a reasoning part.
ThinkTags bool
// NoForcedToolChoice: the model rejects "required" and named tool
// choices; send "auto" instead and mark the response.
NoForcedToolChoice bool
// ExtraBody is merged into every request body: vendor fields the
// dialect does not model. It must not repeat a field the adapter sets.
ExtraBody map[string]any
}
Options configures a Client. A coremodels.Open caller never builds one by hand; it comes from a resolved profile.