openaichat

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 9, 2026 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Overview

Package openaichat is the openai-chat dialect: POST {base_url}/chat/completions, the wire shape Vertex AI's partner models, xAI, vLLM, SGLang, Ollama, llama.cpp, NVIDIA NIM and the managed long tail all speak (docs/design.md §7.1).

It is written on net/http and its own wire types rather than an SDK. These servers are OpenAI-shaped, not OpenAI: they add fields (reasoning_content), omit fields (usage details, tool-call ids) and differ on what they accept, and pointer-typed wire structs keep a count the server never sent distinct from a zero it did — the distinction usage.Detail exists for.

Retries happen in the HTTP client (package retry), so every attempt gets a fresh credential and a provider's Retry-After is honored.

Index

Constants

View Source
const (
	// RefusalKey marks a response whose text is the model's refusal.
	RefusalKey = "core_models.refusal"
	// FinishReasonKey holds a finish reason with no genai equivalent.
	FinishReasonKey = "core_models.finish_reason"
)

Metadata keys this dialect writes beside usage.MetadataKey.

View Source
const ToolChoiceKey = "core_models.tool_choice_downgraded"

ToolChoiceKey marks a response whose request asked for a forced tool call the model cannot take, and was sent with "auto" instead.

Variables

This section is empty.

Functions

This section is empty.

Types

type APIError

type APIError struct {
	StatusCode int
	Type       string
	Code       string
	Message    string
}

APIError is a non-2xx answer, or an error event inside a stream. StatusCode is 0 for an in-stream error.

func (*APIError) Error

func (e *APIError) Error() string

func (*APIError) HTTPStatus

func (e *APIError) HTTPStatus() int

HTTPStatus reports the status, for retry classifiers that sit above this package.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is a connection to one server. Safe for concurrent use.

func New

func New(opts Options) (*Client, error)

New returns a Client for opts.

func (*Client) Model

func (c *Client) Model(id string) llm.LLM

Model returns the model id on this server as an llm.LLM, with the Client's options.

func (*Client) ModelWith added in v0.2.0

func (c *Client) ModelWith(id string, mo ModelOptions) llm.LLM

ModelWith returns the model id with mo laid over the Client's options.

type ModelOptions added in v0.2.0

type ModelOptions struct {
	ResponseSchema     *bool
	ReasoningEcho      *bool
	NoForcedToolChoice *bool
}

ModelOptions are the per-model refinements of Options: one server serves many models, and what it honors can differ between them.

type Options

type Options struct {
	// BaseURL is the API root; requests go to BaseURL + "/chat/completions".
	BaseURL string
	// Credential authenticates every request as a bearer token. Nil
	// sends none.
	Credential *auth.Credential
	// HTTPClient supplies the base transport and timeout. Nil means a
	// client with http.DefaultTransport.
	HTTPClient *http.Client
	// Retry is the HTTP-layer retry policy. Nil means retry.Default().
	Retry *retry.Policy

	// Backend and Region are stamped into every usage.Detail.
	Backend string
	Region  string

	// ResponseSchema: the server enforces response_format json_schema.
	// A request that needs one is refused when false.
	ResponseSchema bool
	// ReasoningEcho: send prior reasoning back as reasoning_content.
	ReasoningEcho bool
	// CachedTokensUnreliable: record cached tokens as not reported even
	// when the server sends them.
	CachedTokensUnreliable bool
	// ThinkTags: the server puts reasoning inline in content as a leading
	// <think>…</think> block; split it into a reasoning part.
	ThinkTags bool
	// NoForcedToolChoice: the model rejects "required" and named tool
	// choices; send "auto" instead and mark the response.
	NoForcedToolChoice bool
}

Options configures a Client. A coremodels.Open caller never builds one by hand; it comes from a resolved profile.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL