Documentation
¶
Overview ¶
Package openaicompat is the adapter for any server that speaks the plain OpenAI chat-completions API: a user's own endpoint (vLLM, llama.cpp, Ollama, LM Studio, a hosted provider) whose real identity this code never knows. That API is the provider's native one here, so this is not a shim over a different provider.
It sends only the shape every compatible server shares: max_tokens rather than max_completion_tokens, reasoning_effort only when the call sets a thinking level, and no reasoning echo on assistant history. Responses are read with the OpenRouter-dialect parser (internal/chatwire), a superset of the plain shape: fields a plain server never sends are simply absent.
Index ¶
- type Adapter
- func (a *Adapter) Generate(ctx context.Context, msgs []ac.Message, tools []ac.ToolSpec, ...) (*ac.LLMResponse, error)
- func (a *Adapter) GenerateStream(ctx context.Context, msgs []ac.Message, tools []ac.ToolSpec, ...) (<-chan ac.StreamEvent, error)
- func (a *Adapter) ProviderName() string
- func (a *Adapter) SupportsTools() bool
- type Config
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Adapter ¶
type Adapter struct {
// contains filtered or unexported fields
}
Adapter wraps an OpenAI-compatible chat-completions API as an llm.LLM.
func (*Adapter) Generate ¶
func (a *Adapter) Generate(ctx context.Context, msgs []ac.Message, tools []ac.ToolSpec, opts ...ac.CallOption) (*ac.LLMResponse, error)
Generate runs a one-shot chat completion.
func (*Adapter) GenerateStream ¶
func (a *Adapter) GenerateStream(ctx context.Context, msgs []ac.Message, tools []ac.ToolSpec, opts ...ac.CallOption) (<-chan ac.StreamEvent, error)
GenerateStream runs a streaming chat completion, emitting the agentcore StreamEvent sequence the loop expects.
func (*Adapter) ProviderName ¶
ProviderName implements agentcore.ProviderNamer.
func (*Adapter) SupportsTools ¶
SupportsTools reports true: tool calling is part of the shared shape. A server that lacks it answers with an error rather than this adapter guessing.
type Config ¶
type Config struct {
Model string
BaseURL string
// APIKey is optional: local servers often take none, and the header is
// left off when it is empty. agentcore.WithAPIKey overrides it per call,
// so one adapter can carry many users' keys. Leave it empty on an adapter
// that serves user endpoints: a call with no per-call key falls back to
// it, which would send this key to that user's endpoint.
APIKey string
// Transport, if set, carries every request. A host calling a
// user-supplied URL installs its SSRF-restricting dialer here.
Transport http.RoundTripper
// Meter, if set, receives token and latency Usage for every call.
// Streaming usage arrives only from servers that send it unasked.
Meter llm.Meter
// Temperature and TopP are optional sampler params. nil leaves them off
// the wire.
Temperature *float64
TopP *float64
// StreamIdleTimeout aborts a stream that has produced nothing for this
// long. Zero uses two minutes; negative disables the watchdog.
StreamIdleTimeout time.Duration
}
Config configures the adapter. Model and BaseURL are required; BaseURL includes the version path, e.g. "https://host/v1".