openaicompat

package
v0.6.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 11, 2026 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package openaicompat is the adapter for any server that speaks the plain OpenAI chat-completions API: a user's own endpoint (vLLM, llama.cpp, Ollama, LM Studio, a hosted provider) whose real identity this code never knows. That API is the provider's native one here, so this is not a shim over a different provider.

It sends only the shape every compatible server shares: max_tokens rather than max_completion_tokens, reasoning_effort only when the call sets a thinking level, and no reasoning echo on assistant history. Responses are read with the OpenRouter-dialect parser (internal/chatwire), a superset of the plain shape: fields a plain server never sends are simply absent.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Adapter

type Adapter struct {
	// contains filtered or unexported fields
}

Adapter wraps an OpenAI-compatible chat-completions API as an llm.LLM.

func New

func New(cfg Config) (*Adapter, error)

New builds an Adapter from config.

func (*Adapter) Generate

func (a *Adapter) Generate(ctx context.Context, msgs []ac.Message, tools []ac.ToolSpec, opts ...ac.CallOption) (*ac.LLMResponse, error)

Generate runs a one-shot chat completion.

func (*Adapter) GenerateStream

func (a *Adapter) GenerateStream(ctx context.Context, msgs []ac.Message, tools []ac.ToolSpec, opts ...ac.CallOption) (<-chan ac.StreamEvent, error)

GenerateStream runs a streaming chat completion, emitting the agentcore StreamEvent sequence the loop expects.

func (*Adapter) ProviderName

func (a *Adapter) ProviderName() string

ProviderName implements agentcore.ProviderNamer.

func (*Adapter) SupportsTools

func (a *Adapter) SupportsTools() bool

SupportsTools reports true: tool calling is part of the shared shape. A server that lacks it answers with an error rather than this adapter guessing.

type Config

type Config struct {
	Model   string
	BaseURL string
	// APIKey is optional: local servers often take none, and the header is
	// left off when it is empty. agentcore.WithAPIKey overrides it per call,
	// so one adapter can carry many users' keys. Leave it empty on an adapter
	// that serves user endpoints: a call with no per-call key falls back to
	// it, which would send this key to that user's endpoint.
	APIKey string
	// Transport, if set, carries every request. A host calling a
	// user-supplied URL installs its SSRF-restricting dialer here.
	Transport http.RoundTripper
	// Meter, if set, receives token and latency Usage for every call.
	// Streaming usage arrives only from servers that send it unasked.
	Meter llm.Meter
	// Temperature and TopP are optional sampler params. nil leaves them off
	// the wire.
	Temperature *float64
	TopP        *float64
	// StreamIdleTimeout aborts a stream that has produced nothing for this
	// long. Zero uses two minutes; negative disables the watchdog.
	StreamIdleTimeout time.Duration
}

Config configures the adapter. Model and BaseURL are required; BaseURL includes the version path, e.g. "https://host/v1".

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL