litellm

package module
v1.9.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 2, 2026 License: Apache-2.0 Imports: 14 Imported by: 2

README

LiteLLM Go

中文 | English

LiteLLM is a small, explicit Go SDK for calling LLM providers through one typed core model. The root package owns the provider-agnostic API; concrete providers live in provider/<name> subpackages.

The SDK maps structure only. It does not infer what a model supports, does not validate vendor values locally, and does not rewrite your input: the vendor API decides, and its error is returned as a typed *litellm.Error.

Install

go get github.com/voocel/litellm

Requires Go 1.26 or newer.

Quick Start

package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/voocel/litellm"
	"github.com/voocel/litellm/provider/openai"
)

func main() {
	provider, err := openai.New(openai.Config{APIKey: os.Getenv("OPENAI_API_KEY")})
	if err != nil {
		log.Fatal(err)
	}
	client, err := litellm.New(provider)
	if err != nil {
		log.Fatal(err)
	}

	resp, err := client.Chat(context.Background(), litellm.Request{
		Model: "gpt-5.6",
		Messages: []litellm.Message{
			litellm.System("You are concise."),
			litellm.UserText("Explain Go interfaces in one sentence."),
		},
		MaxTokens: new(120),
	})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(resp.Text())
}

A provider is safe to share across clients. litellm.New accepts ClientOptions such as WithObservers and WithCaptureRawResponse.

Core Model

Messages and responses use ordered Block values: TextBlock, ImageBlock, ReasoningBlock, ToolUseBlock, ToolResultBlock and ToolReferenceBlock. Response.Blocks is the canonical content; Text(), Reasoning() and ToolCalls() are views.

msgs := []litellm.Message{
	litellm.User(litellm.Text("What is in this image?"), litellm.ImageURL("https://example.com/cat.png")),
}

For multi-turn tool workflows, append the previous response blocks as they are, and keep them whole when you store history:

msgs = append(msgs,
	litellm.Assistant(resp.Blocks...),
	litellm.ToolResultText("call_1", `{"ok":true}`),
)

Data a vendor needs back, such as a reasoning signature or an item id, travels in a block's State and is sent only to the provider that produced it, so history can move between providers (details). The Client checks the structure of messages and never rewrites history. Tool-call pairing and repair are conversation policy, owned by the layer that manages the session.

Messages encode as JSON with each block tagged by its type, state included, so json.Marshal stores history and json.Unmarshal reads it back as it was.

Raw provider response bodies are kept only with litellm.WithCaptureRawResponse(true).

Streaming

A stream is a sequence of blocks. BlockStart opens the block at Index, the position it takes in Response.Blocks; TextDelta, ReasoningDelta and ToolUseDelta grow it; BlockEnd closes it with the completed block, including metadata that arrives late, such as State. Blocks may interleave, and all end before DoneEvent. UsageEvent, WarningEvent and ProviderEvent (native events without a typed equivalent) carry the rest.

Handle aggregates a stream and passes each event to a callback; Collect only aggregates. On failure both return the partial response with the error.

stream, err := client.Stream(ctx, litellm.Request{
	Model:    "gpt-5.6",
	Messages: []litellm.Message{litellm.UserText("Tell me a short joke.")},
})
if err != nil {
	log.Fatal(err)
}
defer stream.Close()

resp, err := litellm.Handle(stream, func(event litellm.Event) error {
	switch e := event.(type) {
	case litellm.ReasoningDelta:
		fmt.Print(e.Text)
	case litellm.TextDelta:
		fmt.Print(e.Text)
	}
	return nil
})

Client.Stream aggregates as events are read, so Handle and Collect return the complete response even after some events were read with Next. A stream is consumed by one goroutine.

A deadline on ctx bounds the whole call, which for a model that thinks at length must be long. To also end a stream over a connection that hung, litellm.WithStreamIdleTimeout(d) fails it, with a temporary network error, once it waits d for data. Any data counts, vendor pings and gateway heartbeats included, so set d above the longest silence of a healthy stream, as a vendor may be silent while the model thinks.

Tools

tool, err := litellm.NewTool("get_weather", "Get weather for a city.", map[string]any{
	"type": "object",
	"properties": map[string]any{
		"city": map[string]any{"type": "string"},
	},
	"required": []string{"city"},
})
if err != nil {
	log.Fatal(err)
}
tool.Strict = new(true)

resp, err := client.Chat(ctx, litellm.Request{
	Model:      "gpt-5.6",
	Messages:   []litellm.Message{litellm.UserText("Weather in Paris?")},
	Tools:      []litellm.Tool{tool},
	ToolChoice: &litellm.ToolChoice{Mode: litellm.ToolChoiceAuto},
})

ToolChoice takes a Mode (Auto, None, Required) or a tool Name; nil leaves the vendor default.

ToolUseBlock.Arguments is the text the model wrote. It is meant to be a JSON object but may not be one, as when the reply was cut off at the output limit; the Client then adds a litellm.tool_arguments_invalid warning, and providers whose wire format needs an object reject the call in history with a validation error naming it.

A tool result holds text, images and tool references on every provider. Where a tool result carries text only, as in Chat Completions, the images follow the turn's tool messages in a user message.

Structured Output

format, err := litellm.NewResponseFormatJSONSchema("person", "", map[string]any{
	"type": "object",
	"properties": map[string]any{
		"name": map[string]any{"type": "string"},
	},
	"required": []string{"name"},
})
if err != nil {
	log.Fatal(err)
}
format.JSONSchema.Strict = new(true)

resp, err := client.Chat(ctx, litellm.Request{
	Model:          "gpt-5.6",
	Messages:       []litellm.Message{litellm.UserText("Generate a person.")},
	ResponseFormat: format,
})

Thinking

Thinking == nil sends no thinking fields and keeps the vendor default. Otherwise thinking is on, and Disabled turns it off; Effort and BudgetTokens are sent as given, and IncludeOutput asks for reasoning text where the vendor makes it optional.

resp, err := client.Chat(ctx, litellm.Request{
	Model:     "claude-sonnet-5",
	Messages:  []litellm.Message{litellm.UserText("Explain the tradeoffs.")},
	MaxTokens: new(2048),
	Thinking:  &litellm.Thinking{Effort: "low"},
})

Which values a model accepts is the vendor's decision. providers.md lists the exact wire mapping per provider.

Prompt Caching

Mark a cache breakpoint on a block; the prompt prefix up to and including it may be cached. TTL sets how long, passed as is, such as "1h"; empty is the vendor's default, five minutes on Anthropic and Bedrock. Breakpoints are hints: providers without a slot drop them, and those that cannot send a TTL use the default (providers.md).

litellm.User(litellm.TextBlock{Text: longDocument, Cache: &litellm.CacheControl{TTL: "1h"}})

Provider Options

Request.ProviderOptions carries native wire fields: each key is a top-level field of the vendor's request body, and its value is JSON. Keys are checked against the provider's list and rejected when unknown. When a key names a field the adapter also generates, an object is merged into it and an array is appended to it; any other collision is an error.

options, err := litellm.NewProviderOptions(map[string]any{
	openai.ProviderOptionPromptCacheKey: "session-42",
	openai.ProviderOptionServiceTier:    "flex",
})
if err != nil {
	log.Fatal(err)
}
resp, err := client.Chat(ctx, litellm.Request{
	Model:           "gpt-5.6",
	Messages:        []litellm.Message{litellm.UserText("Hello")},
	ProviderOptions: options,
})

Providers

Package API
provider/openai OpenAI Chat Completions (default) or Responses
provider/anthropic Anthropic Messages
provider/gemini Gemini generateContent
provider/bedrock Amazon Bedrock Converse (SigV4)
provider/deepseek, glm, grok, mimo, minimax, ollama, openrouter, qwen each vendor's Chat Completions dialect
provider/compat any other OpenAI-compatible endpoint (vLLM, LM Studio, gateways)
gateway a litellm gateway
anthropic.New(anthropic.Config{APIKey: os.Getenv("ANTHROPIC_API_KEY")})
gemini.New(gemini.Config{APIKey: os.Getenv("GEMINI_API_KEY")})
ollama.New(ollama.Config{})
compat.New(compat.Config{BaseURL: "http://localhost:8000/v1"})

bedrock.New(bedrock.Config{
	Region: "us-east-1",
	Credentials: bedrock.StaticCredentials(
		os.Getenv("AWS_ACCESS_KEY_ID"),
		os.Getenv("AWS_SECRET_ACCESS_KEY"),
		os.Getenv("AWS_SESSION_TOKEN"),
	),
})

Applications that choose a provider from configuration build it by name; provider.Config holds the shared settings:

import "github.com/voocel/litellm/provider"

provider, err := provider.New("anthropic", provider.Config{APIKey: os.Getenv("ANTHROPIC_API_KEY")})
names := provider.Names() // "anthropic", "bedrock", "compat", ...

openai follows the official protocol only. compat is for any other OpenAI-compatible server and passes provider options through unchecked, since it cannot know the server's fields.

Each provider's Config.Name, or provider.Config.Name, renames it: responses, errors and replay state carry the name. Give each endpoint of one protocol its own, such as an Anthropic-compatible proxy, so a reasoning signature is replayed only where it was issued.

client.Capabilities() reports the protocol facts of the adapter: whether MaxTokens is required, whether it sends Thinking.Effort and Thinking.Disabled, and the accepted option keys. It is static per provider, and ok is false for a provider that declares nothing, such as a gateway. Which models think is model data, in the catalog; whether a model honors a request is the vendor's call.

OpenAI Responses

Set openai.Config.API = openai.APIResponses to route Chat and Stream through the Responses API with the same request and response types. Native Responses fields are provider options; using an option of the other API is an error. Hosted tools and server-side compaction are not offered, since litellm does not model their output, nor is previous_response_id: history is sent whole.

provider, err := openai.New(openai.Config{APIKey: os.Getenv("OPENAI_API_KEY"), API: openai.APIResponses})

options, err := litellm.NewProviderOptions(map[string]any{
	openai.ProviderOptionStore:   false,
	openai.ProviderOptionInclude: []any{"reasoning.encrypted_content"},
})

Gateway

Package gateway runs calls on a gateway that holds the vendor keys, such as for an agent in a sandbox that must never see one. The client is an ordinary provider, which provider.New("gateway", …) also builds; the gateway serves gateway.Server, whose Route picks the Client for each call and may rewrite the request:

// gateway
srv := &gateway.Server{Route: func(r *http.Request, req *litellm.Request) (*litellm.Client, error) {
	if req.Model != "smart" {
		return nil, errors.New("unknown model")
	}
	req.Model = "claude-sonnet-4-5"
	return anthropicClient, nil
}}
http.Handle("/v1/llm", auth(srv))

// sandbox
p, _ := gateway.New(gateway.Config{BaseURL: "https://gw.example.com/v1/llm", APIKey: teamToken})

Calls keep everything on the way: the request whole but for the key, blocks with their state, and errors with their type, retry facts and upstream provider. The upstream rejecting the gateway's own vendor key is the exception: it reaches the caller as a provider error, "upstream key rejected", rather than an auth error, which would blame the caller's key. The Server neither authenticates nor meters: put the caller on the request context in your authentication, and meter with an Observer on the routed Clients, which see that context.

While the upstream is silent, the Server writes a heartbeat line every 15 seconds, which keeps proxies from cutting the connection and which the client skips. It refuses request bodies over gateway.MaxRequestBytes (64 MiB). The caller cannot see the vendor's capabilities, so Route sets MaxTokens where the vendor requires it.

Errors

HTTP failures and errors inside a stream are classified the same way; switch on ErrorTypeOf instead of matching messages:

switch litellm.ErrorTypeOf(err) {
case litellm.ErrorTypeContextOverflow:
	// compact history and resend
case litellm.ErrorTypeRateLimit, litellm.ErrorTypeOverloaded:
	time.Sleep(litellm.RetryAfter(err)) // 0 when the provider sent no Retry-After
case litellm.ErrorTypeContentFilter:
	// do not retry
case litellm.ErrorTypeCanceled:
	// the caller cancelled; errors.Is(err, context.Canceled) holds as well
}

IsTemporaryError reports a failure that a fresh request may avoid: rate limits, overload, network failures, a stream cut off before its end, and server faults, whether an HTTP 5xx or reported inside a stream. Messages render as provider: code: message. Context overflow and content filtering are detected from vendor codes and messages even when a proxy rewrites the status; they are never marked temporary.

Retry

Providers never retry. Error.Temporary describes a possibly transient failure, not permission to replay: the vendor may already have processed and billed the request. To opt in, wrap the HTTP client you pass to the provider:

import "github.com/voocel/litellm/retry"

provider, err := openai.New(openai.Config{
	APIKey:     os.Getenv("OPENAI_API_KEY"),
	HTTPClient: retry.NewHTTPClient(nil, retry.DefaultPolicy()),
})
  • Retried: complete 408, 429, 500, 502, 503, 504 and 529 responses, unless the body shows exhausted quota, an auth failure, content filtering or context overflow.
  • Not retried, returning the original response or error: network failures, interrupted streams, requests whose body cannot be resent, and, with RespectRetryAfter (on in DefaultPolicy), a Retry-After beyond MaxRetryAfter (60s by default).
  • Bedrock: the retried request reuses its SigV4 signature, which stays valid for five minutes.

Observers And OTel

An Observer starts a CallObserver for every Chat/Stream invocation, including local validation failures. CallInfo.Request is a snapshot of the caller's request, shared by all observers. OnEvent receives stream events (and Chat warnings) as they arrive; End runs once with the status (completed, failed, canceled, closed), duration, error and the final or partial response. Consume streams to termination or Close them.

type logCall struct{ info litellm.CallInfo }

func (c logCall) OnEvent(litellm.Event) {}
func (c logCall) End(r litellm.CallResult) {
	log.Printf("%s/%s: %s in %s, err=%v", c.info.Provider, c.info.Request.Model, r.Status, r.Duration, r.Err)
}

observer := litellm.ObserverFunc(func(ctx context.Context, info litellm.CallInfo) (context.Context, litellm.CallObserver) {
	return ctx, logCall{info}
})
client, err := litellm.New(provider, litellm.WithObservers(observer))

The optional github.com/voocel/litellm/otel module creates a GenAI semantic-convention span per call and propagates its context to the transport. Content capture is off by default; enable it explicitly, since messages may contain user data:

import litellmotel "github.com/voocel/litellm/otel"

observer := litellmotel.New(tracer, litellmotel.WithCaptureContent(true))

Usage And Model Catalog

Token counts are plain ints; a count the vendor does not report is zero. Input includes cache reads and writes; output includes reasoning; detail counts are subsets, such as CacheWrite1hTokens, the writes cached for an hour. Pricing.Cost prices the input not read from or written to the cache at the input rate, so a vendor that reports no cache counts is priced as uncached input. Hour-long writes are priced at CacheWrite1hCostPerToken, and are an error without it. So is usage without input tokens: every call has some, so the vendor reported none, and its cost is unknown rather than zero.

The catalog holds model facts (context window, output limit, reasoning support and prices) from LiteLLM's model list, and never loads remote data implicitly:

import (
	"fmt"

	"github.com/voocel/litellm/catalog"
)

var models catalog.Catalog
if err := models.LoadFromURL(ctx, catalog.DefaultURL); err != nil {
	return err
}

model, ok := models.Get("anthropic/claude-sonnet-4-5") // exact model-list key
if !ok {
	return fmt.Errorf("model not found in catalog")
}
if model.Pricing == nil {
	return fmt.Errorf("model pricing is unavailable")
}

cost, err := model.Pricing.Cost(resp.Usage)
if err != nil {
	return err
}

Names are the list's keys, whose vendor prefixes follow LiteLLM's provider names, such as xai/, zai/ and dashscope/, rather than provider.Names(). The catalog does not translate them, since one model may be listed under several sites at different prices; keep the catalog name of each model you price. provider.CatalogName(name, model) gives the name for a built-in provider, such as xai/grok-4 for grok.

Get matches the complete key exactly. If vendor/model is absent, it returns ok == false even when model exists.

ok == false means the model is absent; Pricing == nil means its price is unknown. A non-nil Pricing with zero rates means usage is free.

Model.Reasoning is *bool: nil means unknown, false means unsupported, and true means supported. Token limits of zero are unknown. Set and model-list loading validate names, limits and rates; a failed load leaves the catalog unchanged. Providers never consult the catalog to change requests.

Custom Providers

Implement the provider interface; CapabilityProvider is optional:

type Provider interface {
	Name() string
	Chat(context.Context, *litellm.Request) (*litellm.Response, error)
	Stream(context.Context, *litellm.Request) (litellm.Stream, error)
}

To test code that calls models, litellmtest.New returns a provider that streams scripted replies and records the requests it got.

License

Apache License

Documentation

Overview

Package litellm provides a small, explicit multi-provider LLM SDK core.

The root package owns the provider-agnostic domain model: Request, Response, Message, Block, Stream, Event, structured errors, warnings, and observers. Concrete providers live in the subpackages of package provider, which builds them by name; provider/compat connects any OpenAI-compatible endpoint. catalog, retry and gateway are optional utilities; gateway carries calls over HTTP to a server that holds the vendor keys.

Quick Start

Create a provider with its package-specific config, then bind a Client:

import (
    "context"
    "fmt"
    "os"

    "github.com/voocel/litellm"
    "github.com/voocel/litellm/provider/anthropic"
)

provider, err := anthropic.New(anthropic.Config{
    APIKey: os.Getenv("ANTHROPIC_API_KEY"),
})
if err != nil {
    panic(err)
}
client, err := litellm.New(provider)
if err != nil {
    panic(err)
}

maxTokens := 1024
resp, err := client.Chat(context.Background(), litellm.Request{
    Model:     "claude-sonnet-4-5",
    MaxTokens: &maxTokens,
    Messages:  []litellm.Message{litellm.UserText("Explain AI in one sentence.")},
})
if err != nil {
    panic(err)
}
fmt.Println(resp.Text())

Blocks

Message and Response content is represented as ordered Blocks. This preserves the order of text, reasoning, tool use, and tool results across multi-turn agent workflows. Data a vendor needs back, such as a reasoning signature, travels in a block's ProviderState and is sent only to the provider that produced it.

Streaming

A stream is a sequence of blocks. BlockStart opens the block at Index, the position it takes in Response.Blocks; deltas of the same kind grow it; BlockEnd closes it with the completed block. Blocks may interleave, and all end before DoneEvent. Handle aggregates the stream and passes each event to a callback; Collect only aggregates. Stream is for single-goroutine consumption.

stream, err := client.Stream(ctx, req)
if err != nil {
    panic(err)
}
defer stream.Close()

resp, err := litellm.Handle(stream, func(event litellm.Event) error {
    if e, ok := event.(litellm.TextDelta); ok {
        fmt.Print(e.Text)
    }
    return nil
})

Design

The SDK is intentionally not a router, agent runtime, account system, or request scheduler, and its gateway leaves authentication and metering to the application. It binds one Client to one Provider and exposes explicit configuration and structural validation. It maps structure only: it does not infer model features, validate vendor values locally, or rewrite user input; the vendor API is the authority. ProviderOptions carry native wire fields. Usage counts are plain ints; a count the vendor does not report is zero.

Errors

Errors are *Error values. Switch on ErrorTypeOf to act on a failure, and use IsTemporaryError to tell whether a fresh request may succeed: a stream cut off before its end is a temporary network error, and a call the caller's context cancelled is ErrorTypeCanceled.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IsTemporaryError added in v1.9.0

func IsTemporaryError(err error) bool

IsTemporaryError reports whether the failure may resolve over time. It does not establish that repeating the operation is safe, even before any output.

func RetryAfter added in v1.9.0

func RetryAfter(err error) time.Duration

RetryAfter returns the server-suggested delay carried by err, or zero.

func WrapError added in v1.5.0

func WrapError(provider string, fallback ErrorType, err error) error

WrapError attributes err to provider. Existing *Error values keep their type and gain the provider when unset; context cancellation and deadlines become canceled and timeout errors; anything else becomes a fallback-typed error.

Types

type Annotation added in v1.8.0

type Annotation struct {
	Type  string          `json:"type"`
	Text  string          `json:"text,omitempty"`
	URL   string          `json:"url,omitempty"`
	Extra json.RawMessage `json:"extra,omitempty"`
}

Annotation is a citation attached to text. Extra keeps the vendor entry verbatim.

type Block added in v1.8.0

type Block interface {
	// contains filtered or unexported methods
}

Block is one piece of message content. The implementations are TextBlock, ImageBlock, ReasoningBlock, ToolUseBlock, ToolResultBlock and ToolReferenceBlock.

func UnmarshalBlock added in v1.9.3

func UnmarshalBlock(raw json.RawMessage) (Block, error)

UnmarshalBlock decodes a block encoded as JSON by its "type".

func UnmarshalBlocks added in v1.9.3

func UnmarshalBlocks(raws []json.RawMessage) ([]Block, error)

UnmarshalBlocks decodes blocks encoded as JSON, each by its "type".

type BlockEnd added in v1.9.0

type BlockEnd struct {
	Index int
	Block Block
}

BlockEnd closes a block. Providers set Block only to deliver metadata that arrives late (State, annotations, logprobs, a tool's id and name): its non-empty metadata fields replace the started ones, while its text and arguments are ignored. Streams read through Handle, Collect or a Client deliver the completed block instead.

type BlockStart added in v1.9.0

type BlockStart struct {
	Index int
	Block Block
}

BlockStart opens a TextBlock, ReasoningBlock or ToolUseBlock. Block carries the metadata known up front, such as a tool's ID and Name, and may carry initial text.

type CacheControl added in v1.5.0

type CacheControl struct {
	TTL string `json:"ttl,omitempty"`
}

CacheControl marks a cache breakpoint: the prompt prefix up to and including this block may be cached. TTL is how long, passed as is, such as "1h"; empty is the vendor's default, five minutes on Anthropic and Bedrock. Like the breakpoint, a TTL is a hint: adapters that cannot send it use the default.

type CallInfo added in v1.9.0

type CallInfo struct {
	Provider  string
	Streaming bool
	StartedAt time.Time
	Request   *Request
}

CallInfo describes the caller's request before validation. Request is a snapshot shared by all observers: read it, never modify it.

type CallObserver added in v1.9.0

type CallObserver interface {
	OnEvent(Event)
	End(CallResult)
}

CallObserver belongs to one invocation. Events and the Response are snapshots shared by all observers and isolated from the caller: read, never modify. OnEvent receives stream events as they arrive, and Chat warnings as WarningEvent. End runs exactly once; observers end in reverse registration order so nested observation scopes unwind correctly. Streams must be consumed to termination or explicitly closed; cancellation alone does not run callbacks in a background goroutine.

type CallResult added in v1.9.0

type CallResult struct {
	Status   CallStatus
	Response *Response
	Err      error
	Duration time.Duration
}

CallResult describes SDK execution, not errors in application consumer callbacks. Response may be partial on failure or early Close. A later resource cleanup error does not revise an already completed invocation; Close returns that error.

type CallStatus added in v1.9.0

type CallStatus string

CallStatus is how an invocation ended. CallClosed means the caller closed a stream before it finished.

const (
	CallCompleted CallStatus = "completed"
	CallFailed    CallStatus = "failed"
	CallCanceled  CallStatus = "canceled"
	CallClosed    CallStatus = "closed"
)

type Capabilities added in v1.8.1

type Capabilities struct {
	// MaxTokensRequired reports that the vendor rejects requests without
	// Request.MaxTokens.
	MaxTokensRequired bool
	// ThinkingEffort and DisableThinking report that the adapter sends
	// Thinking.Effort and Thinking.Disabled; it rejects them otherwise, before
	// the request is sent. Which efforts a model takes is the vendor's call.
	ThinkingEffort  bool
	DisableThinking bool
	// ProviderOptions lists the accepted Request.ProviderOptions keys, sorted.
	// A provider that passes every key through, such as compat, lists none.
	ProviderOptions []string
}

Capabilities states protocol facts of a provider adapter. It is static per provider: whether a model honors a request is the vendor's call, and the vendor's error is the source of truth.

type CapabilityProvider added in v1.8.1

type CapabilityProvider interface {
	Capabilities() Capabilities
}

CapabilityProvider is implemented by providers that declare Capabilities.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client runs requests through a Provider. It copies and validates each request, attributes errors to the provider, and notifies Observers. A Client is safe for concurrent use.

func New

func New(provider Provider, opts ...ClientOption) (*Client, error)

New returns a Client for provider.

func (*Client) Capabilities added in v1.8.1

func (c *Client) Capabilities() (caps Capabilities, ok bool)

Capabilities reports what the provider adapter can express on the wire; ok is false when the provider does not implement CapabilityProvider, so the facts are unknown.

func (*Client) Chat added in v1.5.0

func (c *Client) Chat(ctx context.Context, req Request) (*Response, error)

Chat sends req and returns the complete response. On error the response may still be returned when the provider produced one.

func (*Client) ProviderName added in v1.5.7

func (c *Client) ProviderName() string

ProviderName returns the provider's Name.

func (*Client) Stream

func (c *Client) Stream(ctx context.Context, req Request) (Stream, error)

Stream sends req and returns its event stream, which the caller must Close. Consume it with Next, Handle or Collect.

type ClientOption

type ClientOption func(*Client) error

ClientOption configures a Client.

func WithCaptureRawResponse added in v1.8.0

func WithCaptureRawResponse(enabled bool) ClientOption

WithCaptureRawResponse keeps the raw vendor body in Response.Raw.

func WithObservers added in v1.9.0

func WithObservers(observers ...Observer) ClientOption

WithObservers adds observers, skipping nil ones.

func WithStreamIdleTimeout added in v1.8.0

func WithStreamIdleTimeout(timeout time.Duration) ClientOption

WithStreamIdleTimeout fails a stream, with a temporary network error, once it waits timeout for data from its connection, as when the connection hung. Any data counts, vendor pings and gateway heartbeats included, so a healthy stream fails only when silent for longer, as a vendor may be while the model thinks. Zero disables the check.

type DoneEvent added in v1.8.0

type DoneEvent struct {
	FinishReason    FinishReason
	FinishReasonRaw string
	Provider        string
	Model           string
}

DoneEvent ends a successful stream.

type Error added in v1.9.0

type Error struct {
	Type       ErrorType
	Code       string
	Message    string
	Provider   string
	StatusCode int
	// Temporary describes a potentially transient failure, not permission to replay
	// a request. The provider may already have processed or billed the operation.
	Temporary bool
	// RetryAfter is the server-suggested delay; zero means unspecified.
	RetryAfter time.Duration
	Cause      error
}

Error is the error type returned by the client and all providers.

func NewError added in v1.5.0

func NewError(provider string, errorType ErrorType, message string, cause error) *Error

NewError builds an error of the given type. provider may be empty and cause nil; Temporary follows the type.

func NewNetworkError added in v1.5.0

func NewNetworkError(provider, message string, cause error) *Error

NewNetworkError builds a transport error. Context cancellation is ErrorTypeCanceled and a deadline a timeout, neither temporary; other causes are temporary.

func (*Error) Error added in v1.9.0

func (e *Error) Error() string

Error renders "provider: code: message (HTTP status)". Messages carry no provider prefix. The status tells a caller what a vague vendor message, such as "Provider returned error", does not.

func (*Error) Unwrap added in v1.9.0

func (e *Error) Unwrap() error

Unwrap returns Cause.

type ErrorType added in v1.5.0

type ErrorType string

ErrorType classifies an Error.

const (
	ErrorTypeAuth            ErrorType = "auth"
	ErrorTypeRateLimit       ErrorType = "rate_limit"
	ErrorTypeNetwork         ErrorType = "network"
	ErrorTypeValidation      ErrorType = "validation"
	ErrorTypeProvider        ErrorType = "provider"
	ErrorTypeTimeout         ErrorType = "timeout"
	ErrorTypeQuota           ErrorType = "quota"
	ErrorTypeModel           ErrorType = "model"
	ErrorTypeInternal        ErrorType = "internal"
	ErrorTypeContextOverflow ErrorType = "context_overflow"
	ErrorTypeOverloaded      ErrorType = "overloaded"
	ErrorTypeContentFilter   ErrorType = "content_filter"
	// ErrorTypeCanceled is a call the caller's context cancelled.
	ErrorTypeCanceled ErrorType = "canceled"
)

func ErrorTypeOf added in v1.9.3

func ErrorTypeOf(err error) ErrorType

ErrorTypeOf returns the type of the *Error err wraps, or "" when it wraps none.

type Event added in v1.8.0

type Event interface {
	// contains filtered or unexported methods
}

Event is a stream event. See the types below for the block lifecycle.

type FinishReason added in v1.8.0

type FinishReason string

FinishReason is a normalized stop reason; FinishReasonRaw keeps the vendor value. Empty means the vendor reported none.

const (
	FinishReasonStop     FinishReason = "stop"
	FinishReasonLength   FinishReason = "length"
	FinishReasonToolCall FinishReason = "tool_calls"
	FinishReasonError    FinishReason = "error"
	// FinishReasonSafety also covers explicit refusals; the refusal text, if
	// any, is an ordinary TextBlock.
	FinishReasonSafety FinishReason = "safety"
	// FinishReasonOther is a reported reason without a normalized equivalent.
	FinishReasonOther FinishReason = "other"
)

type HTTPClient added in v1.9.0

type HTTPClient interface {
	Do(*http.Request) (*http.Response, error)
}

HTTPClient sends provider requests; *http.Client satisfies it.

type ImageBlock added in v1.8.0

type ImageBlock struct {
	URL     string        `json:"url,omitempty"`
	Data    []byte        `json:"data,omitempty"`
	MIME    string        `json:"mime,omitempty"`
	FileURI string        `json:"file_uri,omitempty"`
	Detail  string        `json:"detail,omitempty"`
	Cache   *CacheControl `json:"cache,omitempty"`
}

ImageBlock is an image. Set one source: URL, Data with MIME, or FileURI, a vendor file reference. Detail is sent where the wire format has it.

func ImageURL added in v1.8.0

func ImageURL(url string) ImageBlock

ImageURL returns an ImageBlock for a URL, which may be a data URL.

func (ImageBlock) MarshalJSON added in v1.9.3

func (b ImageBlock) MarshalJSON() ([]byte, error)

type JSONSchema added in v1.2.1

type JSONSchema struct {
	Name        string `json:"name"`
	Description string `json:"description,omitempty"`
	Schema      Schema `json:"schema,omitempty"`
	Strict      *bool  `json:"strict,omitempty"`
}

JSONSchema is a named schema for structured output. Strict is as in Tool.

type Message

type Message struct {
	Role   Role    `json:"role"`
	Blocks []Block `json:"blocks"`
}

Message is one conversation turn.

func Assistant added in v1.8.0

func Assistant(blocks ...Block) Message

Assistant returns an assistant message, such as a prior reply to replay.

func AssistantText added in v1.8.0

func AssistantText(text string) Message

AssistantText returns an assistant message with one text block.

func System added in v1.8.0

func System(text string) Message

System returns a system message.

func ToolResult added in v1.8.0

func ToolResult(toolUseID string, blocks ...Block) Message

ToolResult returns a tool message answering the tool call toolUseID.

func ToolResultText added in v1.8.0

func ToolResultText(toolUseID, text string) Message

ToolResultText returns a tool message with a text result.

func User added in v1.8.0

func User(blocks ...Block) Message

User returns a user message.

func UserText added in v1.8.0

func UserText(text string) Message

UserText returns a user message with one text block.

func (*Message) UnmarshalJSON added in v1.9.3

func (m *Message) UnmarshalJSON(data []byte) error

type Observer added in v1.9.0

type Observer interface {
	Start(context.Context, CallInfo) (context.Context, CallObserver)
}

Observer starts one observation per Chat/Stream invocation, including local validation failures. Its returned non-nil context reaches subsequent observers and the provider; derive it from the input to preserve cancellation and values. Returning a nil CallObserver opts out of observing this call. Callbacks are synchronous; the SDK does not recover observer panics.

type ObserverFunc added in v1.9.0

type ObserverFunc func(context.Context, CallInfo) (context.Context, CallObserver)

ObserverFunc adapts a function to Observer.

func (ObserverFunc) Start added in v1.9.0

Start calls f.

type Provider

type Provider interface {
	Name() string
	Chat(context.Context, *Request) (*Response, error)
	Stream(context.Context, *Request) (Stream, error)
}

Provider adapts one vendor protocol. Chat and Stream receive a validated copy of the request that the provider owns. Implementations may also implement CapabilityProvider.

type ProviderEvent added in v1.8.0

type ProviderEvent struct {
	Name string
	Raw  json.RawMessage
}

ProviderEvent exposes a provider-native event that has no typed equivalent. It does not affect the aggregated Response.

type ProviderOptions added in v1.8.0

type ProviderOptions map[string]json.RawMessage

ProviderOptions contains JSON values owned by the request. Use NewProviderOptions or Set to encode Go values; the client copies each value before observation or execution.

func NewProviderOptions added in v1.9.0

func NewProviderOptions(values map[string]any) (ProviderOptions, error)

NewProviderOptions serializes values immediately, so subsequent mutations of the supplied Go objects cannot change the request. Encoding errors are returned.

func (ProviderOptions) Decode added in v1.9.0

func (o ProviderOptions) Decode() (map[string]any, error)

Decode gives a provider an independent JSON tree. Numbers remain json.Number to preserve integer precision until the provider validates its wire type.

func (ProviderOptions) Set added in v1.9.0

func (o ProviderOptions) Set(key string, value any) error

Set encodes a value into an initialized options map.

type ProviderState added in v1.9.0

type ProviderState struct {
	// Provider is the Name of the provider that produced the block.
	Provider string `json:"provider"`
	// Model is the requested model.
	Model string `json:"model,omitempty"`
	// Data holds the vendor's native fields as JSON.
	Data json.RawMessage `json:"data,omitempty"`
}

ProviderState is data a provider attaches to a block it produced so the block can be sent back to it: a reasoning signature, encrypted reasoning or an item id. Keep it with the block when storing history; only the provider named by Provider reads it.

Replay rule: portable content (text, reasoning text, tool calls) maps to wherever the target wire format can carry it, while State is sent only to the provider that produced it. A block the wire cannot carry without its State, such as Anthropic thinking, is dropped; where the vendor documents a placeholder for foreign content, the provider supplies it. Blocks built by the caller (nil State) count as foreign.

type ReasoningBlock added in v1.8.0

type ReasoningBlock struct {
	Text    string         `json:"text"`
	Summary bool           `json:"summary,omitempty"`
	State   *ProviderState `json:"state,omitempty"`
}

ReasoningBlock is model reasoning. Summary marks Text as a summary rather than the full reasoning. Text is empty when the vendor returns the reasoning encrypted or redacted; State then carries it.

func (ReasoningBlock) MarshalJSON added in v1.9.3

func (b ReasoningBlock) MarshalJSON() ([]byte, error)

type ReasoningDelta added in v1.8.0

type ReasoningDelta struct {
	Index int
	Text  string
}

ReasoningDelta appends text to a ReasoningBlock.

type Request

type Request struct {
	Model    string    `json:"model"`
	Messages []Message `json:"messages"`

	MaxTokens   *int     `json:"max_tokens,omitempty"`
	Temperature *float64 `json:"temperature,omitempty"`
	TopP        *float64 `json:"top_p,omitempty"`
	Stop        []string `json:"stop,omitempty"`

	Tools      []Tool      `json:"tools,omitempty"`
	ToolChoice *ToolChoice `json:"tool_choice,omitempty"`

	ResponseFormat *ResponseFormat `json:"response_format,omitempty"`
	Thinking       *Thinking       `json:"thinking,omitempty"`

	ProviderOptions ProviderOptions `json:"provider_options,omitempty"`
}

Request is a provider-neutral chat request. Nil pointers and empty fields are omitted from the wire, leaving the vendor default.

type Response

type Response struct {
	Blocks []Block
	Usage  Usage

	Model    string
	Provider string

	FinishReason    FinishReason
	FinishReasonRaw string
	Warnings        []Warning
	Raw             json.RawMessage
}

Response is a complete model reply. Providers set Raw to the vendor body of a non-streaming reply; the Client keeps it only with WithCaptureRawResponse.

func Collect added in v1.8.0

func Collect(stream Stream) (*Response, error)

Collect consumes the stream and returns the aggregated Response. On failure it returns the partial response together with the error. EOF before DoneEvent is io.ErrUnexpectedEOF; partial responses must not be treated as completed output.

func Handle added in v1.8.0

func Handle(stream Stream, fn func(Event) error) (*Response, error)

Handle consumes the stream, invoking fn for each event as it arrives, and returns the aggregated Response. Events are aggregated before fn runs, so a BlockEnd passed to fn carries the completed block. A nil fn behaves exactly like Collect. On any failure Handle returns the partial response and the error. The caller still owns Close.

func (*Response) Reasoning

func (r *Response) Reasoning() string

Reasoning concatenates the reasoning block texts.

func (*Response) Text added in v1.8.0

func (r *Response) Text() string

Text concatenates the text blocks.

func (*Response) ToolCalls

func (r *Response) ToolCalls() []ToolUseBlock

ToolCalls returns the tool use blocks in order.

type ResponseFormat added in v1.2.1

type ResponseFormat struct {
	Type       ResponseFormatType `json:"type"`
	JSONSchema *JSONSchema        `json:"json_schema,omitempty"`
}

ResponseFormat constrains the output format. JSONSchema is used with ResponseFormatJSONSchema. Providers without native schema support may use a prompt instead and report a Warning; this does not enforce schema adherence, including when Strict is set. See providers.md for the mapping.

func NewResponseFormatJSONObject added in v1.2.1

func NewResponseFormatJSONObject() *ResponseFormat

NewResponseFormatJSONObject requests a JSON object.

func NewResponseFormatJSONSchema added in v1.2.1

func NewResponseFormatJSONSchema(name, description string, schema any) (*ResponseFormat, error)

NewResponseFormatJSONSchema requests output matching schema, converted with SchemaFrom.

func NewResponseFormatText added in v1.2.1

func NewResponseFormatText() *ResponseFormat

NewResponseFormatText requests plain text output.

type ResponseFormatType added in v1.8.0

type ResponseFormatType string

ResponseFormatType selects the output format.

const (
	ResponseFormatText       ResponseFormatType = "text"
	ResponseFormatJSONObject ResponseFormatType = "json_object"
	ResponseFormatJSONSchema ResponseFormatType = "json_schema"
)

type Role added in v1.8.0

type Role string

Role identifies the author of a Message.

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

type Schema added in v1.8.0

type Schema json.RawMessage

Schema is a JSON Schema document.

func SchemaFrom added in v1.8.0

func SchemaFrom(v any) (Schema, error)

SchemaFrom returns v as a Schema. JSON text (Schema, json.RawMessage, []byte or string) is validated and copied; any other value is marshaled. A nil v returns a nil Schema.

func (Schema) MarshalJSON added in v1.9.3

func (s Schema) MarshalJSON() ([]byte, error)

MarshalJSON writes the schema as the JSON document it is.

func (*Schema) UnmarshalJSON added in v1.9.3

func (s *Schema) UnmarshalJSON(data []byte) error

UnmarshalJSON keeps a copy of the JSON document.

type Stream added in v1.8.0

type Stream interface {
	Next() (Event, error)
	Close() error
}

Stream is consumed by one goroutine. A successful stream emits exactly one DoneEvent. Failures are returned by Next, never encoded as events. Callers must Close the stream, including after completion or failure.

type TextBlock added in v1.8.0

type TextBlock struct {
	Text        string          `json:"text"`
	Annotations []Annotation    `json:"annotations,omitempty"`
	Logprobs    json.RawMessage `json:"logprobs,omitempty"`
	State       *ProviderState  `json:"state,omitempty"`
	Cache       *CacheControl   `json:"cache,omitempty"`
}

TextBlock is plain text. Annotations and Logprobs are response metadata.

func Text added in v1.8.0

func Text(text string) TextBlock

Text returns a TextBlock.

func (TextBlock) MarshalJSON added in v1.9.3

func (b TextBlock) MarshalJSON() ([]byte, error)

type TextDelta added in v1.9.0

type TextDelta struct {
	Index int
	Text  string
}

TextDelta appends text to a TextBlock.

type Thinking added in v1.8.0

type Thinking struct {
	Disabled      bool   `json:"disabled,omitempty"`
	Effort        string `json:"effort,omitempty"`
	BudgetTokens  *int   `json:"budget_tokens,omitempty"`
	IncludeOutput bool   `json:"include_output,omitempty"`
}

Thinking configures model reasoning; a nil *Thinking leaves it to the vendor default. Disabled turns reasoning off and allows no other field. Effort and BudgetTokens are sent as given; which values a model accepts is the vendor's decision. IncludeOutput is a hint to return reasoning where the vendor makes it optional; providers without such a switch ignore it.

type Tool

type Tool struct {
	Name        string `json:"name"`
	Description string `json:"description,omitempty"`
	Parameters  Schema `json:"parameters,omitempty"`
	Strict      *bool  `json:"strict,omitempty"`
}

Tool declares a function the model may call. Strict, when set, asks the vendor to enforce, or not, that calls fit Parameters; nil leaves its default.

func NewTool added in v1.5.4

func NewTool(name, description string, parameters any) (Tool, error)

NewTool builds a Tool, converting parameters with SchemaFrom.

type ToolChoice added in v1.8.0

type ToolChoice struct {
	Mode ToolChoiceMode `json:"mode,omitempty"`
	Name string         `json:"name,omitempty"`
}

ToolChoice selects a policy, or a named tool when Name is set. A nil choice leaves selection to the provider. Mode and Name are mutually exclusive.

type ToolChoiceMode added in v1.9.0

type ToolChoiceMode string

ToolChoiceMode is a tool selection policy.

const (
	ToolChoiceAuto     ToolChoiceMode = "auto"
	ToolChoiceNone     ToolChoiceMode = "none"
	ToolChoiceRequired ToolChoiceMode = "required"
)

type ToolReferenceBlock added in v1.8.0

type ToolReferenceBlock struct {
	ToolName string        `json:"tool_name"`
	Cache    *CacheControl `json:"cache,omitempty"`
}

ToolReferenceBlock names a tool inside tool result content, as returned by a tool search tool.

func (ToolReferenceBlock) MarshalJSON added in v1.9.3

func (b ToolReferenceBlock) MarshalJSON() ([]byte, error)

type ToolResultBlock added in v1.8.0

type ToolResultBlock struct {
	ToolUseID string        `json:"tool_use_id"`
	Content   []Block       `json:"content,omitempty"`
	IsError   bool          `json:"is_error,omitempty"`
	Cache     *CacheControl `json:"cache,omitempty"`
}

ToolResultBlock answers the ToolUseBlock with ID ToolUseID. Content holds TextBlock, ImageBlock or ToolReferenceBlock values.

func (ToolResultBlock) MarshalJSON added in v1.9.3

func (b ToolResultBlock) MarshalJSON() ([]byte, error)

func (*ToolResultBlock) UnmarshalJSON added in v1.9.3

func (b *ToolResultBlock) UnmarshalJSON(data []byte) error

type ToolUseBlock added in v1.8.0

type ToolUseBlock struct {
	ID        string         `json:"id"`
	Name      string         `json:"name"`
	Arguments string         `json:"arguments,omitempty"`
	State     *ProviderState `json:"state,omitempty"`
	Cache     *CacheControl  `json:"cache,omitempty"`
}

ToolUseBlock is a tool call from the assistant. Arguments is the text the model produced, which is meant to be a JSON object but may not be one, as when the response was cut off at the output limit.

func (ToolUseBlock) MarshalJSON added in v1.9.3

func (b ToolUseBlock) MarshalJSON() ([]byte, error)

type ToolUseDelta added in v1.8.0

type ToolUseDelta struct {
	Index     int
	Arguments string
}

ToolUseDelta appends a fragment of a ToolUseBlock's JSON arguments.

type Usage

type Usage struct {
	InputTokens        int `json:"input_tokens,omitempty"`
	OutputTokens       int `json:"output_tokens,omitempty"`
	ReasoningTokens    int `json:"reasoning_tokens,omitempty"`
	CacheReadTokens    int `json:"cache_read_tokens,omitempty"`
	CacheWriteTokens   int `json:"cache_write_tokens,omitempty"`
	CacheWrite1hTokens int `json:"cache_write_1h_tokens,omitempty"`
}

Usage holds the token counts a response reported; a count the vendor did not report is zero. InputTokens includes cache reads and writes; OutputTokens includes reasoning; CacheWriteTokens includes the writes cached for an hour, CacheWrite1hTokens.

type UsageEvent added in v1.8.0

type UsageEvent struct {
	Usage Usage
}

UsageEvent carries cumulative usage; a later event replaces an earlier one.

type Warning added in v1.8.0

type Warning struct {
	Code     string `json:"code"`
	Provider string `json:"provider,omitempty"`
	Message  string `json:"message"`
}

Warning reports a non-fatal issue, such as dropped content or a generated ID. Code has the form "<source>.<snake_case>", such as "litellm.tool_arguments_invalid".

type WarningEvent added in v1.8.0

type WarningEvent struct {
	Warning Warning
}

WarningEvent carries a Warning; it is also collected into Response.Warnings.

Directories

Path Synopsis
Package catalog holds model facts (context window, output limit, reasoning support and per-token prices) loaded from LiteLLM's model list or set explicitly.
Package catalog holds model facts (context window, output limit, reasoning support and per-token prices) loaded from LiteLLM's model list or set explicitly.
examples
anthropic command
bedrock command
deepseek command
gemini command
glm command
grok command
minimax command
ollama command
openai command
openrouter command
qwen command
Package gateway carries model calls over HTTP, so that an application can reach models through a gateway that holds the vendor keys and meters the usage: an agent in a sandbox, say, that must never see a key.
Package gateway carries model calls over HTTP, so that an application can reach models through a gateway that holds the vendor keys and meters the usage: an agent in a sandbox, say, that must never see a key.
internal
claude
Package claude holds the Claude request mapping shared by the Anthropic and Bedrock adapters.
Package claude holds the Claude request mapping shared by the Anthropic and Bedrock adapters.
idle
Package idle fails a response body whose reads wait too long for data, so that a stream over a connection that hung ends instead of waiting forever.
Package idle fails a response body whose reads wait too long for data, so that a stream over a connection that hung ends instead of waiting forever.
wire
Package wire holds what provider adapters share to speak vendor wire protocols: HTTP transport and Server-Sent Events, error and stop-reason classification, stream block indexing, ProviderOptions merging, replay state, and token and data URL helpers.
Package wire holds what provider adapters share to speak vendor wire protocols: HTTP transport and Server-Sent Events, error and stop-reason classification, stream block indexing, ProviderOptions merging, replay state, and token and data URL helpers.
Package litellmtest provides a scripted Provider for testing code that calls models, without a network or a vendor.
Package litellmtest provides a scripted Provider for testing code that calls models, without a network or a vendor.
otel module
Package provider builds the built-in providers by name, for applications that choose a provider from configuration.
Package provider builds the built-in providers by name, for applications that choose a provider from configuration.
anthropic
Package anthropic connects to the Anthropic Messages API.
Package anthropic connects to the Anthropic Messages API.
bedrock
Package bedrock connects to the Amazon Bedrock Converse API.
Package bedrock connects to the Amazon Bedrock Converse API.
compat
Package compat connects to any OpenAI-compatible Chat Completions endpoint, such as vLLM, LM Studio or a gateway, and holds the Config shared by the vendor wrappers (deepseek, glm, grok, mimo, minimax, ollama, openrouter, qwen).
Package compat connects to any OpenAI-compatible Chat Completions endpoint, such as vLLM, LM Studio or a gateway, and holds the Config shared by the vendor wrappers (deepseek, glm, grok, mimo, minimax, ollama, openrouter, qwen).
deepseek
Package deepseek connects to the DeepSeek API.
Package deepseek connects to the DeepSeek API.
gemini
Package gemini connects to the Gemini API.
Package gemini connects to the Gemini API.
glm
Package glm connects to the Zhipu GLM API.
Package glm connects to the Zhipu GLM API.
grok
Package grok connects to the xAI Grok API.
Package grok connects to the xAI Grok API.
internal/openaicompat
Package openaicompat implements the Chat Completions protocol shared by OpenAI and the OpenAI-compatible providers.
Package openaicompat implements the Chat Completions protocol shared by OpenAI and the OpenAI-compatible providers.
internal/openaicompat/compattest
Package compattest runs openaicompat providers against canned HTTP responses in tests.
Package compattest runs openaicompat providers against canned HTTP responses in tests.
mimo
Package mimo connects to the Xiaomi MiMo API.
Package mimo connects to the Xiaomi MiMo API.
minimax
Package minimax connects to the MiniMax API.
Package minimax connects to the MiniMax API.
ollama
Package ollama connects to Ollama's OpenAI-compatible endpoint.
Package ollama connects to Ollama's OpenAI-compatible endpoint.
openai
Package openai connects to the OpenAI API, or any endpoint that implements it exactly (such as Azure OpenAI), through Chat Completions or Responses.
Package openai connects to the OpenAI API, or any endpoint that implements it exactly (such as Azure OpenAI), through Chat Completions or Responses.
openrouter
Package openrouter connects to the OpenRouter API.
Package openrouter connects to the OpenRouter API.
qwen
Package qwen connects to the Alibaba Cloud DashScope compatible-mode API.
Package qwen connects to the Alibaba Cloud DashScope compatible-mode API.
Package retry provides opt-in HTTP retries for providers.
Package retry provides opt-in HTTP retries for providers.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL