Documentation
¶
Overview ¶
Package litellm provides a small, explicit multi-provider LLM SDK core.
The root package owns the provider-agnostic domain model: Request, Response, Message, Block, Stream, Event, structured errors, warnings, and observers. Concrete providers live in provider subpackages; compat connects any OpenAI-compatible endpoint. pricing and retry are optional utilities.
Quick Start ¶
Create a provider with its package-specific config, then bind a Client:
import (
"context"
"fmt"
"os"
"github.com/voocel/litellm"
"github.com/voocel/litellm/provider/anthropic"
)
provider, err := anthropic.New(anthropic.Config{
APIKey: os.Getenv("ANTHROPIC_API_KEY"),
})
if err != nil {
panic(err)
}
client, err := litellm.New(provider)
if err != nil {
panic(err)
}
maxTokens := 1024
resp, err := client.Chat(context.Background(), litellm.Request{
Model: "claude-sonnet-4-5",
MaxTokens: &maxTokens,
Messages: []litellm.Message{litellm.UserText("Explain AI in one sentence.")},
})
if err != nil {
panic(err)
}
fmt.Println(resp.Text())
Blocks ¶
Message and Response content is represented as ordered Blocks. This preserves the order of text, reasoning, tool use, and tool results across multi-turn agent workflows. Data a vendor needs back, such as a reasoning signature, travels in a block's ProviderState and is sent only to the provider that produced it.
Streaming ¶
A stream is a sequence of blocks. BlockStart opens the block at Index, the position it takes in Response.Blocks; deltas of the same kind grow it; BlockEnd closes it with the completed block. Blocks may interleave, and all end before DoneEvent. Handle aggregates the stream and passes each event to a callback; Collect only aggregates. Stream is for single-goroutine consumption.
stream, err := client.Stream(ctx, req)
if err != nil {
panic(err)
}
defer stream.Close()
resp, err := litellm.Handle(stream, func(event litellm.Event) error {
if e, ok := event.(litellm.TextDelta); ok {
fmt.Print(e.Text)
}
return nil
})
Design ¶
The SDK is intentionally not a gateway, router, agent runtime, account system, or request scheduler. It binds one Client to one Provider and exposes explicit configuration and structural validation. It maps structure only: it does not infer model features, validate vendor values locally, or rewrite user input; the vendor API is the authority. ProviderOptions carry native wire fields, and optional usage counters distinguish unknown from zero.
Index ¶
- Constants
- func IsAuthError(err error) bool
- func IsContentFilterError(err error) bool
- func IsContextOverflowError(err error) bool
- func IsInternalError(err error) bool
- func IsModelError(err error) bool
- func IsNetworkError(err error) bool
- func IsOverloadedError(err error) bool
- func IsProviderError(err error) bool
- func IsQuotaError(err error) bool
- func IsRateLimitError(err error) bool
- func IsStreamIdleError(err error) bool
- func IsTemporaryError(err error) bool
- func IsTimeoutError(err error) bool
- func IsValidationError(err error) bool
- func RetryAfter(err error) time.Duration
- func WrapError(provider string, fallback ErrorType, err error) error
- type Annotation
- type Block
- type BlockEnd
- type BlockStart
- type CacheControl
- type CallInfo
- type CallObserver
- type CallResult
- type CallStatus
- type Capabilities
- type CapabilityProvider
- type Client
- func (c *Client) Capabilities() (caps Capabilities, ok bool)
- func (c *Client) Chat(ctx context.Context, req Request) (*Response, error)
- func (c *Client) ListModels(ctx context.Context) ([]ModelInfo, error)
- func (c *Client) ProviderName() string
- func (c *Client) Stream(ctx context.Context, req Request) (Stream, error)
- type ClientOption
- type DoneEvent
- type Error
- type ErrorType
- type Event
- type FinishReason
- type HTTPClient
- type ImageBlock
- type JSONSchema
- type Message
- type ModelInfo
- type ModelLister
- type Observer
- type ObserverFunc
- type Provider
- type ProviderEvent
- type ProviderOptions
- type ProviderState
- type ReasoningBlock
- type ReasoningDelta
- type Request
- type Response
- type ResponseFormat
- type ResponseFormatType
- type Role
- type Schema
- type Stream
- type StrictMode
- type TextBlock
- type TextDelta
- type Thinking
- type ThinkingMode
- type Tool
- type ToolChoice
- type ToolChoiceMode
- type ToolReferenceBlock
- type ToolResultBlock
- type ToolUseBlock
- type ToolUseDelta
- type Usage
- type UsageEvent
- type Warning
- type WarningEvent
Constants ¶
const ( CacheTTL5m = "5m" CacheTTL1h = "1h" )
Common CacheControl TTL values.
Variables ¶
This section is empty.
Functions ¶
func IsAuthError ¶ added in v1.5.0
IsAuthError reports whether err wraps an *Error of type ErrorTypeAuth.
func IsContentFilterError ¶ added in v1.8.6
IsContentFilterError reports whether err wraps an *Error of type ErrorTypeContentFilter.
func IsContextOverflowError ¶ added in v1.6.0
IsContextOverflowError reports whether err wraps an *Error of type ErrorTypeContextOverflow.
func IsInternalError ¶ added in v1.9.0
IsInternalError reports whether err wraps an *Error of type ErrorTypeInternal.
func IsModelError ¶ added in v1.5.0
IsModelError reports whether err wraps an *Error of type ErrorTypeModel.
func IsNetworkError ¶ added in v1.5.0
IsNetworkError reports whether err wraps an *Error of type ErrorTypeNetwork.
func IsOverloadedError ¶ added in v1.6.3
IsOverloadedError reports whether err wraps an *Error of type ErrorTypeOverloaded.
func IsProviderError ¶ added in v1.8.0
IsProviderError reports whether err wraps an *Error of type ErrorTypeProvider.
func IsQuotaError ¶ added in v1.9.0
IsQuotaError reports whether err wraps an *Error of type ErrorTypeQuota.
func IsRateLimitError ¶ added in v1.5.0
IsRateLimitError reports whether err wraps an *Error of type ErrorTypeRateLimit.
func IsStreamIdleError ¶ added in v1.6.8
IsStreamIdleError reports whether err is the WithStreamIdleTimeout failure.
func IsTemporaryError ¶ added in v1.9.0
IsTemporaryError reports whether the failure may resolve over time. It does not establish that repeating the operation is safe, even before any output.
func IsTimeoutError ¶ added in v1.8.0
IsTimeoutError reports whether err wraps an *Error of type ErrorTypeTimeout.
func IsValidationError ¶ added in v1.5.0
IsValidationError reports whether err wraps an *Error of type ErrorTypeValidation.
func RetryAfter ¶ added in v1.9.0
RetryAfter returns the server-suggested delay carried by err, or zero.
Types ¶
type Annotation ¶ added in v1.8.0
type Annotation struct {
Type string
Text string
URL string
Extra json.RawMessage
}
Annotation is a citation attached to text. Extra keeps the vendor entry verbatim.
type Block ¶ added in v1.8.0
type Block interface {
// contains filtered or unexported methods
}
Block is one piece of message content. The implementations are TextBlock, ImageBlock, ReasoningBlock, ToolUseBlock, ToolResultBlock and ToolReferenceBlock.
type BlockEnd ¶ added in v1.9.0
BlockEnd closes a block. Providers set Block only to deliver metadata that arrives late (State, annotations, logprobs, a tool's id and name): its non-empty metadata fields replace the started ones, while its text and arguments are ignored. Streams read through Handle, Collect or a Client deliver the completed block instead.
type BlockStart ¶ added in v1.9.0
BlockStart opens a TextBlock, ReasoningBlock or ToolUseBlock. Block carries the metadata known up front, such as a tool's ID and Name, and may carry initial text.
type CacheControl ¶ added in v1.5.0
type CacheControl struct {
TTL string
}
CacheControl marks a cache breakpoint: the prompt prefix up to and including this block may be cached. TTL is passed to the vendor as is; empty selects the vendor default.
type CallInfo ¶ added in v1.9.0
CallInfo describes the caller's request before validation. Request is a snapshot shared by all observers: read it, never modify it.
type CallObserver ¶ added in v1.9.0
type CallObserver interface {
OnEvent(Event)
End(CallResult)
}
CallObserver belongs to one invocation. Events and the Response are snapshots shared by all observers and isolated from the caller: read, never modify. OnEvent receives stream events as they arrive, and Chat warnings as WarningEvent. End runs exactly once; observers end in reverse registration order so nested observation scopes unwind correctly. Streams must be consumed to termination or explicitly closed; cancellation alone does not run callbacks in a background goroutine.
type CallResult ¶ added in v1.9.0
type CallResult struct {
Status CallStatus
Response *Response
Err error
Duration time.Duration
}
CallResult describes SDK execution, not errors in application consumer callbacks. Response may be partial on failure or early Close. A later resource cleanup error does not revise an already completed invocation; Close returns that error.
type CallStatus ¶ added in v1.9.0
type CallStatus string
CallStatus is how an invocation ended. CallClosed means the caller closed a stream before it finished.
const ( CallCompleted CallStatus = "completed" CallFailed CallStatus = "failed" CallCanceled CallStatus = "canceled" CallClosed CallStatus = "closed" )
type Capabilities ¶ added in v1.8.1
type Capabilities struct {
// Thinking reports that Request.Thinking is sent to the vendor.
Thinking bool
// DisableThinking, ThinkingEffort and ThinkingBudget report that
// ThinkingDisabled, Thinking.Effort and Thinking.BudgetTokens are sent;
// otherwise the adapter rejects them before sending.
DisableThinking bool
ThinkingEffort bool
ThinkingBudget bool
// ProviderOptions lists the accepted Request.ProviderOptions keys, sorted.
// A provider that passes every key through, such as compat, lists none.
ProviderOptions []string
}
Capabilities states what a provider adapter can express on the wire. It is static per provider: whether a model honors a request is the vendor's call, and the vendor's error is the source of truth.
type CapabilityProvider ¶ added in v1.8.1
type CapabilityProvider interface {
Capabilities() Capabilities
}
CapabilityProvider is implemented by providers that declare Capabilities.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client runs requests through a Provider. It copies and validates each request, attributes errors to the provider, and notifies Observers. A Client is safe for concurrent use.
func New ¶
func New(provider Provider, opts ...ClientOption) (*Client, error)
New returns a Client for provider.
func (*Client) Capabilities ¶ added in v1.8.1
func (c *Client) Capabilities() (caps Capabilities, ok bool)
Capabilities reports what the provider adapter can express on the wire; ok is false when the provider does not implement CapabilityProvider, so the facts are unknown.
func (*Client) Chat ¶ added in v1.5.0
Chat sends req and returns the complete response. On error the response may still be returned when the provider produced one.
func (*Client) ListModels ¶ added in v1.5.5
ListModels lists the provider's models; it fails with a validation error when the provider is not a ModelLister.
func (*Client) ProviderName ¶ added in v1.5.7
ProviderName returns the provider's Name.
type ClientOption ¶
ClientOption configures a Client.
func WithCaptureRawResponse ¶ added in v1.8.0
func WithCaptureRawResponse(enabled bool) ClientOption
WithCaptureRawResponse keeps the raw vendor body in Response.Raw.
func WithObservers ¶ added in v1.9.0
func WithObservers(observers ...Observer) ClientOption
WithObservers adds observers, skipping nil ones.
func WithStreamIdleTimeout ¶ added in v1.8.0
func WithStreamIdleTimeout(timeout time.Duration) ClientOption
WithStreamIdleTimeout fails a stream, with an error IsStreamIdleError reports, when no event arrives for timeout. Zero disables the check.
type DoneEvent ¶ added in v1.8.0
type DoneEvent struct {
FinishReason FinishReason
FinishReasonRaw string
Provider string
Model string
}
DoneEvent ends a successful stream.
type Error ¶ added in v1.9.0
type Error struct {
Type ErrorType
Code string
Message string
Provider string
StatusCode int
// Temporary describes a potentially transient failure, not permission to replay
// a request. The provider may already have processed or billed the operation.
Temporary bool
// RetryAfter is the server-suggested delay; zero means unspecified.
RetryAfter time.Duration
Cause error
}
Error is the error type returned by the client and all providers.
func NewError ¶ added in v1.5.0
NewError builds an error of the given type. provider may be empty and cause nil; Temporary follows the type.
func NewNetworkError ¶ added in v1.5.0
NewNetworkError builds a transport error. Context cancellation is a non-temporary network error and a deadline a timeout; other causes are temporary.
type ErrorType ¶ added in v1.5.0
type ErrorType string
ErrorType classifies an Error.
const ( ErrorTypeAuth ErrorType = "auth" ErrorTypeRateLimit ErrorType = "rate_limit" ErrorTypeNetwork ErrorType = "network" ErrorTypeValidation ErrorType = "validation" ErrorTypeProvider ErrorType = "provider" ErrorTypeTimeout ErrorType = "timeout" ErrorTypeQuota ErrorType = "quota" ErrorTypeModel ErrorType = "model" ErrorTypeInternal ErrorType = "internal" ErrorTypeContextOverflow ErrorType = "context_overflow" ErrorTypeOverloaded ErrorType = "overloaded" ErrorTypeContentFilter ErrorType = "content_filter" )
type Event ¶ added in v1.8.0
type Event interface {
// contains filtered or unexported methods
}
Event is a stream event. See the types below for the block lifecycle.
type FinishReason ¶ added in v1.8.0
type FinishReason string
FinishReason is a normalized stop reason; FinishReasonRaw keeps the vendor value. Empty means the vendor reported none.
const ( FinishReasonStop FinishReason = "stop" FinishReasonLength FinishReason = "length" FinishReasonToolCall FinishReason = "tool_calls" FinishReasonError FinishReason = "error" // FinishReasonSafety also covers explicit refusals; the refusal text, if // any, is an ordinary TextBlock. FinishReasonSafety FinishReason = "safety" // FinishReasonOther is a reported reason without a normalized equivalent. FinishReasonOther FinishReason = "other" )
type HTTPClient ¶ added in v1.9.0
HTTPClient sends provider requests; *http.Client satisfies it.
type ImageBlock ¶ added in v1.8.0
type ImageBlock struct {
URL string
Data []byte
MIME string
FileURI string
Detail string
Cache *CacheControl
}
ImageBlock is an image. Set one source: URL, Data with MIME, or FileURI, a vendor file reference. Detail is sent where the wire format has it.
func ImageURL ¶ added in v1.8.0
func ImageURL(url string) ImageBlock
ImageURL returns an ImageBlock for a URL, which may be a data URL.
type JSONSchema ¶ added in v1.2.1
type JSONSchema struct {
Name string
Description string
Schema Schema
Strict StrictMode
}
JSONSchema is a named schema for structured output.
type Message ¶
Message is one conversation turn.
func Assistant ¶ added in v1.8.0
Assistant returns an assistant message, such as a prior reply to replay.
func AssistantText ¶ added in v1.8.0
AssistantText returns an assistant message with one text block.
func ToolResult ¶ added in v1.8.0
ToolResult returns a tool message answering the tool call toolUseID.
func ToolResultText ¶ added in v1.8.0
ToolResultText returns a tool message with a text result.
type ModelInfo ¶
type ModelInfo struct {
ID string
Name string
Provider string
Description string
ContextLength int
InputTokenLimit int
OutputTokenLimit int
Created int64
}
ModelInfo describes a model; fields the vendor does not report are zero.
type ModelLister ¶ added in v1.5.5
ModelLister is implemented by providers that can list models.
type Observer ¶ added in v1.9.0
Observer starts one observation per Chat/Stream invocation, including local validation failures. Its returned non-nil context reaches subsequent observers and the provider; derive it from the input to preserve cancellation and values. Returning a nil CallObserver opts out of observing this call. Callbacks are synchronous; the SDK does not recover observer panics.
type ObserverFunc ¶ added in v1.9.0
ObserverFunc adapts a function to Observer.
func (ObserverFunc) Start ¶ added in v1.9.0
func (f ObserverFunc) Start(ctx context.Context, info CallInfo) (context.Context, CallObserver)
Start calls f.
type Provider ¶
type Provider interface {
Name() string
Chat(context.Context, *Request) (*Response, error)
Stream(context.Context, *Request) (Stream, error)
}
Provider adapts one vendor protocol. Chat and Stream receive a validated copy of the request that the provider owns. Implementations may also implement ModelLister and CapabilityProvider.
type ProviderEvent ¶ added in v1.8.0
type ProviderEvent struct {
Name string
Raw json.RawMessage
}
ProviderEvent exposes a provider-native event that has no typed equivalent. It does not affect the aggregated Response.
type ProviderOptions ¶ added in v1.8.0
type ProviderOptions map[string]json.RawMessage
ProviderOptions contains JSON values owned by the request. Use NewProviderOptions or Set to encode Go values; the client copies each value before observation or execution.
func NewProviderOptions ¶ added in v1.9.0
func NewProviderOptions(values map[string]any) (ProviderOptions, error)
NewProviderOptions serializes values immediately, so subsequent mutations of the supplied Go objects cannot change the request. Encoding errors are returned.
type ProviderState ¶ added in v1.9.0
type ProviderState struct {
// Provider is the Name of the provider that produced the block.
Provider string
// Model is the requested model.
Model string
// Data holds the vendor's native fields as JSON.
Data json.RawMessage
}
ProviderState is data a provider attaches to a block it produced so the block can be sent back to it: a reasoning signature, encrypted reasoning or an item id. Keep it with the block when storing history; only the provider named by Provider reads it.
Replay rule: portable content (text, reasoning text, tool calls) maps to wherever the target wire format can carry it, while State is sent only to the provider that produced it. A block the wire cannot carry without its State, such as Anthropic thinking, is dropped; where the vendor documents a placeholder for foreign content, the provider supplies it. Blocks built by the caller (nil State) count as foreign.
type ReasoningBlock ¶ added in v1.8.0
type ReasoningBlock struct {
Text string
Summary bool
State *ProviderState
Cache *CacheControl
}
ReasoningBlock is model reasoning. Summary marks Text as a summary rather than the full reasoning. Text is empty when the vendor returns the reasoning encrypted or redacted; State then carries it.
type ReasoningDelta ¶ added in v1.8.0
ReasoningDelta appends text to a ReasoningBlock.
type Request ¶
type Request struct {
Model string
Messages []Message
MaxTokens *int
Temperature *float64
TopP *float64
Stop []string
Tools []Tool
ToolChoice *ToolChoice
ResponseFormat *ResponseFormat
Thinking *Thinking
ProviderOptions ProviderOptions
}
Request is a provider-neutral chat request. Nil pointers and empty fields are omitted from the wire, leaving the vendor default.
type Response ¶
type Response struct {
Blocks []Block
Usage Usage
Model string
Provider string
FinishReason FinishReason
FinishReasonRaw string
Warnings []Warning
Raw json.RawMessage
}
Response is a complete model reply. Providers set Raw to the vendor body of a non-streaming reply; the Client keeps it only with WithCaptureRawResponse.
func Collect ¶ added in v1.8.0
Collect consumes the stream and returns the aggregated Response. On failure it returns the partial response together with the error. EOF before DoneEvent is io.ErrUnexpectedEOF; partial responses must not be treated as completed output.
func Handle ¶ added in v1.8.0
Handle consumes the stream, invoking fn for each event as it arrives, and returns the aggregated Response. Events are aggregated before fn runs, so a BlockEnd passed to fn carries the completed block. A nil fn behaves exactly like Collect. On any failure Handle returns the partial response and the error. The caller still owns Close.
func (*Response) ToolCalls ¶
func (r *Response) ToolCalls() []ToolUseBlock
ToolCalls returns the tool use blocks in order.
type ResponseFormat ¶ added in v1.2.1
type ResponseFormat struct {
Type ResponseFormatType
JSONSchema *JSONSchema
}
ResponseFormat constrains the output format. JSONSchema is used with ResponseFormatJSONSchema. Providers without native schema support may use a prompt instead and report a Warning; this does not enforce schema adherence, including when StrictEnabled is set. See providers.md for the mapping.
func NewResponseFormatJSONObject ¶ added in v1.2.1
func NewResponseFormatJSONObject() *ResponseFormat
NewResponseFormatJSONObject requests a JSON object.
func NewResponseFormatJSONSchema ¶ added in v1.2.1
func NewResponseFormatJSONSchema(name, description string, schema any, strict StrictMode) (*ResponseFormat, error)
NewResponseFormatJSONSchema requests output matching schema, converted with SchemaFrom.
func NewResponseFormatText ¶ added in v1.2.1
func NewResponseFormatText() *ResponseFormat
NewResponseFormatText requests plain text output.
type ResponseFormatType ¶ added in v1.8.0
type ResponseFormatType string
ResponseFormatType selects the output format.
const ( ResponseFormatText ResponseFormatType = "text" ResponseFormatJSONObject ResponseFormatType = "json_object" ResponseFormatJSONSchema ResponseFormatType = "json_schema" )
type Schema ¶ added in v1.8.0
type Schema json.RawMessage
Schema is a JSON Schema document.
func SchemaFrom ¶ added in v1.8.0
SchemaFrom returns v as a Schema. JSON text (Schema, json.RawMessage, []byte or string) is validated and copied; any other value is marshaled. A nil v returns a nil Schema.
type Stream ¶ added in v1.8.0
Stream is consumed by one goroutine. A successful stream emits exactly one DoneEvent. Failures are returned by Next, never encoded as events. Callers must Close the stream, including after completion or failure.
type StrictMode ¶ added in v1.8.0
type StrictMode int
StrictMode controls strict schema adherence. StrictDefault leaves the wire flag unset.
const ( StrictDefault StrictMode = iota StrictEnabled StrictDisabled )
func (StrictMode) Value ¶ added in v1.9.0
func (m StrictMode) Value() (strict, set bool)
Value returns the wire strict flag and whether the mode sets one.
type TextBlock ¶ added in v1.8.0
type TextBlock struct {
Text string
Annotations []Annotation
Logprobs json.RawMessage
State *ProviderState
Cache *CacheControl
}
TextBlock is plain text. Annotations and Logprobs are response metadata.
type Thinking ¶ added in v1.8.0
type Thinking struct {
Mode ThinkingMode
Effort string
BudgetTokens *int
IncludeOutput bool
}
Thinking configures model reasoning. Effort and BudgetTokens are sent as given; which values a model accepts is the vendor's decision. IncludeOutput is a hint to return reasoning where the vendor makes it optional; providers without such a switch ignore it.
type ThinkingMode ¶ added in v1.8.0
type ThinkingMode int
ThinkingMode is ThinkingEnabled by default; a nil *Thinking leaves thinking to the vendor default.
const ( ThinkingEnabled ThinkingMode = iota ThinkingDisabled )
type Tool ¶
type Tool struct {
Name string
Description string
Parameters Schema
Strict StrictMode
}
Tool declares a function the model may call.
type ToolChoice ¶ added in v1.8.0
type ToolChoice struct {
Mode ToolChoiceMode
Name string
}
ToolChoice selects a policy, or a named tool when Name is set. A nil choice leaves selection to the provider. Mode and Name are mutually exclusive.
type ToolChoiceMode ¶ added in v1.9.0
type ToolChoiceMode string
ToolChoiceMode is a tool selection policy.
const ( ToolChoiceAuto ToolChoiceMode = "auto" ToolChoiceNone ToolChoiceMode = "none" ToolChoiceRequired ToolChoiceMode = "required" )
type ToolReferenceBlock ¶ added in v1.8.0
type ToolReferenceBlock struct {
ToolName string
Cache *CacheControl
}
ToolReferenceBlock names a tool inside tool result content, as returned by a tool search tool.
type ToolResultBlock ¶ added in v1.8.0
type ToolResultBlock struct {
ToolUseID string
Content []Block
IsError bool
Cache *CacheControl
}
ToolResultBlock answers the ToolUseBlock with ID ToolUseID. Content holds TextBlock, ImageBlock or ToolReferenceBlock values.
type ToolUseBlock ¶ added in v1.8.0
type ToolUseBlock struct {
ID string
Name string
Arguments json.RawMessage
State *ProviderState
Cache *CacheControl
}
ToolUseBlock is a tool call from the assistant. Arguments is the JSON the model produced.
type ToolUseDelta ¶ added in v1.8.0
ToolUseDelta appends a fragment of a ToolUseBlock's JSON arguments.
type Usage ¶
type Usage struct {
InputTokens *int
OutputTokens *int
TotalTokens *int
ReasoningTokens *int
CacheReadTokens *int
CacheWriteTokens *int
}
Usage contains reported token counts. A nil count is unknown; a pointer to zero is a known zero. InputTokens includes cache reads and writes; OutputTokens includes reasoning. Detail counts are subsets, not additions.
func (Usage) CacheWrite ¶ added in v1.9.0
CacheWrite returns CacheWriteTokens and whether it is known.
func (Usage) HasTokens ¶ added in v1.8.0
HasTokens reports whether any token count is known, including a known zero.
type UsageEvent ¶ added in v1.8.0
type UsageEvent struct {
Usage Usage
}
UsageEvent carries cumulative usage; a later event replaces an earlier one.
type Warning ¶ added in v1.8.0
Warning reports a non-fatal issue, such as dropped content or a generated ID. Code has the form "<source>.<snake_case>", such as "litellm.tool_arguments_invalid".
type WarningEvent ¶ added in v1.8.0
type WarningEvent struct {
Warning Warning
}
WarningEvent carries a Warning; it is also collected into Response.Warnings.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
examples
|
|
|
anthropic
command
|
|
|
bedrock
command
|
|
|
deepseek
command
|
|
|
gemini
command
|
|
|
glm
command
|
|
|
grok
command
|
|
|
minimax
command
|
|
|
ollama
command
|
|
|
openai
command
|
|
|
openrouter
command
|
|
|
qwen
command
|
|
|
internal
|
|
|
claude
Package claude holds the Claude request mapping shared by the Anthropic and Bedrock adapters.
|
Package claude holds the Claude request mapping shared by the Anthropic and Bedrock adapters. |
|
wire
Package wire holds what provider adapters share to speak vendor wire protocols: HTTP transport and Server-Sent Events, error and stop-reason classification, stream block indexing, ProviderOptions merging, replay state, and token and data URL helpers.
|
Package wire holds what provider adapters share to speak vendor wire protocols: HTTP transport and Server-Sent Events, error and stop-reason classification, stream block indexing, ProviderOptions merging, replay state, and token and data URL helpers. |
|
otel
module
|
|
|
Package pricing computes request cost from Usage and per-token rates, set explicitly or loaded into a Registry from LiteLLM's price list.
|
Package pricing computes request cost from Usage and per-token rates, set explicitly or loaded into a Registry from LiteLLM's price list. |
|
provider
|
|
|
anthropic
Package anthropic connects to the Anthropic Messages API.
|
Package anthropic connects to the Anthropic Messages API. |
|
bedrock
Package bedrock connects to the Amazon Bedrock Converse API.
|
Package bedrock connects to the Amazon Bedrock Converse API. |
|
compat
Package compat connects to any OpenAI-compatible Chat Completions endpoint, such as vLLM, LM Studio or a gateway, and holds the Config shared by the vendor wrappers (deepseek, glm, grok, mimo, minimax, ollama, openrouter, qwen).
|
Package compat connects to any OpenAI-compatible Chat Completions endpoint, such as vLLM, LM Studio or a gateway, and holds the Config shared by the vendor wrappers (deepseek, glm, grok, mimo, minimax, ollama, openrouter, qwen). |
|
deepseek
Package deepseek connects to the DeepSeek API.
|
Package deepseek connects to the DeepSeek API. |
|
gemini
Package gemini connects to the Gemini API.
|
Package gemini connects to the Gemini API. |
|
glm
Package glm connects to the Zhipu GLM API.
|
Package glm connects to the Zhipu GLM API. |
|
grok
Package grok connects to the xAI Grok API.
|
Package grok connects to the xAI Grok API. |
|
internal/openaicompat
Package openaicompat implements the Chat Completions protocol shared by OpenAI and the OpenAI-compatible providers.
|
Package openaicompat implements the Chat Completions protocol shared by OpenAI and the OpenAI-compatible providers. |
|
internal/openaicompat/compattest
Package compattest runs openaicompat providers against canned HTTP responses in tests.
|
Package compattest runs openaicompat providers against canned HTTP responses in tests. |
|
mimo
Package mimo connects to the Xiaomi MiMo API.
|
Package mimo connects to the Xiaomi MiMo API. |
|
minimax
Package minimax connects to the MiniMax API.
|
Package minimax connects to the MiniMax API. |
|
ollama
Package ollama connects to Ollama's OpenAI-compatible endpoint.
|
Package ollama connects to Ollama's OpenAI-compatible endpoint. |
|
openai
Package openai connects to the OpenAI API, or any endpoint that implements it exactly (such as Azure OpenAI), through Chat Completions or Responses.
|
Package openai connects to the OpenAI API, or any endpoint that implements it exactly (such as Azure OpenAI), through Chat Completions or Responses. |
|
openrouter
Package openrouter connects to the OpenRouter API.
|
Package openrouter connects to the OpenRouter API. |
|
qwen
Package qwen connects to the Alibaba Cloud DashScope compatible-mode API.
|
Package qwen connects to the Alibaba Cloud DashScope compatible-mode API. |
|
Package retry provides opt-in HTTP retries for providers.
|
Package retry provides opt-in HTTP retries for providers. |