Documentation
¶
Overview ¶
Package provider defines the core interfaces that external LLM providers must implement. External provider packages should import this package to implement the Provider interface.
Index ¶
- type ChatCompletionChoice
- type ChatCompletionChunk
- type ChatCompletionRequest
- type ChatCompletionResponse
- type ChatCompletionStream
- type EmbeddingData
- type EmbeddingEncodingFormat
- type EmbeddingProvider
- type EmbeddingRequest
- type EmbeddingResponse
- type EmbeddingUsage
- type Message
- type Provider
- type ResponseFormat
- type Role
- type Tool
- type ToolCall
- type ToolFunction
- type ToolSpec
- type Usage
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ChatCompletionChoice ¶
type ChatCompletionChoice struct {
Index int `json:"index"`
Message Message `json:"message"`
Delta *Message `json:"delta,omitempty"`
FinishReason *string `json:"finish_reason"`
Logprobs any `json:"logprobs,omitempty"`
}
ChatCompletionChoice represents a single choice in the response
type ChatCompletionChunk ¶
type ChatCompletionChunk struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
Model string `json:"model"`
SystemFingerprint *string `json:"system_fingerprint,omitempty"`
Choices []ChatCompletionChoice `json:"choices"`
Usage *Usage `json:"usage,omitempty"`
ProviderMetadata map[string]any `json:"provider_metadata,omitempty"` // Provider-specific metadata
}
ChatCompletionChunk represents a chunk in streaming response
type ChatCompletionRequest ¶
type ChatCompletionRequest struct {
Model string `json:"model"`
Messages []Message `json:"messages"`
MaxTokens *int `json:"max_tokens,omitempty"`
Temperature *float64 `json:"temperature,omitempty"`
TopP *float64 `json:"top_p,omitempty"`
TopK *int `json:"top_k,omitempty"` // Anthropic, Gemini, Ollama
Stream *bool `json:"stream,omitempty"`
Stop []string `json:"stop,omitempty"`
PresencePenalty *float64 `json:"presence_penalty,omitempty"`
FrequencyPenalty *float64 `json:"frequency_penalty,omitempty"`
LogitBias map[string]int `json:"logit_bias,omitempty"`
User *string `json:"user,omitempty"`
Tools []Tool `json:"tools,omitempty"`
ToolChoice any `json:"tool_choice,omitempty"`
Seed *int `json:"seed,omitempty"` // OpenAI, X.AI - for reproducible outputs
N *int `json:"n,omitempty"` // OpenAI - number of completions
ResponseFormat *ResponseFormat `json:"response_format,omitempty"` // OpenAI, Gemini - JSON mode
Logprobs *bool `json:"logprobs,omitempty"` // OpenAI - return log probabilities
TopLogprobs *int `json:"top_logprobs,omitempty"` // OpenAI - number of top logprobs
}
ChatCompletionRequest represents a request for chat completion
type ChatCompletionResponse ¶
type ChatCompletionResponse struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
Model string `json:"model"`
SystemFingerprint *string `json:"system_fingerprint,omitempty"`
Choices []ChatCompletionChoice `json:"choices"`
Usage Usage `json:"usage"`
ProviderMetadata map[string]any `json:"provider_metadata,omitempty"` // Provider-specific metadata
}
ChatCompletionResponse represents a response from chat completion
type ChatCompletionStream ¶
type ChatCompletionStream interface {
// Recv receives the next chunk from the stream
Recv() (*ChatCompletionChunk, error)
// Close closes the stream
Close() error
}
ChatCompletionStream represents a streaming chat completion response
type EmbeddingData ¶ added in v0.16.0
type EmbeddingData struct {
// Object is the object type (always "embedding")
Object string `json:"object"`
// Index is the index of this embedding in the input array
Index int `json:"index"`
// Embedding is the vector representation as float64 values
Embedding []float64 `json:"embedding"`
}
EmbeddingData represents a single embedding vector
type EmbeddingEncodingFormat ¶ added in v0.16.0
type EmbeddingEncodingFormat string
EmbeddingEncodingFormat specifies the format of the embedding vectors
const ( // EmbeddingEncodingFormatFloat returns embeddings as float arrays (default) EmbeddingEncodingFormatFloat EmbeddingEncodingFormat = "float" // EmbeddingEncodingFormatBase64 returns embeddings as base64-encoded bytes EmbeddingEncodingFormatBase64 EmbeddingEncodingFormat = "base64" )
type EmbeddingProvider ¶ added in v0.16.0
type EmbeddingProvider interface {
// CreateEmbedding creates embeddings for the given input texts
CreateEmbedding(ctx context.Context, req *EmbeddingRequest) (*EmbeddingResponse, error)
// Close closes the provider and cleans up resources
Close() error
// Name returns the provider name
Name() string
}
EmbeddingProvider defines the interface for embedding providers. Embedding providers convert text into vector representations for semantic search, similarity matching, and retrieval-augmented generation (RAG).
Example usage:
provider, _ := omnillm.GetEmbeddingProvider("openai", omnillm.WithAPIKey(key))
resp, _ := provider.CreateEmbedding(ctx, &provider.EmbeddingRequest{
Model: "text-embedding-3-small",
Input: []string{"Hello world", "How are you?"},
})
vectors := resp.Data // []EmbeddingData with Vector []float64
type EmbeddingRequest ¶ added in v0.16.0
type EmbeddingRequest struct {
// Model is the embedding model to use (e.g., "text-embedding-3-small")
Model string `json:"model"`
// Input is the text(s) to embed. Can be a single string or array of strings.
Input []string `json:"input"`
// EncodingFormat specifies the format of the returned embeddings (optional)
// Defaults to "float". Some providers support "base64" for efficiency.
EncodingFormat EmbeddingEncodingFormat `json:"encoding_format,omitempty"`
// Dimensions specifies the number of dimensions for the output vectors (optional)
// Only supported by some models (e.g., text-embedding-3-small/large)
Dimensions *int `json:"dimensions,omitempty"`
// User is an optional unique identifier for the end-user
User *string `json:"user,omitempty"`
}
EmbeddingRequest represents a request for text embeddings
type EmbeddingResponse ¶ added in v0.16.0
type EmbeddingResponse struct {
// Object is the object type (always "list" for embeddings)
Object string `json:"object"`
// Data contains the embedding vectors
Data []EmbeddingData `json:"data"`
// Model is the model used to generate the embeddings
Model string `json:"model"`
// Usage contains token usage information
Usage EmbeddingUsage `json:"usage"`
// ProviderMetadata contains provider-specific metadata
ProviderMetadata map[string]any `json:"provider_metadata,omitempty"`
}
EmbeddingResponse represents a response from embedding creation
type EmbeddingUsage ¶ added in v0.16.0
type EmbeddingUsage struct {
// PromptTokens is the number of tokens in the input
PromptTokens int `json:"prompt_tokens"`
// TotalTokens is the total number of tokens used
TotalTokens int `json:"total_tokens"`
}
EmbeddingUsage represents token usage for embedding requests
type Message ¶
type Message struct {
Role Role `json:"role"`
Content string `json:"content"`
Name *string `json:"name,omitempty"`
ToolCallID *string `json:"tool_call_id,omitempty"`
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
}
Message represents a chat message
type Provider ¶
type Provider interface {
// CreateChatCompletion creates a new chat completion
CreateChatCompletion(ctx context.Context, req *ChatCompletionRequest) (*ChatCompletionResponse, error)
// CreateChatCompletionStream creates a streaming chat completion
CreateChatCompletionStream(ctx context.Context, req *ChatCompletionRequest) (ChatCompletionStream, error)
// Close closes the provider and cleans up resources
Close() error
// Name returns the provider name
Name() string
}
Provider defines the interface that all LLM providers must implement. External packages can implement this interface and inject via omnillm.ClientConfig.CustomProvider.
Example usage in external package:
import "github.com/plexusone/omnillm-core/provider"
func NewMyProvider(apiKey string) provider.Provider {
return &myProvider{apiKey: apiKey}
}
type ResponseFormat ¶
type ResponseFormat struct {
Type string `json:"type"` // "text" or "json_object"
}
ResponseFormat specifies the format of the response
type ToolCall ¶
type ToolCall struct {
ID string `json:"id"`
Type string `json:"type"`
Function ToolFunction `json:"function"`
}
ToolCall represents a tool function call
type ToolFunction ¶
ToolFunction represents the function being called
Directories
¶
| Path | Synopsis |
|---|---|
|
Package providertest provides conformance tests for LLM provider implementations.
|
Package providertest provides conformance tests for LLM provider implementations. |