README
¶
github.com/tyemirov/utils/llm
Overview
llm provides the reusable plumbing for chat-completion based flows. It exposes:
Message,ChatRequest, and theChatClientinterface so call sites can build prompts without depending on HTTP details.Client, a minimal HTTP adapter to/chat/completionswith request validation, timeout handling, and error trimming.Factory, a wrapper that layers configurable retry/backoff semantics and still satisfies theChatClientinterface.- Helper types (
Config,ResponseFormat,RetryPolicy, etc.) so packages can describe their needs declaratively.
The package is dependency-free beyond the Go standard library.
Usage
Construct a client (or factory) with llm.Config:
llmConfig := llm.Config{
BaseURL: os.Getenv("OPENAI_BASE_URL"),
APIKey: os.Getenv("OPENAI_API_KEY"),
Model: "gpt-4.1-mini",
MaxCompletionTokens: 512,
Temperature: 0.2,
HTTPClient: &http.Client{Timeout: 30 * time.Second},
RequestTimeout: 60 * time.Second,
RetryAttempts: 3,
RetryInitialBackoff: 200 * time.Millisecond,
RetryMaxBackoff: 2 * time.Second,
RetryBackoffFactor: 2,
}
- Use
client, _ := llm.NewClient(cfg)when you want a single request/response without automatic retries. - Use
factory, _ := llm.NewFactory(cfg)when you want retry/backoff semantics. The factory still implementsChatClient.
Documentation
¶
Overview ¶
Package llm provides reusable client and factory implementations for chat-based large language model integrations. It exposes lightweight interfaces (`ChatClient`), request/response types (`Message`, `ChatRequest`), and configurable HTTP plumbing so higher-level packages can inject or mock LLM dependencies without duplicating API code.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrEmptyResponse = errors.New("llm returned empty response")
ErrEmptyResponse indicates the LLM returned only whitespace or an empty payload.
Functions ¶
This section is empty.
Types ¶
type ChatClient ¶
type ChatClient interface {
Chat(ctx context.Context, request ChatRequest) (string, error)
}
ChatClient issues chat completion requests.
type ChatRequest ¶
type ChatRequest struct {
Model string
Messages []Message
MaxTokens int
Temperature *float64
ResponseFormat *ResponseFormat
}
ChatRequest describes a chat completion request.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client communicates with an LLM chat completion endpoint.
type Config ¶
type Config struct {
BaseURL string
APIKey string
Model string
MaxCompletionTokens int
Temperature float64
HTTPClient HTTPClient
RequestTimeout time.Duration
RetryAttempts int
RetryInitialBackoff time.Duration
RetryMaxBackoff time.Duration
RetryBackoffFactor float64
}
Config configures an LLM client.
type Factory ¶
type Factory struct {
// contains filtered or unexported fields
}
Factory constructs chat clients and enforces retry/backoff behaviour.
func NewFactory ¶
func NewFactory(configuration Config, options ...FactoryOption) (*Factory, error)
NewFactory builds a Factory using the provided configuration.
type FactoryOption ¶
type FactoryOption func(*Factory)
FactoryOption customises factory behaviour.
func WithRetryPolicy ¶
func WithRetryPolicy(policy RetryPolicy) FactoryOption
WithRetryPolicy overrides the retry policy applied by the factory.
func WithSleepFunc ¶
func WithSleepFunc(fn SleepFunc) FactoryOption
WithSleepFunc overrides the sleep function used between retries (intended for tests).
type HTTPClient ¶
HTTPClient issues HTTP requests.
type ResponseFormat ¶
type ResponseFormat struct {
Type string `json:"type"`
Name string `json:"name"`
Schema json.RawMessage `json:"schema,omitempty"`
Strict bool `json:"strict,omitempty"`
}
ResponseFormat configures structured responses.
Source Files