Documentation
¶
Index ¶
- func NewAnthropicLLM(apiKey, model string) (driven.LLMService, error)
- func NewCohereEmbedding(apiKey, model string) (driven.EmbeddingService, error)
- func NewOllamaEmbedding(baseURL, model string) (driven.EmbeddingService, error)
- func NewOllamaLLM(baseURL, model string) (driven.LLMService, error)
- func NewOpenAIEmbedding(apiKey, model, baseURL string, opts ...OpenAIEmbeddingOption) (driven.EmbeddingService, error)
- func NewOpenAILLM(apiKey, model, baseURL string, opts ...OpenAILLMOption) (driven.LLMService, error)
- func NewVoyageEmbedding(apiKey, model string) (driven.EmbeddingService, error)
- type Factory
- type OpenAIEmbedding
- func (e *OpenAIEmbedding) Close() error
- func (e *OpenAIEmbedding) Dimensions() int
- func (e *OpenAIEmbedding) Embed(ctx context.Context, texts []string) ([][]float32, error)
- func (e *OpenAIEmbedding) EmbedQuery(ctx context.Context, query string) ([]float32, error)
- func (e *OpenAIEmbedding) HealthCheck(ctx context.Context) error
- func (e *OpenAIEmbedding) Model() string
- type OpenAIEmbeddingOption
- func WithEmbeddingMaxRetries(n int) OpenAIEmbeddingOption
- func WithEmbeddingMaxRetryElapsed(d time.Duration) OpenAIEmbeddingOption
- func WithEmbeddingRPMLimit(rpm int64) OpenAIEmbeddingOption
- func WithEmbeddingTPMLimit(tpm int64) OpenAIEmbeddingOption
- func WithEmbeddingTransportSleep(fn func(ctx context.Context, d time.Duration) error) OpenAIEmbeddingOption
- type OpenAILLM
- type OpenAILLMOption
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewAnthropicLLM ¶
func NewAnthropicLLM(apiKey, model string) (driven.LLMService, error)
func NewCohereEmbedding ¶
func NewCohereEmbedding(apiKey, model string) (driven.EmbeddingService, error)
func NewOllamaEmbedding ¶
func NewOllamaEmbedding(baseURL, model string) (driven.EmbeddingService, error)
func NewOllamaLLM ¶
func NewOllamaLLM(baseURL, model string) (driven.LLMService, error)
func NewOpenAIEmbedding ¶
func NewOpenAIEmbedding(apiKey, model, baseURL string, opts ...OpenAIEmbeddingOption) (driven.EmbeddingService, error)
NewOpenAIEmbedding creates a new OpenAI embedding service.
The constructor reads two env vars to size the rate-limiter:
- EMBEDDER_TPM (default 1000000): tokens-per-minute budget
- EMBEDDER_RPM (default 3000): requests-per-minute budget
Retry policy (5 attempts, 60s total) is hard-coded; tests override via the WithEmbeddingMaxRetries / WithEmbeddingMaxRetryElapsed options.
These defaults are conservative and work without tuning. Pass opts to override any of them programmatically (e.g. in tests).
The public surface (Embed, EmbedQuery, Dimensions, Model, HealthCheck, Close) is unchanged from the previous version.
func NewOpenAILLM ¶
func NewOpenAILLM(apiKey, model, baseURL string, opts ...OpenAILLMOption) (driven.LLMService, error)
NewOpenAILLM creates a new OpenAI LLM service.
Rate limiting is configured from env vars because real ceilings vary by account tier, deployment (custom fine-tunes, OpenAI-compatible proxies), and promotion. The operator who knows the account's tier sets these:
- LLM_TPM (default 200000): tokens-per-minute budget
- LLM_RPM (default 500): requests-per-minute budget
Defaults are conservative — sized for OpenAI's tier-1 ceiling on the most-restricted modern chat models. Operators on higher tiers should raise the env vars to match.
Retry behaviour is hard-coded (5 attempts, 60s total budget). These values are a transport policy decision, not an operator-tunable knob; tests use WithLLMMaxRetries / WithLLMMaxRetryElapsed when they need to disable or shorten retries.
The public surface (Complete, Model, Ping, Close) is unchanged from the previous version.
func NewVoyageEmbedding ¶
func NewVoyageEmbedding(apiKey, model string) (driven.EmbeddingService, error)
Types ¶
type Factory ¶
type Factory struct{}
Factory creates AI services based on configuration
func (*Factory) CreateEmbeddingService ¶
func (f *Factory) CreateEmbeddingService(settings *domain.EmbeddingSettings, credentials *driven.AICredentials) (driven.EmbeddingService, error)
CreateEmbeddingService creates an embedding service from settings and credentials
func (*Factory) CreateLLMService ¶
func (f *Factory) CreateLLMService(settings *domain.LLMSettings, credentials *driven.AICredentials) (driven.LLMService, error)
CreateLLMService creates an LLM service from settings and credentials
type OpenAIEmbedding ¶
type OpenAIEmbedding struct {
// contains filtered or unexported fields
}
OpenAIEmbedding implements EmbeddingService using OpenAI's embedding API.
func (*OpenAIEmbedding) Close ¶
func (e *OpenAIEmbedding) Close() error
Close releases resources held by the embedding service.
func (*OpenAIEmbedding) Dimensions ¶
func (e *OpenAIEmbedding) Dimensions() int
Dimensions returns the embedding dimension size.
func (*OpenAIEmbedding) EmbedQuery ¶
EmbedQuery generates an embedding for a search query.
func (*OpenAIEmbedding) HealthCheck ¶
func (e *OpenAIEmbedding) HealthCheck(ctx context.Context) error
HealthCheck verifies the embedding service is available.
func (*OpenAIEmbedding) Model ¶
func (e *OpenAIEmbedding) Model() string
Model returns the model name being used.
type OpenAIEmbeddingOption ¶ added in v0.4.0
type OpenAIEmbeddingOption func(*OpenAIEmbedding)
OpenAIEmbeddingOption configures an OpenAIEmbedding at construction time. Use the With* functions to create options.
func WithEmbeddingMaxRetries ¶ added in v0.4.0
func WithEmbeddingMaxRetries(n int) OpenAIEmbeddingOption
WithEmbeddingMaxRetries sets the maximum number of retry attempts for the embedding client. Defaults to 5 (hard-coded in NewOpenAIEmbedding).
func WithEmbeddingMaxRetryElapsed ¶ added in v0.4.0
func WithEmbeddingMaxRetryElapsed(d time.Duration) OpenAIEmbeddingOption
WithEmbeddingMaxRetryElapsed sets the maximum total elapsed time for retries. Defaults to 60s (hard-coded in NewOpenAIEmbedding).
func WithEmbeddingRPMLimit ¶ added in v0.4.0
func WithEmbeddingRPMLimit(rpm int64) OpenAIEmbeddingOption
WithEmbeddingRPMLimit sets the requests-per-minute budget while preserving the existing TPM bucket settings. Mostly for tests.
func WithEmbeddingTPMLimit ¶ added in v0.4.0
func WithEmbeddingTPMLimit(tpm int64) OpenAIEmbeddingOption
WithEmbeddingTPMLimit sets the tokens-per-minute budget. RPM gate is dropped — pair with WithEmbeddingRPMLimit if request-rate gating is also needed. Production wiring uses env vars (EMBEDDER_TPM/EMBEDDER_RPM); this option is primarily for tests.
func WithEmbeddingTransportSleep ¶ added in v0.4.0
func WithEmbeddingTransportSleep(fn func(ctx context.Context, d time.Duration) error) OpenAIEmbeddingOption
WithEmbeddingTransportSleep replaces the Transport's sleep function with fn. This is intended for tests that need to control or eliminate sleep delays without relying on real wall-clock time.
type OpenAILLM ¶ added in v0.2.2
type OpenAILLM struct {
// contains filtered or unexported fields
}
OpenAILLM implements LLMService using OpenAI's chat completion API.
func (*OpenAILLM) Complete ¶ added in v0.2.2
func (l *OpenAILLM) Complete(ctx context.Context, req domain.CompletionRequest) (domain.CompletionResponse, error)
Complete sends a completion request to the LLM and returns the response.
type OpenAILLMOption ¶ added in v0.4.0
type OpenAILLMOption func(*OpenAILLM)
OpenAILLMOption configures an OpenAILLM at construction time. Use the With* functions to create options.
func WithLLMMaxRetries ¶ added in v0.4.0
func WithLLMMaxRetries(n int) OpenAILLMOption
WithLLMMaxRetries sets the maximum number of retry attempts for the LLM client. Defaults to 5 (hard-coded in NewOpenAILLM).
func WithLLMMaxRetryElapsed ¶ added in v0.4.0
func WithLLMMaxRetryElapsed(d time.Duration) OpenAILLMOption
WithLLMMaxRetryElapsed sets the maximum total elapsed time for retries. Defaults to 60s (hard-coded in NewOpenAILLM).
func WithLLMRPMLimit ¶ added in v0.4.0
func WithLLMRPMLimit(rpm int64) OpenAILLMOption
WithLLMRPMLimit sets the requests-per-minute budget. The TPM bucket the transport already owns is preserved if it was constructed by NewOpenAILLM — replacing it with a bucket that includes RPM gating is the operation.
Production wiring uses the env vars; this option is primarily for tests.
func WithLLMTPMLimit ¶ added in v0.4.0
func WithLLMTPMLimit(tpm int64) OpenAILLMOption
WithLLMTPMLimit sets the tokens-per-minute budget for the LLM client's rate-limiter bucket. The RPM gate is dropped — callers using this option should pair with WithLLMRPMLimit if they need request-rate gating.
Production wiring uses the env vars (LLM_TPM/LLM_RPM); this option is primarily for tests that need a known budget without env manipulation.
func WithLLMTransportSleep ¶ added in v0.4.0
WithLLMTransportSleep replaces the Transport's sleep function with fn. This is intended for tests that need to control or eliminate sleep delays without relying on real wall-clock time.
Source Files