Documentation
¶
Overview ¶
Package llm — HTTP client pooling per provider (Block 3.5).
langchaingo constructs a fresh net/http.Transport inside each provider constructor by default. For a long-running server that calls the same provider on every request, that leaks connections: every call-site allocates its own idle-conn pool, TLS session cache, and DNS resolver bucket. Pooling one *http.Client per provider (constructed here) fixes the leak.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Option ¶
type Option func(*callOptions)
Option configures LLM completion.
func WithJSONMode ¶
func WithJSONMode() Option
func WithMaxTokens ¶
func WithTemperature ¶
type Provider ¶
type Provider interface {
Complete(ctx context.Context, prompt string, opts ...Option) (string, error)
Embed(ctx context.Context, text string) ([]float32, error)
EmbedBatch(ctx context.Context, texts []string) ([][]float32, error)
Name() string
ModelID() string
// BatchCeiling returns the maximum number of texts that can be
// passed to EmbedBatch in a single call. Callers that need to
// process larger inputs must slice to this ceiling. Zero means
// "no declared ceiling" (rare — only for providers that don't
// care). Block 3.4.
BatchCeiling() int
}
Provider is the unified LLM interface.
func NewProvider ¶
NewProvider constructs the configured provider. When cfg.Provider is "none", NewProvider returns (nil, nil) — the caller must treat a nil Provider as "LLM disabled" and guard accordingly.
func ProviderForProject ¶
ProviderForProject resolves the LLM provider for a given project slug, honoring cfg.LLMOverrides. An unknown / empty slug falls back to cfg.LLM gracefully. Providers are not cached — callers that issue many requests per second should memoize at the call site.
Source Files
Directories