Documentation
¶
Overview ¶
Package llm adapts langchaingo models for the agent loop and single-shot module tasks.
Index ¶
- Constants
- Variables
- func AgentEnabled(name string) bool
- func AgentModelName(name string) string
- func Complete(ctx context.Context, model llms.Model, systemPrompt string, ...) (string, error)
- func CompleteWithRetry(ctx context.Context, model llms.Model, systemPrompt string, ...) (string, error)
- func DeepSeekThinkingHTTPClientForTest(base http.RoundTripper) *http.Client
- func GenerateWithTemplate(ctx context.Context, modelName string, template ChatTemplate, ...) (string, error)
- func GetOrCreateModel(ctx context.Context, modelName string) (llms.Model, string, error)
- func IdleTimeoutReadCloserForTest(rc io.ReadCloser, idleTimeout time.Duration) io.ReadCloser
- func IsRetryableLLMError(err error) bool
- func IsStreamIdleError(err error) bool
- func LLMGenerate(ctx context.Context, modelName, prompt string) (string, error)
- func NewModel(ctx context.Context, modelName string) (llms.Model, string, error)
- func ProviderForModel(modelName string) string
- func ReasoningCallOptions(modelName string, maxTokens int) []llms.CallOption
- func ResetModelPoolForTest()
- func SetModelCreatorForTest(fn modelCreatorFn)
- func StreamIdleTransportForTest(base http.RoundTripper) http.RoundTripper
- func SupportsReasoningStream(modelName string) bool
- type AssistantResult
- type ChatTemplate
- type FakeModel
- type ResponseScript
- type RetryConfig
- type StreamOptions
- type Usage
Constants ¶
const ( AgentChat = "chat" AgentBillClassify = "bill_classify" AgentExtractTags = "extract_tags" AgentSimilarTags = "similar_tags" AgentNewsSummary = "news_summary" AgentRepoReviewComment = "repo_review_comment" )
Agent name constants identify logical agent roles for LLM helpers.
const ( ProviderOpenAI = "openai" ProviderOpenAICompatible = "openai_compatible" ProviderGemini = "gemini" ProviderAnthropic = "anthropic" )
Model provider constants match config.models provider values.
Variables ¶
var ErrAborted = errors.New("agent llm: aborted")
ErrAborted indicates the LLM call was cancelled.
var ErrStreamIdle = errors.New("stream idle timeout")
ErrStreamIdle indicates the LLM stream stopped delivering deltas for longer than the idle limit.
var ErrStreamStarted = errors.New("agent llm: stream already started")
ErrStreamStarted indicates a retryable transport error occurred after streaming deltas were already delivered to the client; callers must not retry.
Functions ¶
func AgentEnabled ¶
AgentEnabled reports whether a named agent is configured with a model and enabled.
func AgentModelName ¶
AgentModelName returns the configured model for a named agent when enabled.
func Complete ¶
func Complete( ctx context.Context, model llms.Model, systemPrompt string, messages []llms.MessageContent, modelName string, maxTokens int, ) (string, error)
Complete performs a non-streaming completion for auxiliary tasks such as summarization. Retry policy follows chat_agent.llm_retry when configured.
func CompleteWithRetry ¶ added in v0.95.0
func CompleteWithRetry( ctx context.Context, model llms.Model, systemPrompt string, messages []llms.MessageContent, modelName string, maxTokens int, retryCfg RetryConfig, ) (string, error)
CompleteWithRetry performs a non-streaming completion with an explicit retry policy.
func DeepSeekThinkingHTTPClientForTest ¶ added in v0.94.0
func DeepSeekThinkingHTTPClientForTest(base http.RoundTripper) *http.Client
DeepSeekThinkingHTTPClientForTest exposes the DeepSeek thinking HTTP client for tests.
func GenerateWithTemplate ¶
func GenerateWithTemplate(ctx context.Context, modelName string, template ChatTemplate, data map[string]any) (string, error)
GenerateWithTemplate performs a single-shot completion using the given prompt template.
func GetOrCreateModel ¶
GetOrCreateModel returns a cached langchaingo model for the given model name.
func IdleTimeoutReadCloserForTest ¶ added in v0.95.0
func IdleTimeoutReadCloserForTest(rc io.ReadCloser, idleTimeout time.Duration) io.ReadCloser
IdleTimeoutReadCloserForTest exposes the idle timeout reader for tests.
func IsRetryableLLMError ¶ added in v0.95.0
IsRetryableLLMError reports whether an LLM error should be retried. Overflow, auth failures, and aborts are never retried.
func IsStreamIdleError ¶ added in v0.95.0
IsStreamIdleError reports whether an error was caused by a stalled LLM response body read.
func LLMGenerate ¶
LLMGenerate performs a single-shot completion using BaseTemplate and the given model.
func NewModel ¶
NewModel creates a langchaingo model from flowbot model configuration. Provider reasoning is enabled per request in StreamAssistant via ReasoningCallOptions because langchaingo exposes extended thinking through GenerateContent call options.
func ProviderForModel ¶ added in v0.94.0
ProviderForModel returns the configured provider name for a model.
func ReasoningCallOptions ¶ added in v0.94.0
func ReasoningCallOptions(modelName string, maxTokens int) []llms.CallOption
ReasoningCallOptions returns per-request call options that enable provider reasoning streams. Langchaingo applies extended thinking through GenerateContent options rather than model construction.
func ResetModelPoolForTest ¶
func ResetModelPoolForTest()
ResetModelPoolForTest clears cached langchaingo models.
func SetModelCreatorForTest ¶
func SetModelCreatorForTest(fn modelCreatorFn)
SetModelCreatorForTest overrides model construction used by GetOrCreateModel.
func StreamIdleTransportForTest ¶ added in v0.95.0
func StreamIdleTransportForTest(base http.RoundTripper) http.RoundTripper
StreamIdleTransportForTest exposes the stream idle transport for tests.
func SupportsReasoningStream ¶ added in v0.94.0
SupportsReasoningStream reports whether a model should use reasoning stream callbacks.
Types ¶
type AssistantResult ¶
type AssistantResult struct {
Content string
ToolCalls []llms.ToolCall
ModelName string
StopReason string
Usage *Usage
}
AssistantResult is the normalized output of a streaming assistant request.
func StreamAssistant ¶
func StreamAssistant( ctx context.Context, model llms.Model, systemPrompt string, messages []llms.MessageContent, opts StreamOptions, ) (AssistantResult, error)
StreamAssistant performs a streaming LLM call and assembles the assistant result. Transient failures are retried only before any streaming delta is delivered.
type ChatTemplate ¶
type ChatTemplate interface {
Format(ctx context.Context, data map[string]any) ([]llms.MessageContent, error)
}
ChatTemplate formats prompt data into langchaingo messages.
func BaseTemplate ¶
func BaseTemplate() ChatTemplate
BaseTemplate returns a minimal template without chat history for single-shot tasks.
func DefaultMultiChatTemplate ¶
func DefaultMultiChatTemplate() ChatTemplate
DefaultMultiChatTemplate returns the same template as DefaultTemplate for multi-turn chat.
func DefaultTemplate ¶
func DefaultTemplate() ChatTemplate
DefaultTemplate returns a chat template with language-aware system prompt and history support.
type FakeModel ¶
type FakeModel struct {
// contains filtered or unexported fields
}
FakeModel implements llms.Model with a queue of scripted responses.
func NewFakeModel ¶
func NewFakeModel(responses ...ResponseScript) *FakeModel
NewFakeModel creates a fake model with the given response sequence.
func (*FakeModel) Call ¶
func (f *FakeModel) Call(ctx context.Context, prompt string, options ...llms.CallOption) (string, error)
Call implements the deprecated llms.Model Call method.
func (*FakeModel) GenerateContent ¶
func (f *FakeModel) GenerateContent(ctx context.Context, _ []llms.MessageContent, options ...llms.CallOption) (*llms.ContentResponse, error)
GenerateContent returns the next scripted response.
type ResponseScript ¶
type ResponseScript struct {
Content string
// Chunks, when non-empty, are emitted one-by-one via StreamingFunc instead of Content as a single chunk.
Chunks []string
// ReasoningChunks are emitted via StreamingReasoningFunc when configured.
ReasoningChunks []string
ToolCalls []llms.ToolCall
Err error
// ErrAfterStream is returned after streaming deltas have been delivered.
ErrAfterStream error
}
ResponseScript describes one scripted model response for tests.
type RetryConfig ¶ added in v0.95.0
type RetryConfig struct {
// MaxAttempts is the total number of execution attempts. Zero uses DefaultRetryConfig.
MaxAttempts int
// InitialInterval is the delay before the first retry. Zero uses DefaultRetryConfig.
InitialInterval time.Duration
// MaxInterval caps the delay between retries. Zero uses DefaultRetryConfig.
MaxInterval time.Duration
// Multiplier controls delay growth. Zero uses DefaultRetryConfig.
Multiplier float64
// OnRetry is called before each retry attempt. May be nil.
OnRetry func(attempt int, delay time.Duration, err error)
}
RetryConfig controls transient LLM call retries.
func DefaultRetryConfig ¶ added in v0.95.0
func DefaultRetryConfig() RetryConfig
DefaultRetryConfig returns the default LLM retry policy.
func RetryConfigFromChatAgent ¶ added in v0.95.0
func RetryConfigFromChatAgent(cfg config.LLMRetryConfig) RetryConfig
RetryConfigFromChatAgent builds RetryConfig from chat agent settings.
type StreamOptions ¶
type StreamOptions struct {
ModelName string
Temperature float64
MaxTokens int
Tools []llms.Tool
OnTextDelta func(delta string) error
OnReasoningDelta func(delta string) error
// Retry overrides the default transient retry policy when non-zero MaxAttempts is set.
Retry RetryConfig
}
StreamOptions configures a streaming assistant request.