llm

package
v0.97.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 20, 2026 License: GPL-3.0 Imports: 21 Imported by: 0

Documentation

Overview

Package llm adapts langchaingo models for the agent loop and single-shot module tasks.

Index

Constants

View Source
const (
	AgentChat              = "chat"
	AgentBillClassify      = "bill_classify"
	AgentExtractTags       = "extract_tags"
	AgentSimilarTags       = "similar_tags"
	AgentNewsSummary       = "news_summary"
	AgentRepoReviewComment = "repo_review_comment"
)

Agent name constants identify logical agent roles for LLM helpers.

View Source
const (
	ProviderOpenAI           = "openai"
	ProviderOpenAICompatible = "openai_compatible"
	ProviderGemini           = "gemini"
	ProviderAnthropic        = "anthropic"
)

Model provider constants match config.models provider values.

Variables

View Source
var ErrAborted = errors.New("agent llm: aborted")

ErrAborted indicates the LLM call was cancelled.

View Source
var ErrStreamIdle = errors.New("stream idle timeout")

ErrStreamIdle indicates the LLM stream stopped delivering deltas for longer than the idle limit.

View Source
var ErrStreamStarted = errors.New("agent llm: stream already started")

ErrStreamStarted indicates a retryable transport error occurred after streaming deltas were already delivered to the client; callers must not retry.

Functions

func AgentEnabled

func AgentEnabled(name string) bool

AgentEnabled reports whether a named agent is configured with a model and enabled.

func AgentModelName

func AgentModelName(name string) string

AgentModelName returns the configured model for a named agent when enabled.

func Complete

func Complete(
	ctx context.Context,
	model llms.Model,
	systemPrompt string,
	messages []llms.MessageContent,
	modelName string,
	maxTokens int,
) (string, error)

Complete performs a non-streaming completion for auxiliary tasks such as summarization. Retry policy follows chat_agent.llm_retry when configured.

func CompleteWithRetry added in v0.95.0

func CompleteWithRetry(
	ctx context.Context,
	model llms.Model,
	systemPrompt string,
	messages []llms.MessageContent,
	modelName string,
	maxTokens int,
	retryCfg RetryConfig,
) (string, error)

CompleteWithRetry performs a non-streaming completion with an explicit retry policy.

func DeepSeekThinkingHTTPClientForTest added in v0.94.0

func DeepSeekThinkingHTTPClientForTest(base http.RoundTripper) *http.Client

DeepSeekThinkingHTTPClientForTest exposes the DeepSeek thinking HTTP client for tests.

func GenerateWithTemplate

func GenerateWithTemplate(ctx context.Context, modelName string, template ChatTemplate, data map[string]any) (string, error)

GenerateWithTemplate performs a single-shot completion using the given prompt template.

func GetOrCreateModel

func GetOrCreateModel(ctx context.Context, modelName string) (llms.Model, string, error)

GetOrCreateModel returns a cached langchaingo model for the given model name.

func IdleTimeoutReadCloserForTest added in v0.95.0

func IdleTimeoutReadCloserForTest(rc io.ReadCloser, idleTimeout time.Duration) io.ReadCloser

IdleTimeoutReadCloserForTest exposes the idle timeout reader for tests.

func IsRetryableLLMError added in v0.95.0

func IsRetryableLLMError(err error) bool

IsRetryableLLMError reports whether an LLM error should be retried. Overflow, auth failures, and aborts are never retried.

func IsStreamIdleError added in v0.95.0

func IsStreamIdleError(err error) bool

IsStreamIdleError reports whether an error was caused by a stalled LLM response body read.

func LLMGenerate

func LLMGenerate(ctx context.Context, modelName, prompt string) (string, error)

LLMGenerate performs a single-shot completion using BaseTemplate and the given model.

func NewModel

func NewModel(ctx context.Context, modelName string) (llms.Model, string, error)

NewModel creates a langchaingo model from flowbot model configuration. Provider reasoning is enabled per request in StreamAssistant via ReasoningCallOptions because langchaingo exposes extended thinking through GenerateContent call options.

func ProviderForModel added in v0.94.0

func ProviderForModel(modelName string) string

ProviderForModel returns the configured provider name for a model.

func ReasoningCallOptions added in v0.94.0

func ReasoningCallOptions(modelName string, maxTokens int) []llms.CallOption

ReasoningCallOptions returns per-request call options that enable provider reasoning streams. Langchaingo applies extended thinking through GenerateContent options rather than model construction.

func ResetModelPoolForTest

func ResetModelPoolForTest()

ResetModelPoolForTest clears cached langchaingo models.

func SetModelCreatorForTest

func SetModelCreatorForTest(fn modelCreatorFn)

SetModelCreatorForTest overrides model construction used by GetOrCreateModel.

func StreamIdleTransportForTest added in v0.95.0

func StreamIdleTransportForTest(base http.RoundTripper) http.RoundTripper

StreamIdleTransportForTest exposes the stream idle transport for tests.

func SupportsReasoningStream added in v0.94.0

func SupportsReasoningStream(modelName string) bool

SupportsReasoningStream reports whether a model should use reasoning stream callbacks.

Types

type AssistantResult

type AssistantResult struct {
	Content    string
	ToolCalls  []llms.ToolCall
	ModelName  string
	StopReason string
	Usage      *Usage
}

AssistantResult is the normalized output of a streaming assistant request.

func StreamAssistant

func StreamAssistant(
	ctx context.Context,
	model llms.Model,
	systemPrompt string,
	messages []llms.MessageContent,
	opts StreamOptions,
) (AssistantResult, error)

StreamAssistant performs a streaming LLM call and assembles the assistant result. Transient failures are retried only before any streaming delta is delivered.

type ChatTemplate

type ChatTemplate interface {
	Format(ctx context.Context, data map[string]any) ([]llms.MessageContent, error)
}

ChatTemplate formats prompt data into langchaingo messages.

func BaseTemplate

func BaseTemplate() ChatTemplate

BaseTemplate returns a minimal template without chat history for single-shot tasks.

func DefaultMultiChatTemplate

func DefaultMultiChatTemplate() ChatTemplate

DefaultMultiChatTemplate returns the same template as DefaultTemplate for multi-turn chat.

func DefaultTemplate

func DefaultTemplate() ChatTemplate

DefaultTemplate returns a chat template with language-aware system prompt and history support.

type FakeModel

type FakeModel struct {
	// contains filtered or unexported fields
}

FakeModel implements llms.Model with a queue of scripted responses.

func NewFakeModel

func NewFakeModel(responses ...ResponseScript) *FakeModel

NewFakeModel creates a fake model with the given response sequence.

func (*FakeModel) Call

func (f *FakeModel) Call(ctx context.Context, prompt string, options ...llms.CallOption) (string, error)

Call implements the deprecated llms.Model Call method.

func (*FakeModel) Calls

func (f *FakeModel) Calls() int

Calls returns how many GenerateContent invocations have occurred.

func (*FakeModel) GenerateContent

func (f *FakeModel) GenerateContent(ctx context.Context, _ []llms.MessageContent, options ...llms.CallOption) (*llms.ContentResponse, error)

GenerateContent returns the next scripted response.

type ResponseScript

type ResponseScript struct {
	Content string
	// Chunks, when non-empty, are emitted one-by-one via StreamingFunc instead of Content as a single chunk.
	Chunks []string
	// ReasoningChunks are emitted via StreamingReasoningFunc when configured.
	ReasoningChunks []string
	ToolCalls       []llms.ToolCall
	Err             error
	// ErrAfterStream is returned after streaming deltas have been delivered.
	ErrAfterStream error
}

ResponseScript describes one scripted model response for tests.

type RetryConfig added in v0.95.0

type RetryConfig struct {
	// MaxAttempts is the total number of execution attempts. Zero uses DefaultRetryConfig.
	MaxAttempts int
	// InitialInterval is the delay before the first retry. Zero uses DefaultRetryConfig.
	InitialInterval time.Duration
	// MaxInterval caps the delay between retries. Zero uses DefaultRetryConfig.
	MaxInterval time.Duration
	// Multiplier controls delay growth. Zero uses DefaultRetryConfig.
	Multiplier float64
	// OnRetry is called before each retry attempt. May be nil.
	OnRetry func(attempt int, delay time.Duration, err error)
}

RetryConfig controls transient LLM call retries.

func DefaultRetryConfig added in v0.95.0

func DefaultRetryConfig() RetryConfig

DefaultRetryConfig returns the default LLM retry policy.

func RetryConfigFromChatAgent added in v0.95.0

func RetryConfigFromChatAgent(cfg config.LLMRetryConfig) RetryConfig

RetryConfigFromChatAgent builds RetryConfig from chat agent settings.

type StreamOptions

type StreamOptions struct {
	ModelName        string
	Temperature      float64
	MaxTokens        int
	Tools            []llms.Tool
	OnTextDelta      func(delta string) error
	OnReasoningDelta func(delta string) error
	// Retry overrides the default transient retry policy when non-zero MaxAttempts is set.
	Retry RetryConfig
}

StreamOptions configures a streaming assistant request.

type Usage

type Usage struct {
	PromptTokens     int
	CompletionTokens int
	TotalTokens      int
	CacheRead        int
	CacheWrite       int
}

Usage captures token consumption from an LLM response.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL