models

package
v1.13.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 12, 2026 License: Apache-2.0 Imports: 27 Imported by: 2

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrToolCallingUnsupported = errors.New("native tool calling unsupported")

ErrToolCallingUnsupported lets wrappers preserve the optional capability without forcing callers to abandon the prompt-based fallback.

Functions

This section is empty.

Types

type Agent 

type Agent interface {
	Generate(context.Context, string) (any, error)
	GenerateWithFiles(context.Context, string, []File) (any, error)

	// GenerateStream returns a channel that yields incremental text chunks.
	// The final chunk has Done=true and FullText set to the complete response.
	// If the provider doesn't support streaming natively, it falls back to
	// a single-chunk response wrapping Generate.
	GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)
}

func NewGeminiLLM 

func NewGeminiLLM(ctx context.Context, model string, promptPrefix string) (Agent, error)

func NewLLMProvider 

func NewLLMProvider(ctx context.Context, provider string, model string, promptPrefix string) (Agent, error)

NewLLMProvider returns a concrete Agent.

func TryCreateCachedLLM added in v1.10.5 

func TryCreateCachedLLM(agent Agent) Agent

TryCreateCachedLLM checks env vars and wraps the agent if caching is enabled.

type AnthropicLLM 

type AnthropicLLM struct {
	Client       *anthropic.Client
	Model        string
	MaxTokens    int
	PromptPrefix string
}

AnthropicLLM implements your Agent interface using Anthropic's Messages API.

func NewAnthropicLLM 

func NewAnthropicLLM(model, promptPrefix string) *AnthropicLLM

NewAnthropicLLM constructs a client. It reads ANTHROPIC_API_KEY from the env.

func (*AnthropicLLM) Generate 

func (a *AnthropicLLM) Generate(ctx context.Context, prompt string) (any, error)

Generate performs a single-turn completion and returns concatenated text.

func (*AnthropicLLM) GenerateStream added in v1.10.6 

func (a *AnthropicLLM) GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream uses Anthropic's streaming messages API.

func (*AnthropicLLM) GenerateWithFiles 

func (a *AnthropicLLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

type CachedLLM added in v1.10.5 

type CachedLLM struct {
	Agent    Agent
	Cache    *cache.LRUCache
	FilePath string
}

CachedLLM wraps an Agent and caches Generate calls.

func NewCachedLLM added in v1.10.5 

func NewCachedLLM(agent Agent, size int, ttl time.Duration, filePath string) *CachedLLM

NewCachedLLM creates a new CachedLLM wrapper.

func (*CachedLLM) Generate added in v1.10.5 

func (c *CachedLLM) Generate(ctx context.Context, prompt string) (any, error)

Generate checks the cache before calling the underlying agent.

func (*CachedLLM) GenerateStream added in v1.10.6 

func (c *CachedLLM) GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream passes through to the underlying agent's streaming. If the prompt is already cached, it returns a single-chunk stream from cache. Otherwise, it streams from the underlying agent and caches the full result when done.

func (*CachedLLM) GenerateWithFiles added in v1.10.5 

func (c *CachedLLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

GenerateWithFiles checks the cache (including file hashes) before calling the underlying agent.

func (*CachedLLM) GenerateWithTools added in v1.13.3 

func (c *CachedLLM) GenerateWithTools(ctx context.Context, prompt string, tools []ToolDefinition) (ToolCallResponse, error)

GenerateWithTools forwards native tool calling when the wrapped model supports it. Tool-call responses are deliberately not cached because tool execution may have side effects.

type DummyLLM 

type DummyLLM struct {
	Prefix string
}

DummyLLM is a lightweight model implementation useful for local testing without API calls.

func NewDummyLLM 

func NewDummyLLM(prefix string) *DummyLLM

func (*DummyLLM) Generate 

func (d *DummyLLM) Generate(_ context.Context, prompt string) (any, error)

func (*DummyLLM) GenerateStream added in v1.10.6 

func (d *DummyLLM) GenerateStream(_ context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream simulates streaming by splitting the response into word-level chunks.

func (*DummyLLM) GenerateWithFiles 

func (d *DummyLLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

type File 

type File struct {
	Name string
	MIME string
	Data []byte
}

File is a lightweight in-memory attachment. Name is used for display; MIME should be best-effort (e.g., "text/markdown").

type GeminiLLM 

type GeminiLLM struct {
	Client       *genai.Client
	Model        string
	PromptPrefix string
}

func (*GeminiLLM) Generate 

func (g *GeminiLLM) Generate(ctx context.Context, prompt string) (any, error)

func (*GeminiLLM) GenerateStream added in v1.10.6 

func (g *GeminiLLM) GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream uses Gemini's streaming API to yield tokens incrementally.

func (*GeminiLLM) GenerateWithFiles 

func (g *GeminiLLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

type OllamaLLM 

type OllamaLLM struct {
	Client       *ollama.Client
	Model        string
	PromptPrefix string
	// contains filtered or unexported fields
}

func NewOllamaLLM 

func NewOllamaLLM(model string, promptPrefix string) (*OllamaLLM, error)

func (*OllamaLLM) Generate 

func (o *OllamaLLM) Generate(ctx context.Context, prompt string) (any, error)

func (*OllamaLLM) GenerateStream added in v1.10.6 

func (o *OllamaLLM) GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream leverages Ollama's native callback-based streaming.

func (*OllamaLLM) GenerateWithFiles 

func (o *OllamaLLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

func (*OllamaLLM) WebSearch added in v0.6.5 

func (o *OllamaLLM) WebSearch(ctx context.Context, query string, limit int) ([]map[string]string, error)

WebSearch queries the Ollama Web Search API and returns top results.

type OpenAILLM 

type OpenAILLM struct {
	Client       *openai.Client
	Model        string
	PromptPrefix string
}

func NewOpenAILLM 

func NewOpenAILLM(model string, promptPrefix string) *OpenAILLM

func (*OpenAILLM) Generate 

func (o *OpenAILLM) Generate(ctx context.Context, prompt string) (any, error)

func (*OpenAILLM) GenerateStream added in v1.10.6 

func (o *OpenAILLM) GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream uses OpenAI's streaming chat completion API.

func (*OpenAILLM) GenerateWithFiles 

func (o *OpenAILLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

func (*OpenAILLM) GenerateWithTools added in v1.13.3 

func (o *OpenAILLM) GenerateWithTools(ctx context.Context, prompt string, definitions []ToolDefinition) (ToolCallResponse, error)

GenerateWithTools uses OpenAI's native function-calling API. The generic Agent interface remains unchanged; callers can opt into this capability via models.ToolCallingAgent.

type OpenRouterLLM added in v1.12.0 

type OpenRouterLLM struct {
	Client       *openrouter.OpenRouter
	Model        string
	PromptPrefix string
}

func NewOpenRouterLLM added in v1.12.0 

func NewOpenRouterLLM(model string, promptPrefix string) *OpenRouterLLM

func (*OpenRouterLLM) Generate added in v1.12.0 

func (o *OpenRouterLLM) Generate(ctx context.Context, prompt string) (any, error)

func (*OpenRouterLLM) GenerateStream added in v1.12.0 

func (o *OpenRouterLLM) GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream uses OpenRouter's streaming chat completion API.

func (*OpenRouterLLM) GenerateWithFiles added in v1.12.0 

func (o *OpenRouterLLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

type StreamChunk added in v1.10.6 

type StreamChunk struct {
	Delta    string // incremental text token
	Done     bool   // true on the final chunk
	FullText string // aggregated text (populated only on the final chunk)
	Err      error  // non-nil if the stream encountered a fatal error
}

StreamChunk represents a single piece of a streaming LLM response. When Done is true, the stream is complete and FullText holds the aggregated output. When Err is non-nil, the stream encountered an error.

type ToolCall added in v1.13.3 

type ToolCall struct {
	ID        string         `json:"id,omitempty"`
	Name      string         `json:"name"`
	Arguments map[string]any `json:"arguments"`
}

ToolCall is a provider-neutral tool invocation selected by a model.

type ToolCallResponse added in v1.13.3 

type ToolCallResponse struct {
	Content   string     `json:"content,omitempty"`
	ToolCalls []ToolCall `json:"tool_calls,omitempty"`
}

ToolCallResponse contains either assistant text, one or more native tool calls, or both depending on the provider response.

type ToolCallingAgent added in v1.13.3 

type ToolCallingAgent interface {
	GenerateWithTools(ctx context.Context, prompt string, tools []ToolDefinition) (ToolCallResponse, error)
}

ToolCallingAgent is an optional capability. It intentionally sits beside Agent so existing custom model implementations remain source-compatible. Agents that do not implement it continue through the prompt-based fallback.

type ToolDefinition added in v1.13.3 

type ToolDefinition struct {
	Name        string         `json:"name"`
	Description string         `json:"description,omitempty"`
	InputSchema map[string]any `json:"input_schema"`
}

ToolDefinition is the provider-neutral description of a callable tool. Providers adapt this shape to their native tool/function-calling API.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.