models

package
v1.13.9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 13, 2026 License: Apache-2.0 Imports: 28 Imported by: 2

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrToolCallingUnsupported = errors.New("native tool calling unsupported")

ErrToolCallingUnsupported lets wrappers preserve the optional capability without forcing callers to abandon the prompt-based fallback.

Functions

This section is empty.

Types

type Agent 

type Agent interface {
	Generate(context.Context, string) (any, error)
	GenerateWithFiles(context.Context, string, []File) (any, error)

	// GenerateStream returns a channel that yields incremental text chunks.
	// The final chunk has Done=true and FullText set to the complete response.
	// If the provider doesn't support streaming natively, it falls back to
	// a single-chunk response wrapping Generate.
	GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)
}

func NewGeminiLLM 

func NewGeminiLLM(ctx context.Context, model string, promptPrefix string) (Agent, error)

func NewLLMProvider 

func NewLLMProvider(ctx context.Context, provider string, model string, promptPrefix string) (Agent, error)

NewLLMProvider returns a concrete Agent.

func NewVertexLLM added in v1.13.9 

func NewVertexLLM(
	ctx context.Context,
	model string,
	promptPrefix string,
	project string,
	location string,
) (Agent, error)

NewVertexLLM creates a Vertex AI model client for project and location. Credentials are intentionally omitted so the Google GenAI SDK uses Application Default Credentials.

func TryCreateCachedLLM added in v1.10.5 

func TryCreateCachedLLM(agent Agent) Agent

TryCreateCachedLLM checks env vars and wraps the agent if caching is enabled.

type AnthropicLLM 

type AnthropicLLM struct {
	Client       *anthropic.Client
	Model        string
	MaxTokens    int
	PromptPrefix string
}

AnthropicLLM implements your Agent interface using Anthropic's Messages API.

func NewAnthropicLLM 

func NewAnthropicLLM(model, promptPrefix string) *AnthropicLLM

NewAnthropicLLM constructs a client. It reads ANTHROPIC_API_KEY from the env.

func (*AnthropicLLM) Generate 

func (a *AnthropicLLM) Generate(ctx context.Context, prompt string) (any, error)

Generate performs a single-turn completion and returns concatenated text.

func (*AnthropicLLM) GenerateStream added in v1.10.6 

func (a *AnthropicLLM) GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream uses Anthropic's streaming messages API.

func (*AnthropicLLM) GenerateWithFiles 

func (a *AnthropicLLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

type CachedLLM added in v1.10.5 

type CachedLLM struct {
	Agent    Agent
	Cache    *cache.LRUCache
	FilePath string
}

CachedLLM wraps an Agent and caches Generate calls.

func NewCachedLLM added in v1.10.5 

func NewCachedLLM(agent Agent, size int, ttl time.Duration, filePath string) *CachedLLM

NewCachedLLM creates a new CachedLLM wrapper.

func (*CachedLLM) Generate added in v1.10.5 

func (c *CachedLLM) Generate(ctx context.Context, prompt string) (any, error)

Generate checks the cache before calling the underlying agent.

func (*CachedLLM) GenerateStream added in v1.10.6 

func (c *CachedLLM) GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream passes through to the underlying agent's streaming. If the prompt is already cached, it returns a single-chunk stream from cache. Otherwise, it streams from the underlying agent and caches the full result when done.

func (*CachedLLM) GenerateWithFiles added in v1.10.5 

func (c *CachedLLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

GenerateWithFiles checks the cache (including file hashes) before calling the underlying agent.

func (*CachedLLM) GenerateWithTools added in v1.13.3 

func (c *CachedLLM) GenerateWithTools(ctx context.Context, prompt string, tools []ToolDefinition) (ToolCallResponse, error)

GenerateWithTools forwards native tool calling when the wrapped model supports it. Tool-call responses are deliberately not cached because tool execution may have side effects.

type DummyLLM 

type DummyLLM struct {
	Prefix string
}

DummyLLM is a lightweight model implementation useful for local testing without API calls.

func NewDummyLLM 

func NewDummyLLM(prefix string) *DummyLLM

func (*DummyLLM) Generate 

func (d *DummyLLM) Generate(_ context.Context, prompt string) (any, error)

func (*DummyLLM) GenerateStream added in v1.10.6 

func (d *DummyLLM) GenerateStream(_ context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream simulates streaming by splitting the response into word-level chunks.

func (*DummyLLM) GenerateWithFiles 

func (d *DummyLLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

type File 

type File struct {
	Name string
	MIME string
	Data []byte
}

File is a lightweight in-memory attachment. Name is used for display; MIME should be best-effort (e.g., "text/markdown").

type GeminiLLM 

type GeminiLLM struct {
	Client       *genai.Client
	Model        string
	PromptPrefix string
}

func (*GeminiLLM) Generate 

func (g *GeminiLLM) Generate(ctx context.Context, prompt string) (any, error)

func (*GeminiLLM) GenerateStream added in v1.10.6 

func (g *GeminiLLM) GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream uses Gemini's streaming API to yield tokens incrementally.

func (*GeminiLLM) GenerateWithFiles 

func (g *GeminiLLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

type OllamaLLM 

type OllamaLLM struct {
	Client       *ollama.Client
	Model        string
	PromptPrefix string
	// contains filtered or unexported fields
}

func NewOllamaLLM 

func NewOllamaLLM(model string, promptPrefix string) (*OllamaLLM, error)

func (*OllamaLLM) Generate 

func (o *OllamaLLM) Generate(ctx context.Context, prompt string) (any, error)

func (*OllamaLLM) GenerateStream added in v1.10.6 

func (o *OllamaLLM) GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream leverages Ollama's native callback-based streaming.

func (*OllamaLLM) GenerateWithFiles 

func (o *OllamaLLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

func (*OllamaLLM) WebSearch added in v0.6.5 

func (o *OllamaLLM) WebSearch(ctx context.Context, query string, limit int) ([]map[string]string, error)

WebSearch queries the Ollama Web Search API and returns top results.

type OpenAILLM 

type OpenAILLM struct {
	Client       *openai.Client
	Model        string
	PromptPrefix string
}

func NewOpenAILLM 

func NewOpenAILLM(model string, promptPrefix string) *OpenAILLM

func (*OpenAILLM) Generate 

func (o *OpenAILLM) Generate(ctx context.Context, prompt string) (any, error)

func (*OpenAILLM) GenerateStream added in v1.10.6 

func (o *OpenAILLM) GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream uses OpenAI's streaming chat completion API.

func (*OpenAILLM) GenerateWithFiles 

func (o *OpenAILLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

func (*OpenAILLM) GenerateWithTools added in v1.13.3 

func (o *OpenAILLM) GenerateWithTools(ctx context.Context, prompt string, definitions []ToolDefinition) (ToolCallResponse, error)

GenerateWithTools uses OpenAI's native function-calling API. The generic Agent interface remains unchanged; callers can opt into this capability via models.ToolCallingAgent.

type OpenRouterLLM added in v1.12.0 

type OpenRouterLLM struct {
	Client       *openrouter.OpenRouter
	Model        string
	PromptPrefix string
}

func NewOpenRouterLLM added in v1.12.0 

func NewOpenRouterLLM(model string, promptPrefix string) *OpenRouterLLM

func (*OpenRouterLLM) Generate added in v1.12.0 

func (o *OpenRouterLLM) Generate(ctx context.Context, prompt string) (any, error)

func (*OpenRouterLLM) GenerateStream added in v1.12.0 

func (o *OpenRouterLLM) GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)

GenerateStream uses OpenRouter's streaming chat completion API.

func (*OpenRouterLLM) GenerateWithFiles added in v1.12.0 

func (o *OpenRouterLLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

type StreamChunk added in v1.10.6 

type StreamChunk struct {
	Delta    string // incremental text token
	Done     bool   // true on the final chunk
	FullText string // aggregated text (populated only on the final chunk)
	Err      error  // non-nil if the stream encountered a fatal error
}

StreamChunk represents a single piece of a streaming LLM response. When Done is true, the stream is complete and FullText holds the aggregated output. When Err is non-nil, the stream encountered an error.

type ToolCall added in v1.13.3 

type ToolCall struct {
	ID        string         `json:"id,omitempty"`
	Name      string         `json:"name"`
	Arguments map[string]any `json:"arguments"`
}

ToolCall is a provider-neutral tool invocation selected by a model.

type ToolCallResponse added in v1.13.3 

type ToolCallResponse struct {
	Content   string     `json:"content,omitempty"`
	ToolCalls []ToolCall `json:"tool_calls,omitempty"`
}

ToolCallResponse contains either assistant text, one or more native tool calls, or both depending on the provider response.

type ToolCallingAgent added in v1.13.3 

type ToolCallingAgent interface {
	GenerateWithTools(ctx context.Context, prompt string, tools []ToolDefinition) (ToolCallResponse, error)
}

ToolCallingAgent is an optional capability. It intentionally sits beside Agent so existing custom model implementations remain source-compatible. Agents that do not implement it continue through the prompt-based fallback.

type ToolDefinition added in v1.13.3 

type ToolDefinition struct {
	Name        string         `json:"name"`
	Description string         `json:"description,omitempty"`
	InputSchema map[string]any `json:"input_schema"`
}

ToolDefinition is the provider-neutral description of a callable tool. Providers adapt this shape to their native tool/function-calling API.

type VertexLLM added in v1.13.9 

type VertexLLM struct {
	Client       *genai.Client
	Model        string
	PromptPrefix string
}

VertexLLM uses Gemini models through Vertex AI. Its client authenticates with Application Default Credentials.

func (*VertexLLM) Generate added in v1.13.9 

func (v *VertexLLM) Generate(ctx context.Context, prompt string) (any, error)

func (*VertexLLM) GenerateStream added in v1.13.9 

func (v *VertexLLM) GenerateStream(ctx context.Context, prompt string) (<-chan StreamChunk, error)

func (*VertexLLM) GenerateWithFiles added in v1.13.9 

func (v *VertexLLM) GenerateWithFiles(ctx context.Context, prompt string, files []File) (any, error)

Source Files

View all Source files

Directories

Path Synopsis
Package middleware provides composable policies for model calls.
 Package middleware provides composable policies for model calls.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.