llm

package
v0.8.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 20, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package llm provides LLM provider abstractions for DraftCrew.

Supported providers: OpenAI, Anthropic, Google Gemini, and any OpenAI-compatible endpoint. Each provider implements the Client interface defined in this package.

Index

Constants

This section is empty.

Variables

View Source 
var ErrStreamNotSupported = errors.New("streaming not supported for this provider")

Functions

This section is empty.

Types

type AnthropicProvider 

type AnthropicProvider struct {
	// contains filtered or unexported fields
}

AnthropicProvider implements the LLM Client interface for Anthropic.

func NewAnthropic 

func NewAnthropic(apiKey, model string, opts ...GenerateOption) *AnthropicProvider

NewAnthropic creates a new Anthropic provider with the given API key and model.

func (*AnthropicProvider) Generate 

func (p *AnthropicProvider) Generate(ctx context.Context, msgs []Message, opts ...GenerateOption) (Response, error)

Generate sends a messages request to Anthropic and returns the response.

func (*AnthropicProvider) Stream 

func (p *AnthropicProvider) Stream(ctx context.Context, msgs []Message, opts ...GenerateOption) (<-chan StreamChunk, error)

Stream sends a streaming messages request to Anthropic.

type BedrockProvider added in v0.6.0 

type BedrockProvider struct {
	// contains filtered or unexported fields
}

func NewBedrock added in v0.6.0 

func NewBedrock(region, model string, opts ...GenerateOption) *BedrockProvider

NewBedrock creates a provider for AWS Bedrock with SigV4-signed requests.

func (*BedrockProvider) Close added in v0.6.0 

func (p *BedrockProvider) Close() error

func (*BedrockProvider) Generate added in v0.6.0 

func (p *BedrockProvider) Generate(ctx context.Context, msgs []Message, opts ...GenerateOption) (Response, error)

func (*BedrockProvider) Stream added in v0.6.0 

func (p *BedrockProvider) Stream(ctx context.Context, msgs []Message, opts ...GenerateOption) (<-chan StreamChunk, error)

type Client 

type Client interface {
	Generate(ctx context.Context, msgs []Message, opts ...GenerateOption) (Response, error)
	Stream(ctx context.Context, msgs []Message, opts ...GenerateOption) (<-chan StreamChunk, error)
}

Client is the interface for LLM providers supporting both generate and stream.

type CompatibleProvider 

type CompatibleProvider struct {
	// contains filtered or unexported fields
}

CompatibleProvider implements the LLM Client interface for OpenAI-compatible APIs.

func NewAzureOpenAI 

func NewAzureOpenAI(endpoint, apiKey, deploymentName string, opts ...GenerateOption) *CompatibleProvider

NewAzureOpenAI creates a new provider for Azure OpenAI Service.

func NewDeepSeek 

func NewDeepSeek(apiKey, model string, opts ...GenerateOption) *CompatibleProvider

NewDeepSeek creates a new provider for DeepSeek's API.

func NewGroq added in v0.6.0 

func NewGroq(apiKey, model string, opts ...GenerateOption) *CompatibleProvider

NewGroq creates a provider for Groq's fast-inference API. Default model: llama-3.3-70b-versatile.

func NewMistral 

func NewMistral(apiKey, model string, opts ...GenerateOption) *CompatibleProvider

NewMistral creates a new provider for Mistral's API.

func NewOllama 

func NewOllama(model string, opts ...GenerateOption) *CompatibleProvider

NewOllama creates a new provider for a local Ollama instance (http://localhost:11434/v1).

func NewOllamaWithURL added in v0.6.0 

func NewOllamaWithURL(baseURL, model string, opts ...GenerateOption) *CompatibleProvider

NewOllamaWithURL creates a new provider for Ollama at the given base URL. Useful when Ollama runs on a remote host or non-default port. Example: NewOllamaWithURL("http://192.168.1.100:11434/v1", "llama3")

func NewOpenAICompatible 

func NewOpenAICompatible(baseURL, apiKey, model string, opts ...GenerateOption) *CompatibleProvider

NewOpenAICompatible creates a new provider for any OpenAI-compatible API endpoint.

func NewOpenRouter 

func NewOpenRouter(apiKey, model string, opts ...GenerateOption) *CompatibleProvider

NewOpenRouter creates a new provider for OpenRouter's API.

func NewTogetherAI added in v0.6.0 

func NewTogetherAI(apiKey, model string, opts ...GenerateOption) *CompatibleProvider

NewTogetherAI creates a provider for Together AI's API. Default model: mistralai/Mixtral-8x22B-Instruct-v0.1.

func (*CompatibleProvider) Generate 

func (p *CompatibleProvider) Generate(ctx context.Context, msgs []Message, opts ...GenerateOption) (Response, error)

Generate sends a chat completion request to the compatible API and returns the response.

func (*CompatibleProvider) Stream 

func (p *CompatibleProvider) Stream(ctx context.Context, msgs []Message, opts ...GenerateOption) (<-chan StreamChunk, error)

Stream sends a streaming chat completion request to the compatible API.

type GeminiProvider 

type GeminiProvider struct {
	// contains filtered or unexported fields
}

GeminiProvider implements the LLM Client interface for Google Gemini.

func NewGemini 

func NewGemini(apiKey, model string, opts ...GenerateOption) *GeminiProvider

NewGemini creates a new Gemini provider with the given API key and model.

func (*GeminiProvider) Generate 

func (p *GeminiProvider) Generate(ctx context.Context, msgs []Message, opts ...GenerateOption) (Response, error)

Generate sends a content generation request to Gemini and returns the response.

func (*GeminiProvider) Stream 

func (p *GeminiProvider) Stream(ctx context.Context, msgs []Message, opts ...GenerateOption) (<-chan StreamChunk, error)

Stream sends a streaming content generation request to Gemini.

type GenerateOption 

type GenerateOption func(*GenerateOptions)

GenerateOption configures a generation request.

func WithMaxTokens 

func WithMaxTokens(n int) GenerateOption

WithMaxTokens sets the maximum number of tokens to generate.

func WithSchema 

func WithSchema(schema any) GenerateOption

WithSchema sets a JSON schema for structured output.

func WithStop 

func WithStop(stop []string) GenerateOption

WithStop sets stop sequences for generation.

func WithTemperature 

func WithTemperature(t float32) GenerateOption

WithTemperature sets the sampling temperature for generation.

func WithToolChoice added in v0.4.0 

func WithToolChoice(tc string) GenerateOption

WithToolChoice sets the tool choice policy ("auto", "required", "none", or a specific tool name).

func WithTools 

func WithTools(tools []ToolDef) GenerateOption

WithTools sets the available tools for function calling.

func WithTopP 

func WithTopP(p float32) GenerateOption

WithTopP sets the nucleus sampling top-p value.

type GenerateOptions 

type GenerateOptions struct {
	Temperature float32
	MaxTokens   int
	TopP        float32
	Stop        []string
	Schema      any
	Tools       []ToolDef
	ToolChoice  string
}

GenerateOptions holds the configuration for a generation request.

type Message 

type Message struct {
	Role      Role       `json:"role"`
	Content   string     `json:"content"`
	Name      string     `json:"name,omitempty"`
	ToolID    string     `json:"tool_call_id,omitempty"`
	ToolCalls []ToolCall `json:"tool_calls,omitempty"`
}

Message represents a single message in a conversation with role and content.

type OpenAIProvider 

type OpenAIProvider struct {
	// contains filtered or unexported fields
}

OpenAIProvider implements the LLM Client interface for OpenAI.

func NewOpenAI 

func NewOpenAI(apiKey, model string, opts ...GenerateOption) *OpenAIProvider

NewOpenAI creates a new OpenAI provider with the given API key and model.

func (*OpenAIProvider) Generate 

func (p *OpenAIProvider) Generate(ctx context.Context, msgs []Message, opts ...GenerateOption) (Response, error)

Generate sends a chat completion request to OpenAI and returns the response.

func (*OpenAIProvider) Stream 

func (p *OpenAIProvider) Stream(ctx context.Context, msgs []Message, opts ...GenerateOption) (<-chan StreamChunk, error)

Stream sends a streaming chat completion request to OpenAI.

type Response 

type Response struct {
	Content   string
	ToolCalls []ToolCall
	Usage     Usage
	Raw       string
}

Response contains the full result of an LLM generation request.

type Role 

type Role string

Role represents a message role in a conversation. 

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

Standard message roles.

type StreamChunk 

type StreamChunk struct {
	Content  string
	ToolCall *ToolCall
	Done     bool
	Error    error
}

StreamChunk represents a single chunk from a streaming LLM response.

type ToolCall 

type ToolCall struct {
	ID       string `json:"id"`
	Type     string `json:"type"`
	FuncName string `json:"function_name"`
	FuncArgs string `json:"function_args"`
}

ToolCall represents a function tool invocation requested by the LLM.

type ToolDef 

type ToolDef struct {
	Name        string `json:"name"`
	Description string `json:"description"`
	Parameters  any    `json:"parameters"`
}

ToolDef describes a tool available to the LLM for function calling.

type Usage 

type Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
}

Usage contains token usage statistics from an LLM response.

type VertexAIProvider added in v0.6.0 

type VertexAIProvider struct {
	// contains filtered or unexported fields
}

func NewVertexAI added in v0.6.0 

func NewVertexAI(projectID, location, model string, opts ...GenerateOption) *VertexAIProvider

func (*VertexAIProvider) Close added in v0.6.0 

func (p *VertexAIProvider) Close() error

func (*VertexAIProvider) Generate added in v0.6.0 

func (p *VertexAIProvider) Generate(ctx context.Context, msgs []Message, opts ...GenerateOption) (Response, error)

func (*VertexAIProvider) Stream added in v0.6.0 

func (p *VertexAIProvider) Stream(ctx context.Context, msgs []Message, opts ...GenerateOption) (<-chan StreamChunk, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.