provider

package
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 5, 2026 License: MIT Imports: 31 Imported by: 3

Documentation

Overview

Package provider defines the AI provider interface for agent backends.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DownloadHuggingFaceFile added in v0.5.3

func DownloadHuggingFaceFile(ctx context.Context, repo, filename, outputDir string, progress func(pct float64)) (string, error)

DownloadHuggingFaceFile downloads any file from a HuggingFace model repo. The file is saved to outputDir/<repo-slug>/<filename> where repo-slug replaces "/" with "--". If the file already exists it is returned as-is. Downloads resume from a .part temp file if interrupted. The optional progress callback receives completion percentage (0.0–1.0).

func EnsureLlamaServer added in v0.5.3

func EnsureLlamaServer(ctx context.Context) (string, error)

EnsureLlamaServer finds or downloads the llama-server binary. Search order: PATH (checked by caller) → cache dir → download from GitHub releases.

func HuggingFaceBaseURL added in v0.5.3

func HuggingFaceBaseURL() string

HuggingFaceBaseURL returns the base URL used for HuggingFace downloads.

func NewAnthropicBedrockProvider

func NewAnthropicBedrockProvider(cfg AnthropicBedrockConfig) (*anthropicBedrockProvider, error)

NewAnthropicBedrockProvider creates a provider that accesses Claude via Amazon Bedrock.

Docs: https://platform.claude.com/docs/en/build-with-claude/claude-on-amazon-bedrock

func ParseThinking added in v0.5.3

func ParseThinking(raw string) (thinking, content string)

ParseThinking extracts <think>...</think> blocks from model output. The first block's content becomes thinking; the remainder becomes content. If no <think> block is present, all text is returned as content.

func SetHuggingFaceBaseURL added in v0.5.3

func SetHuggingFaceBaseURL(url string)

SetHuggingFaceBaseURL overrides the base URL (used in tests).

func ValidateBaseURL added in v0.5.3

func ValidateBaseURL(rawURL string) error

ValidateBaseURL checks that a caller-supplied base URL is safe to contact:

  • Empty string is always allowed (callers fall back to the default URL).
  • Scheme must be "https".
  • Resolved IP must not be loopback, link-local, or RFC 1918 private.

Types

type AnthropicBedrockConfig

type AnthropicBedrockConfig struct {
	// Region is the AWS region (e.g. "us-east-1").
	Region string
	// Model is the Bedrock model ID (e.g. "anthropic.claude-sonnet-4-20250514-v1:0").
	Model string
	// MaxTokens limits the response length.
	MaxTokens int
	// AccessKeyID is the AWS access key (required).
	AccessKeyID string
	// SecretAccessKey is the AWS secret key (required).
	SecretAccessKey string
	// SessionToken is the AWS session token for temporary credentials (optional).
	SessionToken string
	// Profile is the AWS config profile name (reserved for future use).
	Profile string
	// HTTPClient is the HTTP client to use (defaults to http.DefaultClient).
	HTTPClient *http.Client
	// BaseURL overrides the endpoint (for testing).
	BaseURL string
}

AnthropicBedrockConfig configures the Anthropic provider for Amazon Bedrock. Uses AWS IAM SigV4 authentication against the Bedrock Runtime API.

type AuthModeInfo

type AuthModeInfo struct {
	Mode        string // e.g. "personal", "direct", "bedrock"
	DisplayName string // e.g. "GitHub Copilot (Personal/IDE)"
	Description string // What this mode does
	Warning     string // ToS/usage concerns (empty if none)
	DocsURL     string // Link to official documentation
	ServerSafe  bool   // Whether this mode is appropriate for server/service use
}

AuthModeInfo describes an authentication/deployment mode for a provider backend.

func AllAuthModes

func AllAuthModes() []AuthModeInfo

AllAuthModes returns metadata for all known provider authentication modes, including both implemented and scaffolded providers.

func LocalAuthMode added in v0.5.3

func LocalAuthMode(name, displayName string) AuthModeInfo

LocalAuthMode returns an AuthModeInfo for a local (no-API-key) provider.

type ChannelSource

type ChannelSource struct {
	// contains filtered or unexported fields
}

ChannelSource delivers interactions via Go channels, enabling test goroutines to drive the agent loop interactively from within a Go test.

func NewChannelSource

func NewChannelSource() (source *ChannelSource, interactionsCh <-chan Interaction, responsesCh chan<- InteractionResponse)

NewChannelSource creates a ChannelSource and returns the source along with the test-side channels:

  • interactionsCh receives Interactions from the provider (test reads from this)
  • responsesCh accepts InteractionResponses from the test (test writes to this)

func (*ChannelSource) GetResponse

func (cs *ChannelSource) GetResponse(ctx context.Context, interaction Interaction) (*InteractionResponse, error)

GetResponse implements ResponseSource. It sends the interaction on the interactions channel and blocks until a response arrives on the responses channel or the context is cancelled.

type Embedder

type Embedder interface {
	Embed(ctx context.Context, text string) ([]float32, error)
}

Embedder is optionally implemented by providers that support text embedding.

func AsEmbedder

func AsEmbedder(p Provider) (Embedder, bool)

AsEmbedder checks if a Provider also implements Embedder.

type EventBroadcaster

type EventBroadcaster interface {
	BroadcastEvent(eventType, data string)
}

EventBroadcaster is an optional interface for pushing SSE notifications when new test interactions arrive.

type HTTPSource

type HTTPSource struct {
	// contains filtered or unexported fields
}

HTTPSource exposes pending interactions via an API so that humans or QA scripts can act as the LLM. When the agent calls Chat(), the interaction is stored as pending and an event is broadcast. A subsequent API call provides the response, unblocking the waiting goroutine.

func NewHTTPSource

func NewHTTPSource(broadcaster EventBroadcaster) *HTTPSource

NewHTTPSource creates an HTTPSource. The optional EventBroadcaster is used to push notifications when new interactions arrive.

func (*HTTPSource) GetInteraction

func (h *HTTPSource) GetInteraction(id string) (*Interaction, error)

GetInteraction returns the full interaction details for a given ID.

func (*HTTPSource) GetResponse

func (h *HTTPSource) GetResponse(ctx context.Context, interaction Interaction) (*InteractionResponse, error)

GetResponse implements ResponseSource. It adds the interaction to the pending map, broadcasts an event, and blocks until a response is submitted via Respond() or the context is cancelled.

func (*HTTPSource) ListPending

func (h *HTTPSource) ListPending() []InteractionSummary

ListPending returns summaries of all pending interactions.

func (*HTTPSource) PendingCount

func (h *HTTPSource) PendingCount() int

PendingCount returns the number of interactions awaiting responses.

func (*HTTPSource) Respond

func (h *HTTPSource) Respond(id string, resp InteractionResponse) error

Respond submits a response for a pending interaction, unblocking the waiting GetResponse() call.

func (*HTTPSource) SetBroadcaster

func (h *HTTPSource) SetBroadcaster(broadcaster EventBroadcaster)

SetBroadcaster sets or replaces the event broadcaster for push notifications.

type Interaction

type Interaction struct {
	ID        string    `json:"id"`
	Messages  []Message `json:"messages"`
	Tools     []ToolDef `json:"tools"`
	CreatedAt time.Time `json:"created_at"`
}

Interaction represents a single LLM call that needs a response.

type InteractionResponse

type InteractionResponse struct {
	Content   string     `json:"content"`
	ToolCalls []ToolCall `json:"tool_calls,omitempty"`
	Error     string     `json:"error,omitempty"`
	Usage     Usage      `json:"usage,omitempty"`
}

InteractionResponse is the response supplied by a ResponseSource.

type InteractionSummary

type InteractionSummary struct {
	ID        string    `json:"id"`
	MsgCount  int       `json:"msg_count"`
	ToolCount int       `json:"tool_count"`
	CreatedAt time.Time `json:"created_at"`
}

InteractionSummary is a brief view of a pending interaction for list endpoints.

type Message

type Message struct {
	Role       Role       `json:"role"`
	Content    string     `json:"content"`
	ToolCallID string     `json:"tool_call_id,omitempty"` // for tool results
	ToolCalls  []ToolCall `json:"tool_calls,omitempty"`   // for assistant messages with tool calls
}

Message is a single turn in a conversation.

type ModelInfo

type ModelInfo struct {
	ID            string `json:"id"`
	Name          string `json:"name"`
	ContextWindow int    `json:"context_window,omitempty"`
}

ModelInfo describes an available model from a provider.

func ListModels

func ListModels(ctx context.Context, providerType, apiKey, baseURL string) ([]ModelInfo, error)

ListModels fetches available models from the given provider type. Only requires an API key and optional base URL — no saved provider needed.

type OllamaClient added in v0.6.0

type OllamaClient struct {
	// contains filtered or unexported fields
}

OllamaClient provides utility operations against a local Ollama server: pulling models and listing available models. For chat/stream, use the Genkit-backed provider returned by genkit.NewOllamaProvider.

func NewOllamaClient added in v0.6.0

func NewOllamaClient(serverAddress string) *OllamaClient

NewOllamaClient creates an OllamaClient pointing at the given server address. If serverAddress is empty, http://localhost:11434 is used.

func (*OllamaClient) Health added in v0.6.0

func (c *OllamaClient) Health(ctx context.Context) error

Health checks whether the Ollama server is reachable.

func (*OllamaClient) ListModels added in v0.6.0

func (c *OllamaClient) ListModels(ctx context.Context) ([]ModelInfo, error)

ListModels returns the models available on the Ollama server.

func (*OllamaClient) Pull added in v0.6.0

func (c *OllamaClient) Pull(ctx context.Context, model string, progressFn func(pct float64)) error

Pull downloads a model via the Ollama server. progressFn is called with percent completion (0–100); may be nil.

type Provider

type Provider interface {
	// Name returns the provider identifier (e.g., "anthropic", "openai", "mock").
	Name() string

	// Chat sends a non-streaming request and returns the complete response.
	Chat(ctx context.Context, messages []Message, tools []ToolDef) (*Response, error)

	// Stream sends a streaming request. Events are delivered on the returned channel.
	// The channel is closed when the response is complete or an error occurs.
	Stream(ctx context.Context, messages []Message, tools []ToolDef) (<-chan StreamEvent, error)

	// AuthModeInfo returns metadata about this provider's authentication mode.
	AuthModeInfo() AuthModeInfo
}

Provider is an AI backend that powers agent reasoning.

type Response

type Response struct {
	Content   string     `json:"content"`
	Thinking  string     `json:"thinking,omitempty"` // reasoning trace (e.g. from <think> tags)
	ToolCalls []ToolCall `json:"tool_calls,omitempty"`
	Usage     Usage      `json:"usage"`
}

Response is a completed (non-streaming) provider response.

type ResponseSource

type ResponseSource interface {
	// GetResponse receives the current interaction (messages + tools) and returns
	// a response. Implementations may block (e.g. waiting for human input).
	GetResponse(ctx context.Context, interaction Interaction) (*InteractionResponse, error)
}

ResponseSource is the interface that pluggable backends implement to supply responses for the TestProvider.

type Role

type Role string

Role identifies the sender of a chat message.

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

type ScriptedScenario

type ScriptedScenario struct {
	Name        string         `yaml:"name" json:"name"`
	Description string         `yaml:"description,omitempty" json:"description,omitempty"`
	Steps       []ScriptedStep `yaml:"steps" json:"steps"`
	Loop        bool           `yaml:"loop,omitempty" json:"loop,omitempty"`
}

ScriptedScenario is a named sequence of steps loadable from YAML.

func LoadScenario

func LoadScenario(path string) (*ScriptedScenario, error)

LoadScenario reads a ScriptedScenario from a YAML file.

type ScriptedSource

type ScriptedSource struct {
	// contains filtered or unexported fields
}

ScriptedSource returns responses from a pre-defined sequence of steps. It is safe for concurrent use.

func NewScriptedSource

func NewScriptedSource(steps []ScriptedStep, loop bool) *ScriptedSource

NewScriptedSource creates a ScriptedSource from the given steps. If loop is true, steps cycle indefinitely; otherwise GetResponse returns an error when all steps are exhausted.

func NewScriptedSourceFromScenario

func NewScriptedSourceFromScenario(scenario *ScriptedScenario) *ScriptedSource

NewScriptedSourceFromScenario creates a ScriptedSource from a loaded scenario.

func (*ScriptedSource) GetResponse

func (s *ScriptedSource) GetResponse(ctx context.Context, interaction Interaction) (*InteractionResponse, error)

GetResponse implements ResponseSource.

func (*ScriptedSource) Remaining

func (s *ScriptedSource) Remaining() int

Remaining returns how many unconsumed steps remain.

type ScriptedStep

type ScriptedStep struct {
	Content   string        `yaml:"content" json:"content"`
	ToolCalls []ToolCall    `yaml:"tool_calls,omitempty" json:"tool_calls,omitempty"`
	Error     string        `yaml:"error,omitempty" json:"error,omitempty"`
	Delay     time.Duration `yaml:"delay,omitempty" json:"delay,omitempty"`
}

ScriptedStep defines a single scripted response in a test scenario.

type StreamEvent

type StreamEvent struct {
	Type     string    `json:"type"` // "text", "thinking", "tool_call", "done", "error"
	Text     string    `json:"text,omitempty"`
	Thinking string    `json:"thinking,omitempty"`
	Tool     *ToolCall `json:"tool,omitempty"`
	Error    string    `json:"error,omitempty"`
	Usage    *Usage    `json:"usage,omitempty"`
}

StreamEvent is emitted during streaming responses.

type TestProvider

type TestProvider struct {
	// contains filtered or unexported fields
}

TestProvider implements Provider by delegating to a ResponseSource. It enables interactive and scripted E2E testing of the agent execution pipeline.

func NewTestProvider

func NewTestProvider(source ResponseSource, opts ...TestProviderOption) *TestProvider

NewTestProvider creates a TestProvider backed by the given ResponseSource.

func (*TestProvider) AuthModeInfo

func (tp *TestProvider) AuthModeInfo() AuthModeInfo

AuthModeInfo implements Provider.

func (*TestProvider) Chat

func (tp *TestProvider) Chat(ctx context.Context, messages []Message, tools []ToolDef) (*Response, error)

Chat implements Provider.

func (*TestProvider) InteractionCount

func (tp *TestProvider) InteractionCount() int64

InteractionCount returns how many interactions have been processed.

func (*TestProvider) Name

func (tp *TestProvider) Name() string

Name implements Provider.

func (*TestProvider) Source

func (tp *TestProvider) Source() ResponseSource

Source returns the underlying ResponseSource.

func (*TestProvider) Stream

func (tp *TestProvider) Stream(ctx context.Context, messages []Message, tools []ToolDef) (<-chan StreamEvent, error)

Stream implements Provider by wrapping Chat() into stream events.

type TestProviderOption

type TestProviderOption func(*TestProvider)

TestProviderOption configures a TestProvider.

func WithName

func WithName(s string) TestProviderOption

WithName sets the provider name returned by Name().

func WithTimeout

func WithTimeout(d time.Duration) TestProviderOption

WithTimeout sets the maximum time to wait for a response from the source.

type ThinkingStreamParser added in v0.5.3

type ThinkingStreamParser struct {
	// contains filtered or unexported fields
}

ThinkingStreamParser tracks state across streaming chunks to split thinking vs content tokens. Handles <think>/<think> split across chunks.

func (*ThinkingStreamParser) Feed added in v0.5.3

func (p *ThinkingStreamParser) Feed(chunk string) []StreamEvent

Feed processes a streaming chunk and returns zero or more StreamEvents. Events may be of type "thinking" (Thinking field set) or "text" (Text field set).

type ToolCall

type ToolCall struct {
	ID        string         `json:"id"`
	Name      string         `json:"name"`
	Arguments map[string]any `json:"arguments"`
}

ToolCall is a request from the AI to invoke a tool.

type ToolDef

type ToolDef struct {
	Name        string         `json:"name"`
	Description string         `json:"description"`
	Parameters  map[string]any `json:"parameters"` // JSON Schema
}

ToolDef describes a tool the agent can invoke.

type Usage

type Usage struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
}

Usage tracks token consumption.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL