provider

package
v0.1.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 18, 2026 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package provider defines the Provider interface and shared types for AI model inference backends (Ollama, Claude, OpenAI).

Index

Constants

View Source
const DefaultHealthCheckInterval = 60 * time.Second

DefaultHealthCheckInterval is the default interval between health probe cycles.

Variables

View Source
var ErrNoHealthyProvider = errors.New("no healthy provider available")

ErrNoHealthyProvider is returned when no provider in the routing chain is healthy.

Functions

func IsRetryable added in v0.1.19

func IsRetryable(err error) bool

IsRetryable checks whether an error is retryable. Returns true if the error implements the Retryable() bool interface and returns true.

func ValidateRoutingConfig added in v0.1.19

func ValidateRoutingConfig(cfg *RegistryConfig, logger *zap.Logger) []string

ValidateRoutingConfig checks that Ollama providers are not present in code-gen routing chains. Returns a list of warnings for each violation. This is a soft validation -- it logs warnings but does not prevent startup.

func WakeOnLAN added in v0.1.16

func WakeOnLAN(macAddr string) error

WakeOnLAN sends a magic packet to wake a sleeping host. macAddr should be in "AA:BB:CC:DD:EE:FF" format.

Types

type ActivityNotifier added in v0.1.13

type ActivityNotifier interface {
	SetOnActivity(fn func())
}

ActivityNotifier is an optional interface that providers may implement to support streaming progress detection. When a provider supports this, the runner sets a callback that fires whenever the provider produces output bytes, allowing the dispatcher heartbeat to reset on active output.

type ChatRequest

type ChatRequest struct {
	Model      string    `json:"model"`
	Messages   []Message `json:"messages"`
	Stream     bool      `json:"stream"` // always false for now
	WorkingDir string    `json:"-"`      // workspace directory for CLI providers; not serialized
}

ChatRequest contains the parameters for a chat completion.

type ChatResponse

type ChatResponse struct {
	Model   string  `json:"model"`
	Message Message `json:"message"`

	// Token usage (if reported by provider).
	PromptTokens     int `json:"prompt_eval_count,omitempty"`
	CompletionTokens int `json:"eval_count,omitempty"`
}

ChatResponse contains the result of a chat completion.

type ErrProviderTimeout added in v0.1.19

type ErrProviderTimeout struct {
	Provider    string
	Model       string
	TimeoutType TimeoutType
	Duration    time.Duration
}

ErrProviderTimeout is returned when a provider times out during execution. It is retryable, allowing the pool to fall back to the next provider in the routing chain.

func (*ErrProviderTimeout) Error added in v0.1.19

func (e *ErrProviderTimeout) Error() string

func (*ErrProviderTimeout) Retryable added in v0.1.19

func (e *ErrProviderTimeout) Retryable() bool

Retryable returns true, indicating the pool should try the next provider.

type HealthDetail added in v0.1.16

type HealthDetail struct {
	ModelLoaded bool
	VRAMFree    int64
	VRAMTotal   int64
}

HealthDetail contains extended health information from providers that support it (currently Ollama).

type HealthDetailer added in v0.1.16

type HealthDetailer interface {
	HealthDetail(ctx context.Context) (*HealthDetail, error)
}

HealthDetailer is an optional interface that providers may implement to expose richer health information (e.g., model load status, VRAM usage).

type HealthMonitor added in v0.1.16

type HealthMonitor struct {
	// contains filtered or unexported fields
}

HealthMonitor runs periodic health checks against all providers in a Registry and caches the results for fast, lock-free reads by the dispatcher and API layer.

func NewHealthMonitor added in v0.1.16

func NewHealthMonitor(registry *Registry, interval time.Duration, logger *zap.Logger) *HealthMonitor

NewHealthMonitor creates a HealthMonitor that probes all providers in the registry at the given interval. Use Start to begin the probe loop.

func (*HealthMonitor) AllHealth added in v0.1.16

func (hm *HealthMonitor) AllHealth() []ProviderHealth

AllHealth returns cached health for all probed providers.

func (*HealthMonitor) ChainHealthy added in v0.1.16

func (hm *HealthMonitor) ChainHealthy(chain []string) bool

ChainHealthy returns true if at least one provider in the named routing chain is healthy according to cached probe results.

func (*HealthMonitor) GetHealth added in v0.1.16

func (hm *HealthMonitor) GetHealth(name string) ProviderHealth

GetHealth returns the cached health status for a named provider. Returns a zero-value ProviderHealth (Healthy=false) if the provider has not been probed yet.

func (*HealthMonitor) SetWoLTargets added in v0.1.16

func (hm *HealthMonitor) SetWoLTargets(targets map[string]WoLConfig)

SetWoLTargets configures Wake-on-LAN targets for providers. When a provider with a configured WoL target is unhealthy, the monitor sends a magic packet (subject to a 5-minute cooldown per target).

func (*HealthMonitor) Start added in v0.1.16

func (hm *HealthMonitor) Start(ctx context.Context)

Start runs the health check loop in a goroutine. It performs an initial probe immediately and then repeats at the configured interval. The loop exits when ctx is cancelled.

type Message

type Message struct {
	Role    Role   `json:"role"`
	Content string `json:"content"`
}

Message is a single chat message.

type Provider

type Provider interface {
	// Chat sends a chat completion request and returns the response.
	Chat(ctx context.Context, req ChatRequest) (*ChatResponse, error)

	// Healthy returns true if the provider is reachable and ready.
	Healthy(ctx context.Context) bool

	// Name returns the provider identifier (e.g., "ollama", "claude", "openai").
	Name() string
}

Provider is the interface for AI model inference backends. Implementations exist for Ollama (local), Claude, and OpenAI (cloud).

type ProviderConfig

type ProviderConfig struct {
	Type           string `yaml:"type"`            // "claude", "openai", "ollama", "claude-cli"
	APIKeyEnv      string `yaml:"api_key_env"`     // env var name for API key
	BaseURL        string `yaml:"base_url"`        // override base URL (for ollama or custom endpoints)
	DefaultModel   string `yaml:"default_model"`   // default model for this provider
	TimeoutSeconds int    `yaml:"timeout_seconds"` // per-provider timeout; 0 means use provider default
	AccountURL     string `yaml:"account_url"`     // billing/credits page URL
	AllowedTools   string `yaml:"allowed_tools"`   // claude-cli: comma-separated tool list (e.g. "Bash,Read,Edit,Write,Glob,Grep")
	MaxTurns       int    `yaml:"max_turns"`       // claude-cli: max agentic turns per session; 0 means no limit
	WoLMAC         string `yaml:"wol_mac"`         // optional Wake-on-LAN MAC address (e.g. "AA:BB:CC:DD:EE:FF")
}

ProviderConfig describes a single provider entry in YAML config.

type ProviderFactory

type ProviderFactory func(name string, cfg ProviderConfig) (Provider, error)

ProviderFactory constructs a Provider from a ProviderConfig. It is set by the application's wiring layer (e.g., cmd/samverk) to avoid the registry importing sub-packages directly in tests.

type ProviderHealth added in v0.1.16

type ProviderHealth struct {
	Name        string    `json:"name"`
	Healthy     bool      `json:"healthy"`
	LastChecked time.Time `json:"last_checked"`
	LastHealthy time.Time `json:"last_healthy"`
	Error       string    `json:"error,omitempty"`
	// Ollama-specific fields (zero values for non-Ollama providers).
	ModelLoaded bool  `json:"model_loaded,omitempty"`
	VRAMFree    int64 `json:"vram_free_bytes,omitempty"`
	VRAMTotal   int64 `json:"vram_total_bytes,omitempty"`
}

ProviderHealth describes the health status of a single provider.

type ProviderInfo

type ProviderInfo struct {
	Name    string
	Type    string
	Model   string
	Healthy bool
}

ProviderInfo describes a registered provider for listing.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry holds constructed providers and routing rules.

func LoadRegistry

func LoadRegistry(path string, factory ProviderFactory, logger *zap.Logger) (*Registry, error)

LoadRegistry reads a YAML config file, constructs providers using the given factory, and returns a fully wired Registry.

func NewRegistry

func NewRegistry(logger *zap.Logger) *Registry

NewRegistry creates an empty provider registry.

func (*Registry) Get

func (r *Registry) Get(ctx context.Context, agentType string) (Provider, string, error)

Get returns the first healthy provider from the routing chain for the given agent type. If the agent type is not configured, it falls back to "default". Returns (provider, model, error).

func (*Registry) GetAfter added in v0.1.19

func (r *Registry) GetAfter(ctx context.Context, agentType, afterProvider string) (p Provider, model string, err error)

GetAfter returns the first healthy provider in the routing chain that comes after the named provider. This enables timeout failover: when a provider times out, the pool can retry with the next one in the chain without re-checking providers already tried. Returns ErrNoHealthyProvider if no subsequent healthy provider exists.

func (*Registry) List

func (r *Registry) List(ctx context.Context) []ProviderInfo

List returns info about all registered providers with their health status.

func (*Registry) NameOf added in v0.1.19

func (r *Registry) NameOf(p Provider) string

NameOf returns the registry name for a given Provider instance, or empty string if not found. Used by the pool to identify which provider timed out for failover via GetAfter.

func (*Registry) ProviderByName added in v0.1.16

func (r *Registry) ProviderByName(name string) Provider

ProviderByName returns the raw Provider instance for the given name, or nil if the name is not registered.

func (*Registry) Register

func (r *Registry) Register(name, providerType string, p Provider, model string)

Register adds a named provider to the registry.

func (*Registry) Routing added in v0.1.16

func (r *Registry) Routing() map[string][]string

Routing returns a copy of the routing table mapping chain names to provider names.

func (*Registry) SetRouting

func (r *Registry) SetRouting(routing map[string][]string)

SetRouting sets the routing table that maps agent types to provider chains.

type RegistryConfig

type RegistryConfig struct {
	Providers map[string]ProviderConfig `yaml:"providers"`
	Routing   map[string][]string       `yaml:"routing"` // agent_type -> provider names
}

RegistryConfig is the top-level YAML structure for provider configuration.

func LoadRegistryConfig

func LoadRegistryConfig(path string) (*RegistryConfig, error)

LoadRegistryConfig reads and parses a YAML config file.

type Role

type Role string

Role identifies the sender of a chat message.

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
)

type TimeoutType added in v0.1.19

type TimeoutType string

TimeoutType distinguishes between different timeout failure modes.

const (
	// TimeoutStartup means the process produced no output at all within the
	// startup window. This typically indicates the process failed to launch
	// or is stuck initializing.
	TimeoutStartup TimeoutType = "startup"

	// TimeoutStale means the process was producing output but stopped for
	// longer than the stale output threshold. This typically indicates the
	// process hung mid-execution.
	TimeoutStale TimeoutType = "stale"
)

type WoLConfig added in v0.1.16

type WoLConfig struct {
	MAC string `yaml:"wol_mac"` // e.g., "AA:BB:CC:DD:EE:FF"
}

WoLConfig describes Wake-on-LAN settings for a provider host.

Directories

Path Synopsis
Package claude implements the provider.Provider interface using the Anthropic Claude Messages API over raw net/http (no SDK dependency).
Package claude implements the provider.Provider interface using the Anthropic Claude Messages API over raw net/http (no SDK dependency).
Package claudecli implements provider.Provider by shelling out to the Claude Code CLI (`claude --print`).
Package claudecli implements provider.Provider by shelling out to the Claude Code CLI (`claude --print`).
Package ollama implements the provider.Provider interface using the Ollama REST API over raw net/http (no SDK dependency).
Package ollama implements the provider.Provider interface using the Ollama REST API over raw net/http (no SDK dependency).
Package openai implements the provider.Provider interface using the OpenAI Chat Completions API over raw net/http (no SDK dependency).
Package openai implements the provider.Provider interface using the OpenAI Chat Completions API over raw net/http (no SDK dependency).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL