runner

package
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 27, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ClaudeModels = []string{
	"sonnet",
	"opus",
	"haiku",
}

ClaudeModels lists available Claude model aliases accepted by the CLI --model flag.

View Source 
var CodexModels = []string{
	"gpt-5.5",
	"gpt-5.4",
	"gpt-5.4-mini",
	"gpt-5.3-codex",
	"gpt-5.2-codex",
	"gpt-5.2",
	"gpt-5.1-codex-max",
	"gpt-5.1-codex-mini",
}

CodexModels lists known Codex-compatible model names.

View Source 
var CopilotModels = []string{
	"claude-sonnet-4.6",
	"claude-sonnet-4.5",
	"claude-haiku-4.5",
	"claude-opus-4.8",
	"claude-opus-4.7",
	"claude-opus-4.6",
	"claude-opus-4.5",
	"gpt-5.5",
	"gpt-5.4",
	"gpt-5.4-mini",
	"gpt-5.3-codex",
	"gpt-5.2-codex",
	"gpt-5.2",
	"gpt-5-mini",
	"gpt-4.1",
}

CopilotModels is the curated fallback list of model IDs accepted by `copilot --model`. Mirrors the entries shown by the interactive `/model` picker. Kept in sync manually; the CLI does not expose a discovery command.

View Source 
var ErrRateLimited = errors.New("rate limited")

ErrRateLimited is the sentinel returned (wrapped) by all runners when the underlying LLM provider rejects a request because of rate limiting, quota exhaustion, or session usage caps. Callers can detect any flavour with errors.Is(err, runner.ErrRateLimited) and stop the pipeline immediately — continuing would just burn more retries against the same exhausted limit.

View Source 
var OllamaPopularModels = []string{
	"kimi-k2.5:cloud",
	"qwen2.5-coder:latest",
	"deepseek-coder-v2:latest",
	"codellama:latest",
	"llama3.2:latest",
	"mistral:latest",
	"phi4:latest",
	"gemma3:latest",
}

OllamaPopularModels is a curated list of models useful for coding tasks.

View Source 
var OpenCodeAvailableModels = []string{
	"qwen2.5-coder:3b",
	"qwen2.5-coder:latest",
	"llama3.2:latest",
	"deepseek-coder-v2:latest",
	"opencode/big-pickle",
	"opencode/gpt-5-nano",
}

OpenCodeAvailableModels is a fallback list used when the CLI is unavailable.

Functions

func ApplyModelMemory 

func ApplyModelMemory(cfg *config.Config)

ApplyModelMemory fills each agent's MaxContextTokens from the global ModelMemory setting. An explicit per-agent MaxContextTokens always wins and is left untouched. When ModelMemory is disabled (Mode "" or "off") nothing changes.

In "context" mode the configured token limit is applied verbatim to every agent. In "ram" mode the limit is derived per agent from its model via ContextTokensForRAM.

func ClaudeListModels 

func ClaudeListModels() ([]string, error)

ClaudeListModels returns the known Claude model aliases.

func CodexListModels 

func CodexListModels() ([]string, error)

CodexListModels returns available models by merging the currently configured model from ~/.codex/config.toml with the known models list. No API keys or network calls required.

func ContextTokensForRAM 

func ContextTokensForRAM(cfg config.AgentConfig, ramGB float64) int

ContextTokensForRAM estimates the largest context window (in tokens) that the agent's model can run within ramGB gigabytes of memory. It prefers real model metadata (Ollama reports the on-disk weight size) and falls back to a heuristic when metadata is unavailable (LM Studio / oMLX). The result is clamped to a sane range and rounded down to a multiple of 1024.

The conversion is intentionally approximate: an exact figure depends on the model's layer count, head dimensions and KV quantisation, which are not generally exposed by the local backends. Under-estimating is the safe failure mode, so the heuristic errs low.

func ContextTokensForWeights 

func ContextTokensForWeights(weightBytes int64, ramGB float64) int

ContextTokensForWeights estimates the largest context window (in tokens) a model of the given weight size can run within ramGB gigabytes of memory. It is the network-free core of ContextTokensForRAM, exposed so callers that already know a model's weight size (e.g. a UI listing several models) can compute estimates without re-fetching metadata per keystroke. weightBytes <= 0 falls back to the default estimate. The result is clamped and rounded down to a multiple of 1024.

func CopilotListModels 

func CopilotListModels() ([]string, error)

CopilotListModels returns the models the local Copilot CLI accepts. It prefers CLI-reported models when a future release exposes them, otherwise falls back to CopilotModels with the user's currently configured model (from ~/.copilot/settings.json) moved to the front.

func IsLocalRunner 

func IsLocalRunner(cfg config.AgentConfig) bool

IsLocalRunner returns true when the runner uses a local model (opencode with local Ollama model, or direct ollama runner). Local runners have no API cost so fix-attempt limits should not apply.

func LMStudioListInstalled 

func LMStudioListInstalled() ([]string, error)

LMStudioListInstalled returns model IDs available in the running LM Studio server. It prefers the native /api/v0/models endpoint (which reports each model's load state, so loaded models are listed first) and falls back to the OpenAI-compatible /v1/models endpoint.

func MLXListInstalled 

func MLXListInstalled() ([]string, error)

MLXListInstalled returns model IDs reported by the running oMLX server.

func OllamaInstalledWithSizes 

func OllamaInstalledWithSizes() (map[string]int64, error)

OllamaInstalledWithSizes returns installed Ollama models paired with their on-disk weight size in bytes, from a single /api/tags call.

func OllamaListInstalled 

func OllamaListInstalled() ([]string, error)

OllamaListInstalled returns model names available in the local Ollama instance.

func OllamaModelSizeBytes 

func OllamaModelSizeBytes(model string) int64

OllamaModelSizeBytes returns the on-disk size (≈ weight size) of an installed model as reported by /api/tags, or 0 when unavailable. Ollama matches model names loosely (an empty tag means ":latest"), so both the exact name and a ":latest" variant are considered.

func OpenCodeListModels 

func OpenCodeListModels() ([]string, error)

OpenCodeListModels returns available models from the OpenCode CLI.

func Ping 

func Ping(provider, model string) error

Ping performs a lightweight reachability check for the given provider and model. For CLI providers it verifies the binary exists; for Ollama it also checks the model is available.

func ProviderReachable 

func ProviderReachable(provider string) error

ProviderReachable performs a lightweight check that the given provider's backend is usable: the CLI binary is in PATH, or the local HTTP server responds. It deliberately does NOT verify a specific model so the result can be cached up-front (e.g. to gray out unavailable providers in the setup UI). Returns nil when reachable, otherwise a descriptive error.

func ResolveSkills 

func ResolveSkills(systemPrompt string, skillNames []string, loader *skills.Loader) string

ResolveSkills loads skill content and appends it to the system prompt. Returns the enriched system prompt. If loader is nil or no skills match, the original prompt is returned unchanged.

func UnloadFor 

func UnloadFor(ctx context.Context, cfg config.AgentConfig) error

UnloadFor frees the memory held by the local model described by cfg. It constructs the appropriate local runner and calls its Unload method. For cloud runners (which hold no local RAM) it is a no-op returning nil.

Types

type BudgetRunner 

type BudgetRunner struct {
	// contains filtered or unexported fields
}

BudgetRunner wraps an LLMRunner and truncates message content to stay within a token budget. Used for cloud runners where tokens have cost.

func (*BudgetRunner) Complete 

func (r *BudgetRunner) Complete(ctx context.Context, req CompletionRequest) (<-chan Token, error)

type ClaudeRunner 

type ClaudeRunner struct {
	Binary string
	Model  string
}

ClaudeRunner executes the Claude CLI (claude) as an LLM backend.

func (ClaudeRunner) Complete 

func (r ClaudeRunner) Complete(ctx context.Context, req CompletionRequest) (<-chan Token, error)

Complete implements LLMRunner by streaming the Claude CLI output incrementally. Uses `--output-format stream-json --verbose` so the TUI shows progress live instead of a single burst at the end of generation.

type CodexRunner 

type CodexRunner struct {
	Binary string
	Model  string
}

CodexRunner executes the OpenAI Codex CLI as an LLM backend. Uses `codex --quiet` for non-interactive text output.

func (CodexRunner) Complete 

func (r CodexRunner) Complete(ctx context.Context, req CompletionRequest) (<-chan Token, error)

Complete implements LLMRunner by running the Codex CLI with the given prompt. Stdout is streamed incrementally so the TUI shows progress live instead of a single burst at the end of generation.

type CompletionRequest 

type CompletionRequest struct {
	SystemPrompt string
	Skills       []string
	Messages     []ConvMessage
	Model        string
}

CompletionRequest describes what to send to the LLM.

type ConvMessage 

type ConvMessage struct {
	Role    string // "user" or "assistant"
	Content string
}

ConvMessage is a single turn in a conversation.

type CopilotRunner 

type CopilotRunner struct {
	Binary string
	Model  string
}

CopilotRunner executes the GitHub Copilot CLI (`copilot`) as an LLM backend. Uses `copilot -p <prompt>` for non-interactive prompt mode with `--allow-all-tools` so the CLI does not pause on tool-permission prompts.

func (CopilotRunner) Complete 

func (r CopilotRunner) Complete(ctx context.Context, req CompletionRequest) (<-chan Token, error)

Complete implements LLMRunner by running the Copilot CLI with the given prompt. Stdout is streamed incrementally so the TUI shows progress live.

type LLMRunner 

type LLMRunner interface {
	Complete(ctx context.Context, req CompletionRequest) (<-chan Token, error)
}

LLMRunner is the common interface for all LLM backends.

func New 

func New(cfg config.AgentConfig, skillLoader *skills.Loader, promptLanguage string) (LLMRunner, error)

New creates an LLMRunner from agent config. skillLoader may be nil — skills will be silently skipped. promptLanguage is accepted for backwards compatibility but ignored: all agents always reply in English regardless of the user's language.

type LMStudioRunner 

type LMStudioRunner struct {
	Model   string
	BaseURL string
}

LMStudioRunner calls the LM Studio OpenAI-compatible REST API directly. The local model receives the prompt verbatim and is expected to return plain text (no agentic loop, no tool use).

func NewLMStudioRunner 

func NewLMStudioRunner(model string) *LMStudioRunner

func (*LMStudioRunner) Complete 

func (r *LMStudioRunner) Complete(ctx context.Context, req CompletionRequest) (<-chan Token, error)

func (*LMStudioRunner) Unload 

func (r *LMStudioRunner) Unload(ctx context.Context, model string) error

Unload evicts the given model from LM Studio via the `lms` CLI (`lms unload <model>`). LM Studio's REST API has no documented unload endpoint, so this shells out best-effort: if the `lms` CLI is not on PATH the call is a no-op and returns nil.

type LocalModel 

type LocalModel struct {
	Runner      string // "ollama" | "lmstudio" | "mlx"
	Model       string
	WeightBytes int64 // 0 when the backend does not report a size (heuristic used)
}

LocalModel describes a locally-hosted model discovered on one of the local backends, with its weight size when the backend reports it.

func ListLocalModels 

func ListLocalModels() []LocalModel

ListLocalModels enumerates models currently available across the local backends (Ollama, LM Studio, oMLX). Unreachable backends are skipped silently so the result reflects whatever is up right now. Ollama entries include their on-disk weight size; LM Studio and oMLX entries leave WeightBytes at 0 because those servers do not report a reliable size.

type MLXRunner 

type MLXRunner struct {
	Model   string
	BaseURL string
}

MLXRunner calls the oMLX OpenAI-compatible REST API directly. The local model receives the prompt verbatim and is expected to return plain text (no agentic loop, no tool use).

func NewMLXRunner 

func NewMLXRunner(model string) *MLXRunner

func (*MLXRunner) Complete 

func (r *MLXRunner) Complete(ctx context.Context, req CompletionRequest) (<-chan Token, error)

func (*MLXRunner) Unload 

func (r *MLXRunner) Unload(ctx context.Context, model string) error

Unload is a best-effort no-op for oMLX: the server exposes no documented model-eviction endpoint, so DECSR cannot force an unload. It is defined to satisfy the Unloader interface; callers treat a nil return as "unload attempted". oMLX typically holds a single model and swaps it on the next request anyway.

type MockRunner 

type MockRunner struct {
	Responses []string
	MockUsage *TokenUsage // if set, attached to the Done token of each response
	// Requests captures every CompletionRequest passed to Complete, in order.
	// Useful for asserting on prompt construction in tests.
	Requests []CompletionRequest
	// contains filtered or unexported fields
}

MockRunner returns pre-programmed responses for testing.

func (*MockRunner) Complete 

func (m *MockRunner) Complete(_ context.Context, req CompletionRequest) (<-chan Token, error)

func (*MockRunner) Reset 

func (m *MockRunner) Reset()

Reset resets the response index so the mock can be reused.

type OllamaRunner 

type OllamaRunner struct {
	Model   string
	BaseURL string

	// NumCtx, when > 0, sets Ollama's num_ctx option so the model loads with a
	// bounded context window. This is the primary lever for capping a local
	// model's RAM use (the KV cache grows with the context window).
	NumCtx int
}

OllamaRunner calls the Ollama REST API directly (/api/chat) instead of shelling out through the Claude Code CLI.

func NewOllamaRunner 

func NewOllamaRunner(model string) *OllamaRunner

func (*OllamaRunner) Complete 

func (r *OllamaRunner) Complete(ctx context.Context, req CompletionRequest) (<-chan Token, error)

func (*OllamaRunner) Unload 

func (r *OllamaRunner) Unload(ctx context.Context, model string) error

Unload asks Ollama to evict the given model from memory immediately by issuing an empty /api/generate request with keep_alive=0. Best-effort: a nil return means the request was accepted, not that eviction is guaranteed.

type OpenCodeRunner 

type OpenCodeRunner struct {
	Binary string
	Model  string
}

OpenCodeRunner executes the OpenCode CLI in non-interactive mode as an LLM backend. Uses `opencode run` with --format json for structured output.

func (OpenCodeRunner) Complete 

func (r OpenCodeRunner) Complete(ctx context.Context, req CompletionRequest) (<-chan Token, error)

Complete implements LLMRunner by running the OpenCode CLI with `opencode run`.

type RateLimitError 

type RateLimitError struct {
	Provider string
	Detail   string
}

RateLimitError carries provider-specific context about a rate-limit hit while still satisfying errors.Is(err, ErrRateLimited).

func ClassifyRateLimit 

func ClassifyRateLimit(provider string, fragments ...string) *RateLimitError

ClassifyRateLimit returns a non-nil *RateLimitError when any of the given text fragments contains a known rate-limit marker. Returns nil otherwise.

func (*RateLimitError) Error 

func (e *RateLimitError) Error() string

func (*RateLimitError) Is 

func (e *RateLimitError) Is(target error) bool

type SkillRunner 

type SkillRunner struct {
	// contains filtered or unexported fields
}

SkillRunner wraps an LLMRunner and injects skill content into system prompts. Agent responses are always English; the user-language directive has been removed to keep agent output uniform across locales.

func (*SkillRunner) Complete 

func (r *SkillRunner) Complete(ctx context.Context, req CompletionRequest) (<-chan Token, error)

type Token 

type Token struct {
	Text string
	// Reasoning is display-only chain-of-thought from reasoning models. It is
	// shown live in the UI but NOT included in the returned/parsed result.
	Reasoning string
	Done      bool
	Error     error
	Usage     *TokenUsage // non-nil only on the Done token
}

Token is a single streaming chunk from an LLM.

type TokenUsage 

type TokenUsage struct {
	InputTokens  int
	OutputTokens int
	Estimated    bool
}

TokenUsage holds token counts for a single LLM completion. Estimated is true when the count was derived from a heuristic (chars÷4) rather than actual API data.

type Unloader 

type Unloader interface {
	Unload(ctx context.Context, model string) error
}

Unloader is implemented by local runners that can evict their model from memory. Cloud runners do not implement it (they hold no local RAM).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.