llm

type APIConfig ¶

type APIConfig = config.APIConfig

APIConfig is the per-provider credentials struct llm.Client factories receive. Aliased to pkg/config.APIConfig so the canonical definition can live in pkg/config (which pkg/llm and pkg/tools both depend on without forming a cycle).

type Chunk ¶

type Chunk struct {
	Kind  ChunkKind
	Delta string
}

Chunk is one delta emitted during a streaming completion. Delta is the incremental text the provider just produced — never the cumulative buffer. Consumers append it to their own in-flight block.

type ChunkFunc ¶

type ChunkFunc func(Chunk)

ChunkFunc adapts a plain function into a ChunkSink. Convenient for tests and small inline consumers.

func (f ChunkFunc) OnChunk(c Chunk)

type ChunkKind ¶

type ChunkKind int

ChunkKind discriminates what kind of text a streaming delta carries. Providers translate their wire-level signal (text_delta, reasoning_content, thinking_delta, message.content, ...) into one of these values so the agent and UI never need to import a provider package.

const (
	// ChunkText is a delta of the assistant's user-facing reply.
	ChunkText ChunkKind = iota
	// ChunkThinking is a delta of the model's reasoning trace (DeepSeek
	// reasoning_content, Anthropic thinking_delta, Ollama message.thinking).
	ChunkThinking
)

type ChunkSink ¶

type ChunkSink interface {
	OnChunk(Chunk)
}

ChunkSink consumes deltas as they arrive. Providers call OnChunk from the goroutine that owns the streaming read; calls are serialized by the provider, never invoked concurrently for the same Stream call.

Implementations should be fast and non-blocking. The agent's adapter (see internal/agent/stream.go) forwards each chunk to the event sink with the usual emit mutex, which is sufficient.

var DiscardChunks ChunkSink = ChunkFunc(func(Chunk) {})

type Client ¶

type Client interface {
	Name() string
	Model() string
	// SupportsDeferLoading reports whether this provider natively supports
	// defer_loading (Anthropic, OpenAI). When false, the agent MUST NOT
	// mutate the tools array between turns — doing so would change the
	// prefix and invalidate prompt caching for providers that lack a
	// server-side defer_loading / tool_reference mechanism.
	SupportsDeferLoading() bool
	Complete(ctx context.Context, messages []Message, tools []tools.Tool) (Response, error)
	// Stream is the chunk-by-chunk variant of Complete. Implementations push
	// each text/thinking delta through sink as it arrives, then return the
	// fully assembled Response (content, thinking, signature, tool calls,
	// usage). Cancellation rules are identical to Complete.
	//
	// A provider that has no native streaming endpoint MAY fall back to a
	// buffered Complete and emit a single Chunk per kind at the end — this
	// keeps the contract uniform at the cost of no progressive output.
	Stream(ctx context.Context, messages []Message, tools []tools.Tool, sink ChunkSink) (Response, error)
	// Apply tunes request parameters at runtime. Same options accepted by
	// NewXxx — see WithSystem, WithTemperature, etc.
	Apply(opts ...Option)
}

Client abstracts the LLM provider so the agent loop never imports a concrete SDK.

Cancellation contract: Complete MUST honor ctx and abort the in-flight request promptly when ctx is cancelled. On cancellation the implementation must return an error matching ErrInterrupted (via errors.Is). The TUI binds ESC to ctx cancellation, so this contract is what makes user interrupts work end-to-end.

type ClientFactory ¶

type ClientFactory func(api APIConfig, model string, opts ...Option) (Client, error)

ClientFactory builds one llm.Client instance for the given provider credentials, model id, and option list. Each registered provider supplies a ClientFactory that wraps its own New() constructor.

Factories may return an error when the provided APIConfig is invalid (e.g. missing API key for a cloud provider). Returning nil for both Client and err is a programmer error.

type LLMParams ¶

type LLMParams struct {
	Temperature   *float64
	TopP          *float64
	TopK          *int
	MaxTokens     int
	StopSequences []string
	System        string
	Effort        int // from 1~n (every model provider should adapt their own impl)

	// HTTPClient overrides the transport used to talk to the provider.
	// nil → http.DefaultClient.
	HTTPClient *http.Client
}

LLMParams holds tunable request parameters shared across providers. Pointer fields preserve the "explicitly unset" semantic so each client can omit them and fall back to the upstream API's default.

func (p *LLMParams) Apply(opts ...Option)

func (p *LLMParams) HTTP() *http.Client

type Message ¶

type Message struct {
	Role              Role
	Content           string
	Thinking          string
	ThinkingSignature string
	ToolCalls         []*tools.Call
	ToolResults       []*ToolResult
}

Message is one turn of the conversation passed to and from the LLM.

ToolCalls is set on RoleAssistant turns when the model wants to invoke one or more tools. ToolResults is set on RoleTool turns and carries the result of each call, paired by ID with the corresponding ToolCall. A single RoleTool message carries every result for the preceding assistant turn — Anthropic mandates that fan-in, and the OpenAI-style converters fan it back out into per-call messages on the wire.

Thinking is provider-specific reasoning text. The TUI may render it, and providers that require it MUST echo it back in subsequent requests:

• DeepSeek: reasoning_content
• Anthropic: thinking blocks (with ThinkingSignature) — required whenever tool_use follows a thinking block, or the API errors 400.

ThinkingSignature is the opaque crypto signature Anthropic ships with each thinking block. Carry it round-trip; treat as a black box.

type Option ¶

type Option func(*LLMParams)

Option mutates LLMParams. Options are accepted by every client constructor and by the per-client Apply method, so the same knobs work at init and at runtime.

func UnsetTemperature() Option

func UnsetTopK() Option

func UnsetTopP() Option

func WithEffort(e int) Option

func WithHTTPClient(c *http.Client) Option

func WithMaxTokens(v int) Option

func WithStopSequences(seqs ...string) Option

func WithSystem(s string) Option

func WithTemperature(v float64) Option

func WithTopK(v int) Option

func WithTopP(v float64) Option

type Registry ¶

type Registry struct {
	// contains filtered or unexported fields
}

Registry maps provider names to ClientFactories. The agent loop resolves a (providerName, model) pair through Registry.Build at agent construction; downstream apps register additional providers before the first call to Build by writing to DefaultRegistry().

Registry is safe for concurrent use. Register fails on duplicate names — silently overwriting would let a typo route a model id to the wrong implementation. Use MustRegister at init time when a duplicate is a programming bug.

func DefaultRegistry() *Registry

import _ "github.com/johnny1110/evva/pkg/llm/builtins"

llm.DefaultRegistry().MustRegister(claude.ProviderName, claude.Factory)

func NewRegistry() *Registry

func (r *Registry) Build(name, model string, api APIConfig, opts []Option) (Client, error)

func (r *Registry) Has(name string) bool

func (r *Registry) MustRegister(name string, factory ClientFactory)

func (r *Registry) Names() []string

func (r *Registry) Register(name string, factory ClientFactory) error

package main

import (
	"context"
	"fmt"

	"github.com/johnny1110/evva/pkg/llm"
	"github.com/johnny1110/evva/pkg/tools"
)

// nullClient is a deterministic stand-in for examples. Replace with a
// real provider in production downstream code (or import
// `_ "github.com/johnny1110/evva/pkg/llm/builtins"` for the bundled
// Anthropic / DeepSeek / Ollama clients).
type nullClient struct{ model string }

func (n *nullClient) Name() string             { return "null" }
func (n *nullClient) Model() string            { return n.model }
func (*nullClient) SupportsDeferLoading() bool { return false }
func (n *nullClient) Complete(context.Context, []llm.Message, []tools.Tool) (llm.Response, error) {
	return llm.Response{Content: "ok"}, nil
}
func (n *nullClient) Stream(ctx context.Context, m []llm.Message, t []tools.Tool, _ llm.ChunkSink) (llm.Response, error) {
	return n.Complete(ctx, m, t)
}
func (*nullClient) Apply(...llm.Option) {}

// ExampleRegistry_Register demonstrates the canonical "add a custom
// LLM provider" pattern. Pick a unique name, supply a ClientFactory
// that wraps your provider's constructor, and the agent layer can
// then build clients for that provider by name through the same
// machinery the bundled providers use.
func main() {
	if err := llm.DefaultRegistry().Register("null-example",
		func(_ llm.APIConfig, model string, _ ...llm.Option) (llm.Client, error) {
			return &nullClient{model: model}, nil
		},
	); err != nil {
		fmt.Println("register error:", err)
		return
	}
	client, err := llm.DefaultRegistry().Build("null-example", "v0",
		llm.APIConfig{}, nil)
	if err != nil {
		fmt.Println("build error:", err)
		return
	}
	fmt.Println("provider:", client.Name(), "model:", client.Model())
}

Output:
provider: null model: v0

type Response ¶

type Response struct {
	Content           string
	Thinking          string
	ThinkingSignature string
	ToolCalls         []*tools.Call
	Usage             Usage
}

Response is what the LLM returns on each completion turn.

ToolCalls is non-empty when the model wants to invoke tools instead of (or in addition to) replying with text. Each call carries the provider's id in Call.ID — the agent echoes that back in the matching ToolResult.

Thinking carries any provider-specific reasoning trace; empty for providers that don't expose one. See Message.Thinking for the round-trip caveat.

type Role ¶

type Role string

Role labels who emitted a message.

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

type ToolResult ¶

type ToolResult struct {
	ID            string
	Content       string
	IsError       bool
	ContentBlocks []tools.ContentBlock
}

ToolResult pairs a tool call's id with the result the agent dispatched. Lives on RoleTool messages so one message can carry N results from a parallel-dispatched assistant turn.

type Usage ¶

type Usage struct {
	InputTokens         int
	OutputTokens        int
	CacheReadTokens     int
	CacheCreationTokens int
	ReasoningTokens     int
}

Usage reports token counts from a single LLM call. Zero values mean "unknown / not reported" — fields are populated only when the provider returns them, so accumulating zero across turns is safe.

CacheReadTokens / CacheCreationTokens are populated by Anthropic when prompt caching is in play and left zero by every other provider. ReasoningTokens is the share of output spent on hidden reasoning (DeepSeek reasoning_content, OpenAI o1-style chains).

func (u Usage) Add(v Usage) Usage

func (u Usage) Total() int