engine

package
v0.5.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 3, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package engine owns olium's multi-turn agent loop: it orchestrates the provider, tool registry, conversation history, and event fan-out. TUI and headless runners are thin consumers.

Index

Constants

View Source
const DefaultMaxHistoryBytes = 400 << 10

DefaultMaxHistoryBytes bounds the conversation sent on each request.

History is otherwise append-only for the life of a run: at up to 16 KiB per tool result (per result, not per turn - a parallel turn appends one each), an agent with a 750-turn budget reaches a 200K-token window's limit somewhere around turn 40-150 and then dies on a provider 400. The turn budget never binds; the context window does.

400 KiB is roughly 100K tokens, which leaves headroom under the smallest window in common use while keeping far more history than any run needs verbatim. Compaction elides the OLDEST tool results first, so the recent working set is untouched.

View Source
const DefaultMaxToolResultBytes = 16 * 1024

DefaultMaxToolResultBytes is the cap applied to each tool result when the engine appends it to conversation history. Tools that legitimately return more than this (e.g. very large code search results) get head + tail truncation with a clear elision marker so the model still sees the start and the most recent context.

View Source
const DefaultRetryInitialBackoff = time.Second

DefaultRetryInitialBackoff is the first sleep between transient stream retries when Config.RetryInitialBackoff is unset.

View Source
const DefaultToolTimeout = 5 * time.Minute

DefaultToolTimeout bounds each individual tool invocation. A runaway bash/web_fetch call shouldn't be able to hang the whole autopilot session; five minutes is generous for legitimate work (curl against a slow target, grep across a large repo) while still bounded.

Variables

This section is empty.

Functions

func SessionCacheKey added in v0.5.1

func SessionCacheKey(sessionDir string) string

SessionCacheKey derives a provider-side prompt-cache key from a run directory. filepath.Base("") is ".", which every keyless run would share, so an unset directory yields no key rather than a colliding one. Callers that own a real run id should pass that instead - this is the fallback.

Types

type Config

type Config struct {
	Provider provider.Provider
	Tools    *tool.Registry
	Model    string
	System   string
	// Skills, when non-nil, are injected into the system prompt as an
	// <available_skills> block. The model decides when to read them via
	// read_file. Does not constrain or replace Tools.
	Skills *skill.Registry
	// MaxTurns caps how many LLM→tool cycles a single Run may do. 0 = default 32.
	MaxTurns int
	// ToolTimeout bounds each tool invocation. 0 = DefaultToolTimeout.
	// Negative = disabled (tools run without an imposed deadline, honoring
	// only the parent ctx).
	ToolTimeout time.Duration
	// EnablePromptCache opts the provider into cache_control breakpoints
	// on the system prompt and tool list. Ignored by providers that don't
	// support caching. Recommended for long-running multi-turn loops
	// (autopilot) where the system prompt + tools dominate the prefix.
	EnablePromptCache bool
	// MaxToolCalls caps how many tool calls the engine will dispatch for the
	// life of this Engine. 0 = unlimited.
	//
	// Distinct from MaxTurns on purpose: one turn can carry any number of
	// parallel calls, so a turn ceiling does not bound the work done or the
	// money spent. The count deliberately survives Reset, so rotating a
	// section does not refill the budget.
	MaxToolCalls int

	// MaxHistoryBytes bounds the conversation carried into each request.
	// Past it, the oldest tool results are elided (see compactHistory).
	// 0 = DefaultMaxHistoryBytes, negative disables compaction.
	MaxHistoryBytes int

	// MaxToolResultBytes truncates large tool outputs (e.g. `bash ls -R`
	// on a big repo) before appending to history, preventing context
	// overflow across long multi-turn runs. 0 = DefaultMaxToolResultBytes
	// (16 KiB). Negative = disabled (no truncation).
	MaxToolResultBytes int

	// SpillDir, when non-empty, changes oversized-tool-result handling
	// from in-place head+tail truncation to spill-to-disk: the full
	// payload is written to a file under SpillDir/tool-results/ and the
	// in-history content becomes a head excerpt plus a clear pointer
	// (with the on-disk path), so the model can `read_file` the rest if
	// it needs more. Caller is responsible for the directory's lifecycle.
	SpillDir string

	// OnToolResult, when non-nil, is invoked with each tool's content
	// after shrink/spill but before the result is appended to history.
	// The returned string replaces the history-side content; the event
	// stream still emits the raw tool output so operator logs stay clean.
	// Autopilot uses this to pin a scratchpad digest at the tail of every
	// non-scratchpad tool result so plan state stays at the conversation
	// tail across long stretches between update_plan / remember calls.
	OnToolResult func(toolName string, content string, isErr bool) string

	// RetryInitialBackoff sets the first sleep between transient-stream-error
	// retries inside streamOnceWithRetry; each subsequent retry doubles up
	// to maxBackoff (10s). 0 = DefaultRetryInitialBackoff (1s). Tests set
	// this to ms-scale so they don't spend seconds on backoff sleeps.
	RetryInitialBackoff time.Duration

	// NudgeOnEmptyToolCalls caps how many consecutive text-only turns
	// (no tool_calls) the engine tolerates before exiting. After each empty
	// turn within the cap, NudgeOnEmptyMessage is appended as a user-role
	// message and the loop continues so the model gets one more chance to
	// either resume work or call the agent's halt tool. 0 = disabled
	// (legacy behavior: first empty turn ends the run). Use for agentic
	// loops where a capable model is expected to keep calling tools and a
	// silent text-only turn means the model has lost the loop — typical
	// with small open-weight models that don't reliably emit tool_calls.
	NudgeOnEmptyToolCalls int

	// NudgeOnEmptyMessage is the user-role text injected after an empty
	// turn when NudgeOnEmptyToolCalls > 0. Empty = a generic default that
	// asks for either a tool call or a halt. Callers that wire a specific
	// halt tool (autopilot's halt_scan) should override with a message
	// that names it explicitly.
	NudgeOnEmptyMessage string

	// SessionID is forwarded on every provider request as a stable cache key
	// for backends that take one (OpenAI Responses' prompt_cache_key). Unset
	// means the backend falls back to routing by prefix alone.
	SessionID string

	// ReasoningEffort is the thinking/effort level forwarded on every
	// provider request (low|medium|high|xhigh|max; "" = provider default).
	// Until this was wired, the configured reasoning_effort reached the TUI
	// banner and the session transcript but never the API, so every request
	// ran at the provider's default.
	ReasoningEffort string

	// MaxTokens caps each response's output tokens (0 = provider default).
	// On adaptive-thinking models reasoning counts against this, so a
	// ceiling sized for visible text alone truncates long turns.
	MaxTokens int

	// Recorder, when non-nil, receives a copy of every emitted Event plus
	// the initiating user prompt (see EventRecorder). Engine.Run tees
	// through it at a single chokepoint so no emit site or consumer drain
	// loop needs to know about it. Used to persist a Pi-style JSONL session
	// transcript (pkg/olium/sessionlog). Forks do NOT inherit the recorder
	// — concurrent sub-runs would interleave one file. A recorder that
	// implements io.Closer is closed by Engine.CloseRecorder.
	Recorder EventRecorder
}

Config configures an Engine.

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine is the multi-turn agent runtime. One Engine handles one conversation; call Run per user prompt.

func New

func New(cfg Config) *Engine

New constructs an Engine. Skills (if any) are baked into the system prompt at construction time — the registry itself is stable for the session, so there's no reason to re-render on every turn.

func (*Engine) CloseRecorder

func (e *Engine) CloseRecorder() error

CloseRecorder closes the configured session recorder if one is set and it implements io.Closer. Safe to call on an engine with no recorder (no-op) and on a fork (forks carry no recorder). Callers with a clear run lifecycle should defer this so the transcript file is flushed and closed.

func (*Engine) Fork

func (e *Engine) Fork() *Engine

Fork returns a new Engine that shares this engine's provider, tools, system prompt, and configuration but starts with an independent copy of the current conversation history. Use this to branch a multi-turn run: the parent's prefix (system + tool defs + earlier messages) remains in the new engine's prompt, so providers with prompt caching can serve the repeated prefix from cache, but writes to the fork's history don't echo back to the parent.

Typical use: source-analysis explore phase runs once on a parent engine, then 3 format/extension sub-calls Fork() and Run() in parallel — they each see the explore output in history without paying to re-append it in user prompts.

func (*Engine) History

func (e *Engine) History() []provider.Message

History returns a snapshot of the conversation. Safe to call from the UI thread.

func (*Engine) Reset

func (e *Engine) Reset()

Reset clears the conversation history and the per-call failure streaks that go with it - a fresh section starts from a clean slate, so a call that failed twice before the reset must not inherit an escalation banner.

func (*Engine) Run

func (e *Engine) Run(ctx context.Context, userPrompt string) <-chan Event

Run executes one user prompt through the multi-turn tool loop. The returned channel emits Events in order and closes when the run ends (success or error). All events must be drained.

func (*Engine) ToolCallCount added in v0.5.1

func (e *Engine) ToolCallCount() int

ToolCallCount reports how many tool calls this engine has dispatched.

type Event

type Event struct {
	Type EventType `json:"type"`

	// Text / thinking payloads.
	Delta string `json:"delta,omitempty"`

	// Tool events.
	ToolCallID   string         `json:"tool_call_id,omitempty"`
	ToolName     string         `json:"tool_name,omitempty"`
	ToolCategory string         `json:"tool_category,omitempty"` // tool.CategoryBuiltin / tool.CategoryVigolium
	ToolArgs     map[string]any `json:"tool_args,omitempty"`
	ToolResult   string         `json:"tool_result,omitempty"`
	ToolIsErr    bool           `json:"tool_is_error,omitempty"`

	// Turn / run lifecycle.
	StopReason stream.StopReason `json:"stop_reason,omitempty"`
	Usage      *stream.Usage     `json:"usage,omitempty"`

	// ToolCalls carries the model's tool calls for the just-completed turn.
	// Populated only on EventTurnDone, and only when the turn produced tool
	// calls. It exists so out-of-band consumers (the Pi-style JSONL session
	// transcript in pkg/olium/sessionlog) can render a complete assistant
	// message with full tool arguments without reconstructing them from the
	// argument-less EventToolCallStart events — arguments are only known once
	// the provider stream resolves, which is after the start event fires.
	// Live UI / tool-log consumers ignore this field.
	ToolCalls []stream.ToolCall `json:"tool_calls,omitempty"`

	// Stalled is set on EventRunDone when the run ended because the model
	// kept answering with text-only turns after every empty-turn nudge was
	// spent, as opposed to finishing on its own. Consumers use it to report
	// the stop honestly instead of as a natural completion.
	Stalled bool `json:"stalled,omitempty"`

	// BudgetExhausted marks an EventError that is a budget ceiling rather
	// than a failure - the turn or tool-call cap. The caller should finalize
	// the run (summary, triage, bookkeeping) as it would for any other
	// budget stop. Carried as a field so consumers don't classify a terminal
	// reason by string-matching an error this package formatted itself.
	BudgetExhausted bool `json:"budget_exhausted,omitempty"`

	// Error payload.
	Err string `json:"error,omitempty"`
}

Event is the single value type emitted on the engine event channel.

type EventRecorder

type EventRecorder interface {
	// UserPrompt is invoked once at the start of each Run with the prompt
	// text that seeds the turn, before any event is emitted.
	UserPrompt(prompt string)
	// Record is invoked for every Event the engine emits, in order, before
	// the event is forwarded on the public Run channel.
	Record(ev Event)
}

EventRecorder receives a copy of every engine Event, plus the initiating user prompt, so an out-of-band consumer — notably the Pi-style JSONL session transcript in pkg/olium/sessionlog — can persist a run without coupling into the engine's many event-emission sites or any consumer's drain loop. The engine feeds a single recorder sequentially from one goroutine per Run, so implementations need not be reentrant; they may, however, be called across successive Run calls on the same Engine (the interactive TUI reuses one Engine for a whole session).

A recorder that also implements io.Closer is closed by Engine.CloseRecorder.

type EventType

type EventType string

EventType is the union of engine-emitted event kinds. These are a superset of the provider's stream events: the engine forwards text / thinking / tool-call events from the provider and adds its own tool-execution and turn-lifecycle events.

const (
	// Forwarded from the provider.
	EventTextDelta     EventType = "text_delta"
	EventThinkingDelta EventType = "thinking_delta"
	EventToolCallStart EventType = "toolcall_start" // model began a tool call (unexecuted)

	// Emitted by the engine.
	EventToolExecStart    EventType = "tool_exec_start"    // engine about to run a tool
	EventToolExecProgress EventType = "tool_exec_progress" // tool streaming partial output
	EventToolExecEnd      EventType = "tool_exec_end"      // tool finished
	EventTurnDone         EventType = "turn_done"          // assistant turn complete (model finished this request)
	EventRunDone          EventType = "run_done"           // full run complete (no more tool calls pending)
	EventError            EventType = "error"
	// EventInfo carries a non-fatal engine-level notice (e.g. "transient
	// upstream stream error; retrying"). Message lives in Delta. Consumers
	// that don't recognize this type silently drop it — backward compatible.
	EventInfo EventType = "info"
)

type Message

type Message struct {
	Role      Role       `json:"role"`
	Text      string     `json:"text,omitempty"`
	ToolCalls []ToolCall `json:"tool_calls,omitempty"`

	ToolCallID string `json:"tool_call_id,omitempty"`
	Content    string `json:"content,omitempty"`
	IsError    bool   `json:"is_error,omitempty"`
}

Message is a single entry in the conversation history.

Assistant messages may carry Text, ToolCalls, or both. Tool messages reference the ToolCallID they answer and carry the rendered Content.

type Role

type Role string

Role enumerates conversation roles.

const (
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

type ToolCall

type ToolCall struct {
	ID   string         `json:"id"`
	Name string         `json:"name"`
	Args map[string]any `json:"args"`
}

ToolCall is a model-requested tool invocation.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL