model

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 4, 2026 License: MIT Imports: 30 Imported by: 0

Documentation

Overview

Package model provides the ModelRegistry and claw-based NodeExecutor for resolving "provider/model-id" specs and executing LLM nodes.

Index

Constants

View Source
const DefaultBackoffBase = time.Second

DefaultBackoffBase is the base duration for exponential backoff.

View Source
const DefaultMaxAttempts = 3

DefaultMaxAttempts is the default number of LLM call attempts (initial + retries).

Variables

View Source
var ErrCompactionUnsupported = errors.New("model: compaction not supported by executor")

ErrCompactionUnsupported is the sentinel ClawExecutor.Compact returns when the backend has no in-process conversation handle to drop. The runtime re-exports it (runtime.ErrCompactionUnsupported is an alias) so the engine's `errors.Is` check works without importing model directly. Lives here because runtime imports model, not the reverse.

Functions

func ClassifyStreamError

func ClassifyStreamError(body []byte) error

ClassifyStreamError parses a stream error event and returns the appropriate typed error (*ContextOverflowError or *APIError), or nil if the data is not a recognized error event.

func DefaultDangerousCommandPatterns

func DefaultDangerousCommandPatterns() []string

DefaultDangerousCommandPatterns returns conservative regexes that match obviously destructive shell invocations. The list is small on purpose: each entry should have very low false-positive rate against typical engineering workflows.

func NewDefaultLifecycleHooks

func NewDefaultLifecycleHooks(_ EventHooks) *hooks.Runner

NewDefaultLifecycleHooks returns a Runner pre-configured with the hooks every iterion run should have on by default:

  • SafetyHook on PreToolUse: blocks dangerous bash commands (rm -rf /, fork bombs) before they run.

Tool-call observability (PostToolUse, PostToolUseFailure) is NOT wired here: the executor already emits OnToolCall via applyHooks with the correct node ID at every tool outcome (success, error, or hook-blocked). Adding a PostToolUse audit hook here would duplicate the same event under a placeholder node ID — see AuditHook docs.

Hosts that need to add or override hooks should construct their own runner via hooks.NewRunner and pass it via WithLifecycleHooks; this helper is a sane default for the common case.

The default patterns are hardcoded and known to compile, so this function never returns an error — but it preserves the error signature for symmetry with NewLifecycleHooks variants that take user-supplied patterns.

func NewSubagentRunner

func NewSubagentRunner(
	modelReg *Registry,
	toolReg *tool.Registry,
	eventHooks EventHooks,
	lifecycle *clawhooks.Runner,
	defaultModel string,
) func(ctx context.Context, input map[string]any) (string, error)

NewSubagentRunner returns a closure suitable as the executor for the claw `agent` tool. When the LLM dispatches the agent tool, the closure spins up a child conversation against the same model registry, filters the tool registry by AllowedToolsForSubagent, and returns the child's final text alongside its agent_id.

Hooks are propagated to the parent's event stream with a "subagent:<agent_id>" nodeID so a live test can observe the full sub-tree.

The returned executor never recurses into itself: the `agent` tool is filtered out of the child's tool set, regardless of the subagent type's allowlist. defaultModel is used when the input omits the model field; pass an empty string to force an explicit model on every call.

func ParseModelSpec

func ParseModelSpec(spec string) (providerName, modelID string, err error)

ParseModelSpec splits "provider/model-id" into its components.

func RunIDFromContext

func RunIDFromContext(ctx context.Context) string

RunIDFromContext returns the run ID set by WithRunID, or "" when none is wired (e.g. unit tests that exercise the executor without the runtime engine).

func SafetyHook

func SafetyHook(patterns []string) (hooks.Handler, error)

SafetyHook blocks PreToolUse events whose top-level "command" input matches one of `patterns`. Returns an error listing every pattern that fails to compile so operators are not silently running without the safety net they configured.

func SchemaToJSON

func SchemaToJSON(schema *ir.Schema) (json.RawMessage, error)

SchemaToJSON converts an IR Schema to a JSON Schema (json.RawMessage) suitable for use with sdk.WithExplicitSchema.

func ValidateOutput

func ValidateOutput(output map[string]interface{}, schema *ir.Schema) error

ValidateOutput checks that output contains all required fields from the schema with compatible types. It does NOT attempt to repair or coerce invalid values — the node must fail explicitly on schema mismatch.

func WithRunID

func WithRunID(ctx context.Context, runID string) context.Context

WithRunID returns a derived ctx carrying the run ID. The runtime engine calls this once per node execution, before `executor.Execute`. ClawExecutor reads it via RunIDFromContext.

func WithTemplateData

func WithTemplateData(ctx context.Context, td *TemplateData) context.Context

WithTemplateData returns a derived ctx carrying the template data snapshot. The runtime engine calls this once per node execution before invoking executor.Execute. Passing nil clears the snapshot.

Types

type APIError

type APIError struct {
	Message     string
	StatusCode  int
	IsRetryable bool
}

APIError represents a non-overflow API error.

func (*APIError) Error

func (e *APIError) Error() string

type ClawBackend

type ClawBackend struct {
	// contains filtered or unexported fields
}

ClawBackend implements delegate.Backend by calling GenerateTextDirect and GenerateObjectDirect against api.APIClient. It wraps the direct LLM path into the unified Backend interface.

func NewClawBackend

func NewClawBackend(registry *Registry, hk EventHooks, retry RetryPolicy, opts ...ClawBackendOption) *ClawBackend

NewClawBackend creates a new ClawBackend.

func (*ClawBackend) Execute

func (b *ClawBackend) Execute(ctx context.Context, task delegate.Task) (delegate.Result, error)

Execute implements delegate.Backend.

type ClawBackendOption

type ClawBackendOption func(*ClawBackend)

ClawBackendOption configures a ClawBackend at construction time.

func WithBackendLifecycleHooks

func WithBackendLifecycleHooks(r *hooks.Runner) ClawBackendOption

WithBackendLifecycleHooks installs an in-process hook runner fired around tool execution and at session end. A nil runner is a no-op.

type ClawExecutor

type ClawExecutor struct {
	// contains filtered or unexported fields
}

ClawExecutor implements runtime.NodeExecutor by routing LLM calls through pluggable Backend implementations (claw, claude_code, codex, etc.).

func NewClawExecutor

func NewClawExecutor(registry *Registry, wf *ir.Workflow, opts ...ClawExecutorOption) *ClawExecutor

NewClawExecutor creates a ClawExecutor for a given workflow.

func (*ClawExecutor) Close

func (e *ClawExecutor) Close() error

Close releases resources held by the executor, including MCP server connections. It should be called when the executor is no longer needed.

func (*ClawExecutor) Compact

func (e *ClawExecutor) Compact(ctx context.Context, nodeID string) error

Compact satisfies the runtime.Compactor structural interface.

ClawExecutor maintains a session-per-node store of messages accumulated during the previous attempt. On Compact, the pure CompactSessionPure helper from claw-code-go is applied to that list — the next retry's claw backend prepends the (now smaller) list to its opts.Messages so the LLM sees a summarised history instead of the full pre-overflow conversation.

When no session is wired (non-claw backends, or a node that has never been executed) Compact returns ErrCompactionUnsupported, the recovery dispatcher logs the gap, and the retry runs without special treatment — the same behaviour as before session tracking existed.

func (*ClawExecutor) EvictRun

func (e *ClawExecutor) EvictRun(runID string)

EvictRun drops every per-node session belonging to the given run. The runtime engine calls this when a run terminates (success, terminal failure, or cancellation) so a long-lived executor shared across runs does not leak session state from failed nodes.

func (*ClawExecutor) Execute

func (e *ClawExecutor) Execute(ctx context.Context, node ir.Node, input map[string]interface{}) (map[string]interface{}, error)

Execute implements runtime.NodeExecutor.

func (*ClawExecutor) ExecuteHumanLLMForInteraction

func (e *ClawExecutor) ExecuteHumanLLMForInteraction(
	ctx context.Context,
	nodeID string,
	ni *ErrNeedsInteraction,
	fields ir.InteractionFields,
) (answers map[string]interface{}, needsHuman bool, err error)

ExecuteHumanLLMForInteraction handles delegate interaction requests by creating a synthetic HumanNode from the original node's InteractionFields and calling executeHumanLLM. The questions from the ErrNeedsInteraction become the input, and the interaction schema is synthesized from the question keys.

Returns:

  • answers: LLM-generated answers for each question
  • needsHuman: true if the LLM decided to escalate (llm_or_human mode only)
  • err: any error from model execution

func (*ClawExecutor) LifecycleHooks

func (e *ClawExecutor) LifecycleHooks() *hooks.Runner

LifecycleHooks returns the runner installed via WithLifecycleHooks (nil if none). It is intended for backends that need to forward the runner into their own generation paths.

func (*ClawExecutor) MCPHealthCheck

func (e *ClawExecutor) MCPHealthCheck(ctx context.Context, servers []string) error

MCPHealthCheck verifies that the listed MCP servers are reachable by connecting and sending an MCP ping. Should be called before execution starts to fail fast on misconfigured servers.

func (*ClawExecutor) SetVars

func (e *ClawExecutor) SetVars(vars map[string]interface{})

SetVars merges run-level workflow variables into the executor's vars map. Keys present in vars override the matching default seeded from wf.Vars at construction time; keys absent from vars retain their default. Must be called before Execute.

func (*ClawExecutor) SetWorkDir

func (e *ClawExecutor) SetWorkDir(dir string)

SetWorkDir updates the working directory for backend subprocesses (claude_code, codex) and tool node shell exec. The engine calls this at run start when `worktree: auto` produces a per-run worktree path that wasn't known at executor construction time. Safe to call before Execute; not safe to call concurrently with an in-flight Execute.

type ClawExecutorOption

type ClawExecutorOption func(*ClawExecutor)

ClawExecutorOption configures a ClawExecutor.

func WithBackendRegistry

func WithBackendRegistry(dr *delegate.Registry) ClawExecutorOption

WithBackendRegistry sets the backend registry on the executor. When set, nodes with a `backend` property are executed via the named backend instead of the default claw backend.

func WithDefaultBackend

func WithDefaultBackend(name string) ClawExecutorOption

WithDefaultBackend sets the workflow-level default backend.

func WithEventHooks

func WithEventHooks(h EventHooks) ClawExecutorOption

WithEventHooks sets observability callbacks on the executor.

func WithLifecycleHooks

func WithLifecycleHooks(r *hooks.Runner) ClawExecutorOption

WithLifecycleHooks installs an in-process lifecycle hook runner. When set, the runner is consulted around every tool execution (PreToolUse, PostToolUse, PostToolUseFailure) and at session end (Stop). Build the runner once via hooks.NewRunner, register callbacks with runner.Register(event, handler), then pass it here.

A nil runner disables the integration (default).

func WithLogger

func WithLogger(l *iterlog.Logger) ClawExecutorOption

WithLogger sets a leveled logger for the executor.

func WithMCPManager

func WithMCPManager(m *mcp.Manager) ClawExecutorOption

WithMCPManager sets the generic MCP manager used to lazily discover MCP tools.

func WithRetryPolicy

func WithRetryPolicy(rp RetryPolicy) ClawExecutorOption

WithRetryPolicy sets the retry policy for transient LLM errors.

func WithToolPolicy

func WithToolPolicy(p tool.ToolChecker) ClawExecutorOption

WithToolPolicy sets the tool execution policy on the executor. When set, every tool call is checked against the allowlist before execution. A denied tool produces an explicit error.

func WithToolRegistry

func WithToolRegistry(tr *tool.Registry) ClawExecutorOption

WithToolRegistry sets the unified tool registry on the executor.

func WithWorkDir

func WithWorkDir(dir string) ClawExecutorOption

WithWorkDir sets the working directory for backend subprocesses. When set, backend nodes will run their CLI in this directory.

type CompactInfo

type CompactInfo struct {
	// BeforeMessages is the message count before compaction.
	BeforeMessages int

	// AfterMessages is the message count after compaction.
	AfterMessages int

	// RemovedMessageCount is reported by claw's CompactionResult.
	RemovedMessageCount int
}

CompactInfo is passed to the OnCompact hook when the running tool-loop message history is shrunk by claw's pure-function compactor between iterations. Emitted only when compaction actually fired (no event when the transcript was short enough to skip).

type ContextOverflowError

type ContextOverflowError struct {
	Message string
}

ContextOverflowError indicates the prompt exceeded the model's context window.

func (*ContextOverflowError) Error

func (e *ContextOverflowError) Error() string

type DelegateInfo

type DelegateInfo struct {
	BackendName        string        // e.g. "claude_code", "codex"
	Duration           time.Duration // subprocess wall-clock time
	Tokens             int           // estimated total tokens consumed
	ExitCode           int           // process exit code
	Stderr             string        // captured stderr output
	RawOutputLen       int           // byte length of raw stdout
	ParseFallback      bool          // true if structured output fell back to text wrapper
	FormattingPassUsed bool          // true if two-pass execution was used (tools + schema)
	Error              error         // non-nil for OnDelegateError
	Attempt            int           // 1-based retry number (for OnDelegateRetry)
	Delay              time.Duration // backoff delay (for OnDelegateRetry)
}

DelegateInfo describes a backend execution attempt, passed to backend hooks.

type ErrNeedsInteraction

type ErrNeedsInteraction struct {
	NodeID    string
	Questions map[string]interface{} // question_key → question text
	SessionID string                 // delegate session ID for re-invocation
	Backend   string                 // delegate backend name (empty for claw direct)

	// Conversation is the persisted backend-specific conversation history
	// captured at the pause point (claw: marshalled []api.Message). The
	// runtime relays this opaque blob into the checkpoint so that resume
	// can rehydrate the LLM's mid-tool-loop state instead of restarting
	// from system+user prompts. Backends that cannot persist conversation
	// state (CLI: claude_code, codex) leave this nil.
	Conversation json.RawMessage
	// PendingToolUseID is the ID of the tool_use block in Conversation
	// that is awaiting an answer. Required when Conversation is non-nil.
	PendingToolUseID string
}

ErrNeedsInteraction is returned by the executor when a delegate or LLM signals that it needs user input to continue. The runtime engine should handle this by pausing (interaction: human), auto-responding (interaction: llm), or deciding (interaction: llm_or_human) based on the node's InteractionMode.

func (*ErrNeedsInteraction) Error

func (e *ErrNeedsInteraction) Error() string

type EventEmitter

type EventEmitter interface {
	AppendEvent(runID string, evt store.Event) (*store.Event, error)
}

EventEmitter is the subset of store.RunStore used by the event bridge.

type EventHooks

type EventHooks struct {
	OnLLMRequest    func(nodeID string, info LLMRequestInfo)
	OnLLMPrompt     func(nodeID string, systemPrompt string, userMessage string)
	OnLLMResponse   func(nodeID string, info LLMResponseInfo)
	OnLLMRetry      func(nodeID string, info RetryInfo)
	OnLLMStepFinish func(nodeID string, step LLMStepInfo)
	OnLLMCompacted  func(nodeID string, info LLMCompactInfo)
	OnToolCall      func(nodeID string, info LLMToolCallInfo)
	// OnToolNodeResult is called for direct tool nodes (not LLM tool loops)
	// with full input/output content for detailed logging.
	OnToolNodeResult func(nodeID string, toolName string, input []byte, output string, elapsed time.Duration, err error)

	// Delegation lifecycle hooks.
	OnDelegateStarted  func(nodeID string, backendName string)
	OnDelegateFinished func(nodeID string, info DelegateInfo)
	OnDelegateError    func(nodeID string, info DelegateInfo)
	OnDelegateRetry    func(nodeID string, info DelegateInfo)

	// OnNodeFinished fires after a node's executor returns successfully.
	// The output map carries iterion's conventional usage keys (`_tokens`,
	// `_cost_usd`, `_model`) so observers (e.g. the Prometheus exporter)
	// can attribute cost and tokens per-node without re-parsing the event
	// log.
	OnNodeFinished func(nodeID string, output map[string]interface{})
}

EventHooks allows the executor to emit observability events back to the caller.

func ChainHooks

func ChainHooks(a, b EventHooks) EventHooks

ChainHooks composes two EventHooks so callbacks registered on either side run in order (a then b) for every event. Either side may leave any callback nil; the result keeps the non-nil one without an extra closure.

func NewStoreEventHooks

func NewStoreEventHooks(emitter EventEmitter, runID string, logger *iterlog.Logger) EventHooks

NewStoreEventHooks returns EventHooks that emit store events for a given run and log emoji-rich console output via the provided logger. The logger controls which content fields are included in events:

  • debug+: prompts, response text
  • trace: tool call inputs/outputs, tool call details

type FinishReason

type FinishReason string

FinishReason indicates why generation stopped.

const (
	FinishStop          FinishReason = "stop"
	FinishToolCalls     FinishReason = "tool-calls"
	FinishLength        FinishReason = "length"
	FinishContentFilter FinishReason = "content-filter"
	FinishError         FinishReason = "error"
	FinishOther         FinishReason = "other"
)

Canonical FinishReason constants.

type GenerationOptions

type GenerationOptions struct {
	// Model is the model ID (e.g., "claude-sonnet-4-6").
	Model string

	// System is the system prompt (plain string form).
	System string

	// SystemBlocks, when non-empty, takes precedence over System and is sent
	// as the Anthropic array-form `system` field. This is the only way to
	// attach `cache_control` markers to system content for prompt caching.
	SystemBlocks []api.ContentBlock

	// Messages is the initial conversation history.
	Messages []api.Message

	// Tools available for the model to call.
	Tools []GenerationTool

	// MaxSteps is the maximum number of tool-loop iterations (default 10).
	MaxSteps int

	// MaxTokens is the maximum tokens per response (default 8192).
	MaxTokens int

	// Temperature controls randomness (nil = provider default).
	Temperature *float64

	// ExplicitSchema is the JSON schema for structured output (GenerateObjectDirect).
	ExplicitSchema json.RawMessage

	// SchemaName is the name for the synthetic structured-output tool (default "structured_output").
	SchemaName string

	// ProviderOptions are provider-specific options (e.g., reasoning_effort).
	ProviderOptions map[string]any

	// CompactThresholdRatio overrides the default compaction trigger as a
	// fraction of the model's context window. 0 falls back to the built-in
	// default (0.85). Values outside (0, 1] fall back to the default.
	CompactThresholdRatio float64

	// CompactPreserveRecent overrides the default count of recent messages
	// kept verbatim during compaction. 0 falls back to the built-in default
	// (4). Values < 0 fall back to the default.
	CompactPreserveRecent int

	// OnRequest is called before each StreamResponse call.
	OnRequest func(RequestInfo)

	// OnResponse is called after each StreamResponse aggregation completes.
	OnResponse func(ResponseInfo)

	// OnStepFinish is called after each tool-loop step completes.
	OnStepFinish func(StepResult)

	// OnToolCall is called after each tool execution.
	OnToolCall func(ToolCallInfo)

	// OnCompact is called once after each in-loop compaction round
	// that actually shrunk the message history. No-op compactions
	// (transcript too short) do not fire the callback.
	OnCompact func(CompactInfo)

	// Hooks, when non-nil, is consulted around tool execution and at
	// session end. PreToolUse fires before each Execute and may Block
	// (the tool returns a synthetic refusal). PostToolUse fires after
	// successful Execute, PostToolUseFailure fires on error. Stop
	// fires once when the generation loop exits (success or failure).
	Hooks *hooks.Runner
}

GenerationOptions configures a direct generation call.

type GenerationTool

type GenerationTool struct {
	// Name is the tool identifier.
	Name string

	// Description describes what the tool does.
	Description string

	// InputSchema is the JSON schema for the tool's input parameters.
	InputSchema json.RawMessage

	// Execute runs the tool with the given JSON input and returns the result text.
	Execute func(ctx context.Context, input json.RawMessage) (string, error)
}

GenerationTool describes a tool available during direct generation. It replaces sdk.Tool for the native generation engine.

type LLMCompactInfo

type LLMCompactInfo struct {
	BeforeMessages      int
	AfterMessages       int
	RemovedMessageCount int
}

LLMCompactInfo describes a mid-tool-loop compaction round, passed to the OnLLMCompacted hook. Emitted when claw's pure-function compactor shrinks the running message history before the next StreamResponse call so long agentic loops on small-context models do not silently hit context_length_exceeded.

type LLMRequestInfo

type LLMRequestInfo struct {
	Model           string
	MessageCount    int
	ToolCount       int
	ReasoningEffort string
	Timestamp       time.Time
}

LLMRequestInfo describes an LLM request, passed to the OnLLMRequest hook.

type LLMResponseInfo

type LLMResponseInfo struct {
	Latency          time.Duration
	InputTokens      int
	OutputTokens     int
	CacheReadTokens  int
	CacheWriteTokens int
	FinishReason     string
	Error            error
	StatusCode       int
}

LLMResponseInfo describes an LLM response, passed to the OnLLMResponse hook.

type LLMStepInfo

type LLMStepInfo struct {
	Number           int
	Text             string
	ToolCalls        []ToolCallEntry
	FinishReason     string
	InputTokens      int
	OutputTokens     int
	CacheReadTokens  int
	CacheWriteTokens int
}

LLMStepInfo describes a single step in a multi-step LLM generation, passed to the OnLLMStepFinish hook.

type LLMToolCallInfo

type LLMToolCallInfo struct {
	ToolName  string
	InputSize int
	Duration  time.Duration
	Error     error
}

LLMToolCallInfo describes a tool call execution, passed to the OnToolCall hook.

type ModelCapabilities

type ModelCapabilities = llmtypes.ModelCapabilities

ModelCapabilities describes what features a model supports. Re-exported from llmtypes to keep the model package API stable.

type ObjectResult

type ObjectResult[T any] struct {
	// Object is the deserialized result.
	Object T

	// Text is the raw text that was parsed.
	Text string

	// Steps contains results from each generation step.
	Steps []StepResult

	// TotalUsage is the aggregated token usage across all steps.
	TotalUsage Usage

	// FinishReason indicates why generation stopped.
	FinishReason FinishReason
}

ObjectResult is a typed wrapper around a text generation result that includes a deserialized object of type T (for structured output / GenerateObject).

func GenerateObjectDirect

func GenerateObjectDirect[T any](ctx context.Context, client api.APIClient, opts GenerationOptions) (*ObjectResult[T], error)

GenerateObjectDirect generates structured output by injecting a synthetic tool with the given schema and forcing the model to call it. The tool_use input is parsed as the result object of type T.

type ProviderFactory

type ProviderFactory func(modelID string) (api.APIClient, error)

ProviderFactory creates an APIClient for a given model ID. The factory is called once per unique model ID; results are cached.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry resolves model specs of the form "provider/model-id" to APIClient instances. It caches resolved clients for reuse.

func NewRegistry

func NewRegistry() *Registry

NewRegistry creates a model registry pre-loaded with built-in providers.

func (*Registry) Capabilities

func (r *Registry) Capabilities(spec string) (ModelCapabilities, error)

Capabilities returns the capabilities of the model identified by spec. Capabilities are derived from a static table keyed by provider and model family.

func (*Registry) Register

func (r *Registry) Register(providerName string, factory ProviderFactory)

Register adds a provider factory under the given name. Calling Register with an already-registered name replaces the factory and invalidates any previously cached entries for that provider, so subsequent Resolve calls go through the new factory.

func (*Registry) Resolve

func (r *Registry) Resolve(spec string) (api.APIClient, error)

Resolve parses a model spec ("provider/model-id") and returns the corresponding APIClient, creating it via the provider factory if not already cached.

Concurrency: the registry mutex is only held while looking up or creating the per-key cache entry — never during the factory call. Concurrent Resolve calls for the same spec rendezvous on the entry's sync.Once so the factory runs exactly once; concurrent calls for different specs run their factories in parallel.

type RequestInfo

type RequestInfo struct {
	// Model is the model ID.
	Model string

	// MessageCount is the number of messages in the request.
	MessageCount int

	// ToolCount is the number of tools available.
	ToolCount int

	// ReasoningEffort is the resolved reasoning_effort spec sent on the
	// request, when set ("low", "medium", "high", "xhigh", "max"). Empty
	// when the node did not request a reasoning level.
	ReasoningEffort string

	// Timestamp is when the request was initiated.
	Timestamp time.Time
}

RequestInfo is passed to the OnRequest hook before a generation call.

type ResponseInfo

type ResponseInfo struct {
	// Latency is the time from request to response.
	Latency time.Duration

	// Usage is the token consumption for this call.
	Usage Usage

	// FinishReason indicates why generation stopped.
	FinishReason FinishReason

	// Error is non-nil if the call failed.
	Error error

	// StatusCode is the HTTP status code (0 if not applicable).
	StatusCode int
}

ResponseInfo is passed to the OnResponse hook after a generation call completes.

type RetryInfo

type RetryInfo struct {
	Attempt    int           // 1-based retry number (attempt 1 = first retry)
	Error      error         // the error that triggered this retry
	StatusCode int           // HTTP status code if available
	Delay      time.Duration // backoff delay before this retry
}

RetryInfo describes a retry attempt, passed to the OnLLMRetry hook.

type RetryPolicy

type RetryPolicy struct {
	// MaxAttempts is the total number of attempts (1 = no retry). Default: 3.
	MaxAttempts int
	// BackoffBase is the base delay for exponential backoff. Default: 1s.
	BackoffBase time.Duration
}

RetryPolicy controls automatic retry on transient LLM errors.

type StepResult

type StepResult struct {
	// Number is the 1-based step index.
	Number int

	// Text generated in this step.
	Text string

	// ToolCalls requested in this step.
	ToolCalls []ToolCall

	// FinishReason for this step.
	FinishReason FinishReason

	// Usage for this step.
	Usage Usage
}

StepResult describes a single generation step in a tool loop.

type TemplateData

type TemplateData struct {
	// Outputs is the per-node output map. Keys are node IDs.
	Outputs map[string]map[string]interface{}

	// LoopCounters is the current iteration count per loop name
	// (1-indexed once incremented by the engine).
	LoopCounters map[string]int

	// LoopMaxIterations is the declared `as <name>(N)` upper bound
	// per loop name, sourced from ir.Workflow.Loops.
	LoopMaxIterations map[string]int

	// LoopPreviousOutput is the snapshot of the source node's output
	// at the previous traversal of each loop's edge — i.e. one
	// iteration behind the current one. Nil on the first iteration.
	LoopPreviousOutput map[string]map[string]interface{}

	// Artifacts is the publish-name → output map for artifacts that
	// have been produced so far in this run.
	Artifacts map[string]map[string]interface{}

	// RunID is the current run identifier, exposed to prompts as
	// `{{run.id}}`.
	RunID string
}

TemplateData carries runtime state needed to resolve prompt template references in the `outputs.*`, `loop.*`, `artifacts.*`, and `run.*` namespaces. The runtime engine populates a snapshot of its current state before each node execution and attaches it via WithTemplateData so the executor can render prompt bodies that reference upstream outputs, loop counters, and previous-iteration snapshots.

All fields are read-only views — callers must not mutate them.

func TemplateDataFromContext

func TemplateDataFromContext(ctx context.Context) *TemplateData

TemplateDataFromContext returns the TemplateData attached by WithTemplateData, or nil when none is set. Resolvers tolerate nil by leaving cross-namespace refs (`outputs.*`, `loop.*`, etc.) unresolved — callers that don't wire the runtime context still get `input.*` and `vars.*` resolution.

type TextResult

type TextResult struct {
	// Text is the accumulated generated text.
	Text string

	// ToolCalls requested by the model in the final step.
	ToolCalls []ToolCall

	// Steps contains results from each generation step.
	Steps []StepResult

	// TotalUsage is the aggregated token usage across all steps.
	TotalUsage Usage

	// FinishReason indicates why generation stopped.
	FinishReason FinishReason

	// Messages is the final accumulated conversation list, including
	// the initial input plus every assistant turn (with tool_use
	// blocks) and tool_result message produced during the loop. Hosts
	// that maintain a per-node session — e.g. for compaction-aware
	// retries — capture this slice and feed it back as opts.Messages
	// on the next attempt.
	Messages []api.Message
}

TextResult is the final result of a text generation call.

func GenerateTextDirect

func GenerateTextDirect(ctx context.Context, client api.APIClient, opts GenerationOptions) (*TextResult, error)

GenerateTextDirect generates text using api.APIClient.StreamResponse directly. It runs a tool loop: call model → execute tools → append results → repeat, up to MaxSteps iterations.

type ToolCall

type ToolCall struct {
	// ID is a unique identifier for this tool call.
	ID string

	// Name of the tool to invoke.
	Name string

	// Input is the JSON-encoded arguments.
	Input json.RawMessage
}

ToolCall represents the model's request to invoke a tool.

type ToolCallEntry

type ToolCallEntry struct {
	Name  string
	Input json.RawMessage
}

ToolCallEntry describes a single tool call within a step.

type ToolCallInfo

type ToolCallInfo struct {
	// ToolName is the name of the tool that was called.
	ToolName string

	// InputSize is the byte length of the tool input JSON.
	InputSize int

	// Duration is how long the tool execution took.
	Duration time.Duration

	// Error is non-nil if the tool execution failed.
	Error error
}

ToolCallInfo is passed to the OnToolCall hook after a tool executes.

type Usage

type Usage struct {
	InputTokens      int
	OutputTokens     int
	TotalTokens      int
	ReasoningTokens  int
	CacheReadTokens  int
	CacheWriteTokens int
}

Usage tracks token consumption for a request.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL