tools

package
v0.17.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 15, 2026 License: MIT Imports: 65 Imported by: 0

Documentation

Overview

Package tools provides the interface-based tool system for the Sprout AI agent.

Tools are capabilities the LLM can invoke — reading files, executing shell commands, searching code, delegating to subagents, automating browsers, and more. Each tool implements the ToolHandler interface and is registered with the ToolRegistry.

ToolHandler interface

The ToolHandler interface replaced the legacy `type ToolHandler func(ctx, args, agent) (images, output, error)` func type. The old func-based system tightly coupled every tool to the *Agent type, making tools hard to test in isolation and difficult to share across different execution contexts. The new interface-based system provides explicit dependencies through ToolEnv, enabling clean separation of concerns:

type ToolHandler interface {
    Name() string
    Definition() ToolDefinition
    Validate(args map[string]any) error
    Execute(ctx context.Context, env ToolEnv, args map[string]any) (ToolResult, error)
}

Adding a new tool

  1. Create a new file in this package (e.g., `my_tool_handler.go`).
  2. Define a struct and implement all ToolHandler methods (Name, Definition, Validate, Execute, plus the 5 optional metadata methods).
  3. Register it in `AllTools()` in `all.go`.

See AGENTS.md for tool documentation and conventions.

Migration from legacy func-style handlers

The legacy tool system used function types directly coupled to *Agent. The new interface-based system decouples tools from the agent via ToolEnv, which provides explicit dependencies (EventBus, WorkspaceRoot, OutputWriter, etc.).

During the migration period, a dual-dispatch shim in pkg/agent/tool_definitions.go bridges both systems: when ExecuteTool() is called, it first checks the new registry via tools.GetNewToolRegistry().Lookup(name). If a handler is found there, it builds a ToolEnv from the agent context and dispatches through the new interface. If no handler exists in the new registry, it falls back to the legacy func-style handlers. This allows incremental migration without breaking existing functionality.

The subagent tools (run_subagent / run_parallel_subagents) intentionally remain in the seed registry under pkg/agent because they need *Agent access for nested runner orchestration. See pkg/agent_tools/all.go for the canonical tool list.

Shell command and file path security classifier.

This module provides string-based heuristics for classifying tool calls by risk level (SAFE, CAUTION, DANGEROUS). It is designed as a lightweight defense-in-depth layer that operates on raw command strings and path arguments WITHOUT accessing the filesystem.

Important Limitations

This classifier intentionally performs NO filesystem operations (no stat, no resolve, no symlink following). This keeps it fast and concurrency-safe, but means:

  • Symlink attacks are not detected. For example, "rm -rf build/" is classified as safe even if "build" is a symlink to "/etc" or "$HOME".
  • Relative path traversal is not resolved. "rm -rf ../important-project" bypasses all safe-directory checks because the classifier only matches the first path component literally (".." has no special meaning here).
  • Path normalization is not performed. Multiple slashes ("//"), "." segments, and case variations on case-insensitive filesystems are not normalized.
  • Environment variable expansion, glob expansion, and shell aliases are not considered. "rm -rf $BUILD_DIR" is classified as CAUTION (command substitution), not DANGEROUS, because the classifier cannot resolve the variable.
  • The classifier is prefix-based, not semantic. "rm -rf node_modules-new" is safe because it matches "rm -rf node_modules " prefix, even though the actual target is a different directory.

These limitations are acceptable because the classifier's purpose is gate-keeping for LLM-initiated operations in a workspace context — NOT a security boundary. Actual enforcement (filesystem permissions, user approval, interactive confirmation) should be handled by separate layers.

Package tools provides platform-specific JSON encoding for native builds.

Index

Constants

View Source
const (
	// MaxNumberTodosToShowFull defines maximum todos to display fully in summaries
	MaxNumberTodosToShowFull = 3
	// TidyMaxTodos is the maximum to show when many todos exist
	TidyMaxTodos = 2
)
View Source
const (
	ErrCodeRemoteFetchFailed   = "REMOTE_FETCH_FAILED"
	ErrCodeOCRNoTextDetected   = "OCR_NO_TEXT_DETECTED"
	ErrCodeVisionNotAvailable  = "VISION_NOT_AVAILABLE"
	ErrCodeVisionRequestFailed = "VISION_REQUEST_FAILED"
	ErrCodeInvalidResponse     = "INVALID_RESPONSE"
)

Error codes for vision analysis and remote operations

View Source
const (
	ErrCodeInputUnsupported    = "INPUT_UNSUPPORTED_TYPE"
	ErrCodeLocalFileNotFound   = "LOCAL_FILE_NOT_FOUND"
	ErrCodeModelDownloadNeeded = "MODEL_DOWNLOAD_NEEDED"
	ErrModelDownloadNeeded     = "PDF_OCR_MODEL_NEEDS_DOWNLOAD:"
)

Error codes for input and file handling

View Source
const (
	// EnvelopeTypePatchIn is the type for browser→container patches (user edits).
	// The browser sends this when the user makes an edit in the OPFS-backed editor.
	EnvelopeTypePatchIn = "workspace.patch_in"

	// EnvelopeTypePatchOut is the type for container→browser patches (agent writes).
	// The container sends this after every tool-call file write to keep the browser
	// in sync.
	EnvelopeTypePatchOut = "workspace.patch_out"

	// EnvelopeTypeHeartbeat is the bidirectional keep-alive ping. Sent by the
	// browser every 15 seconds; the container responds with its own heartbeat.
	EnvelopeTypeHeartbeat = "workspace.heartbeat"
)

WebSocket envelope type constants for workspace sync protocol.

View Source
const DefaultAskUserTimeout = 30 * time.Minute
View Source
const (
	ErrCodePDFProcessingFailed = "PDF_PROCESSING_FAILED"
)

Error code for PDF processing failures

Variables

View Source
var CreatePullRequestFunc func(ctx context.Context, args map[string]any) (string, error)

CreatePullRequestFunc is a function pointer set by pkg/agent at startup. It bridges the new ToolHandler interface with the legacy handleCreatePullRequest implementation that requires *Agent access.

The function signature matches the legacy handler:

handleCreatePullRequest(ctx, args) → JSON string

The agent sets this pointer during initialization, capturing the *Agent reference in a closure so the handler doesn't need direct access.

Phase 4 of SP-109 will migrate the execute logic into this package, eliminating the need for this indirection.

View Source
var ErrAskUserNoChannel = errors.New("ask_user: no interactive channel available (no WebUI client connected and stdin is not a TTY)")

ErrAskUserNoChannel is returned when no input channel is available (no WebUI client, stdin not a TTY / closed). The LLM should treat this as a hard signal to make a decision itself rather than retry.

View Source
var ErrNoInteractiveSurface = errors.New("no interactive surface available for password prompt")

ErrNoInteractiveSurface is returned when the password prompter cannot present a prompt to the user (e.g., stdin is not a TTY and no WebUI is connected).

View Source
var ListChangesFunc func(ctx context.Context, args map[string]any) (string, error)

ListChangesFunc is a function pointer set by pkg/agent at startup. It bridges the new ToolHandler interface with the legacy handleListChanges implementation that requires *Agent access.

The function signature matches the legacy handler:

handleListChanges(ctx, args) → JSON string

The agent sets this pointer during initialization, capturing the *Agent reference in a closure so the handler doesn't need direct access.

Phase 4 of SP-109 will migrate the execute logic into this package, eliminating the need for this indirection.

View Source
var MCPRefreshFunc func(ctx context.Context, args map[string]any) (string, error)

MCPRefreshFunc is a function pointer set by pkg/agent at startup. It bridges the new ToolHandler interface with the legacy handleMCPRefresh implementation that requires *Agent access.

The function signature matches the legacy handler:

handleMCPRefresh(ctx, args) → JSON string

The agent sets this pointer during initialization, capturing the *Agent reference in a closure so the handler doesn't need direct access.

Phase 4 of SP-109 will migrate the execute logic into this package, eliminating the need for this indirection.

View Source
var RecoverFileFunc func(ctx context.Context, args map[string]any) (string, error)

RecoverFileFunc is a function pointer set by pkg/agent at startup. It bridges the new ToolHandler interface with the legacy handleRecoverFile implementation that requires *Agent access.

The function signature matches the legacy handler:

handleRecoverFile(ctx, args) → JSON string

The agent sets this pointer during initialization, capturing the *Agent reference in a closure so the handler doesn't need direct access.

Phase 4 of SP-109 will migrate the execute logic into this package, eliminating the need for this indirection.

View Source
var RequestClarificationFunc func(ctx context.Context, args map[string]any) (string, error)

RequestClarificationFunc is a function pointer set by pkg/agent at startup. It bridges the new ToolHandler interface with the legacy handleRequestClarification implementation that requires *Agent access.

The function signature matches the legacy handler:

handleRequestClarification(ctx, args) → string, error

The agent sets this pointer during initialization, capturing the *Agent reference in a closure so the handler doesn't need direct access.

Phase 4 of SP-109 will migrate the execute logic into this package, eliminating the need for this indirection.

View Source
var RespondClarificationFunc func(ctx context.Context, args map[string]any) (string, error)

RespondClarificationFunc is a function pointer set by pkg/agent at startup. It bridges the new ToolHandler interface with the legacy handleRespondClarification implementation that requires *Agent access.

The function signature matches the legacy handler:

handleRespondClarification(ctx, args) → string, error

The agent sets this pointer during initialization, capturing the *Agent reference in a closure so the handler doesn't need direct access.

Phase 4 of SP-109 will migrate the execute logic into this package, eliminating the need for this indirection.

View Source
var RevertMyChangesFunc func(ctx context.Context, args map[string]any) (string, error)

RevertMyChangesFunc is a function pointer set by pkg/agent at startup. It bridges the new ToolHandler interface with the legacy handleRevertMyChanges implementation that requires *Agent access.

The function signature matches the legacy handler:

handleRevertMyChanges(ctx, args) → JSON string

The agent sets this pointer during initialization, capturing the *Agent reference in a closure so the handler doesn't need direct access.

Phase 4 of SP-109 will migrate the execute logic into this package, eliminating the need for this indirection.

View Source
var RunAutomateFunc func(ctx context.Context, args map[string]any) (string, error)

RunAutomateFunc is a function pointer set by pkg/agent at startup. It bridges the new ToolHandler interface with the legacy handleRunAutomate implementation that requires *Agent access.

The function signature matches the legacy handler:

handleRunAutomate(ctx, args) → JSON string

The agent sets this pointer during initialization, capturing the *Agent reference in a closure so the handler doesn't need direct access.

Phase 4 of SP-109 will migrate the execute logic into this package, eliminating the need for this indirection.

View Source
var RunParallelSubagentsFunc func(ctx context.Context, args map[string]any) (string, error)

RunParallelSubagentsFunc is a function pointer set by pkg/agent at startup. It bridges the new ToolHandler interface with the legacy handleRunParallelSubagents implementation that requires *Agent access.

The function signature matches the legacy handler:

handleRunParallelSubagents(ctx, args) → JSON string

The agent sets this pointer during initialization, capturing the *Agent reference in a closure so the handler doesn't need direct access.

Phase 4 of SP-109 will migrate the execute logic into this package, eliminating the need for this indirection.

View Source
var RunSubagentFunc func(ctx context.Context, args map[string]any) (string, error)

RunSubagentFunc is a function pointer set by pkg/agent at startup. It bridges the new ToolHandler interface with the legacy handleRunSubagent implementation that requires *Agent access.

The function signature matches the legacy handler:

handleRunSubagent(ctx, args) → JSON string

The agent sets this pointer during initialization, capturing the *Agent reference in a closure so the handler doesn't need direct access.

Phase 4 of SP-109 will migrate the execute logic into this package, eliminating the need for this indirection.

View Source
var ValidTodoPriorities = map[string]bool{
	"high":   true,
	"medium": true,
	"low":    true,
}

ValidTodoPriorities contains all allowable todo priority values

View Source
var ValidTodos = map[string]bool{
	"pending":     true,
	"in_progress": true,
	"completed":   true,
	"cancelled":   true,
}

ValidTodos contains all allowable todo status values

Functions

func AddVisionLatencyFallback added in v0.16.19

func AddVisionLatencyFallback(d time.Duration)

AddVisionLatencyFallback accumulates wall-clock time spent in the OCR fallback path.

func AddVisionLatencyParse added in v0.16.19

func AddVisionLatencyParse(d time.Duration)

AddVisionLatencyParse accumulates wall-clock time spent parsing the provider response.

func AddVisionLatencyRequest added in v0.16.19

func AddVisionLatencyRequest(d time.Duration)

AddVisionLatencyRequest accumulates wall-clock time spent in the provider SendVisionRequest call.

func AddVisionLatencyRetrySleep added in v0.16.19

func AddVisionLatencyRetrySleep(d time.Duration)

AddVisionLatencyRetrySleep accumulates wall-clock time spent sleeping between retry attempts.

func AnalyzeImage

func AnalyzeImage(ctx context.Context, imagePath string, analysisPrompt string, analysisMode string) (string, error)

AnalyzeImage is the tool function called by the agent for image analysis Returns a structured JSON response with metadata for robust error handling

func AppendVisionRecord added in v0.16.19

func AppendVisionRecord(rec VisionMetricsRecord)

AppendVisionRecord appends a vision metrics record to the JSONL sink. Fire-and-forget: never blocks the caller on IO.

func AskUser

func AskUser(req AskUserRequest) (string, error)

AskUser prompts the user with a question and reads input from stdin. Renders options as a numbered list when present and accepts either an index, the option label, or the option value as the response.

On a TTY with single-select options (MultiSelect == false), the options are rendered as an arrow-key picker (console.SelectList) with a trailing "Type your own answer…" item that falls through to the legacy freeform input reader. Multi-select prompts and prompts on a non-TTY stdin fall back to the numbered list + freeform text path so the tool remains scriptable.

Returns ErrAskUserNoChannel if stdin is not a TTY (background daemon, closed stdin, piped) so callers can distinguish "no input channel" from a transient I/O error.

func AskUserWithEventBus

func AskUserWithEventBus(ctx context.Context, req AskUserRequest, eventBus *events.EventBus, clientID, userID, chatID string, mgr *AskUserManager) (string, error)

AskUserWithEventBus prompts the user with a question using the event bus for WebUI mode, falling back to stdin for CLI mode.

func CheckBackgroundOutput

func CheckBackgroundOutput(ctx context.Context, sessionID string) (string, error)

CheckBackgroundOutput retrieves accumulated output for a background session. Returns JSON with session_id, status, and output fields. Works in WebUI mode (TerminalManager) and CLI mode (BackgroundProcessManager).

Equivalent to CheckBackgroundOutputWait(ctx, sessionID, 0).

func CheckBackgroundOutputWait added in v0.16.4

func CheckBackgroundOutputWait(ctx context.Context, sessionID string, waitSeconds int) (string, error)

CheckBackgroundOutputWait is like CheckBackgroundOutput but blocks (up to waitSeconds, capped at maxBackgroundWaitSeconds) until the session exits or the wait elapses, then returns the snapshot. waitSeconds <= 0 means return immediately.

A blocking wait is an LLM-side cost optimization: a 4-hour autonomous run polled every minute = ~240 round trips, each re-sending the full context. One blocking wait per 10 minutes collapses that to ~24, and an early exit returns as soon as the workflow finishes.

func CheckPDFPython3Available

func CheckPDFPython3Available() error

CheckPDFPython3Available validates that a compatible Python runtime is available for PDF processing.

func CheckStaleness added in v0.16.18

func CheckStaleness(path string) error

CheckStaleness is the convenience wrapper called by write handlers. Returns nil when no global checker is set (no-op), otherwise delegates to the checker's Check method.

func CleanupOrphanedBackgroundProcesses added in v0.16.18

func CleanupOrphanedBackgroundProcesses(baseDir string) error

CleanupOrphanedBackgroundProcesses scans the baseDir for .pid files left behind by background processes whose sprout parent exited uncleanly. For each orphaned PID, it attempts to terminate the process (SIGTERM → SIGKILL) and removes both the .pid and .output files.

Returns an error only if the baseDir itself can't be read. Individual file errors are logged but don't cause the function to return an error.

func CleanupOrphanedBackgroundProcessesWithContext added in v0.16.18

func CleanupOrphanedBackgroundProcessesWithContext(ctx context.Context, baseDir string) error

CleanupOrphanedBackgroundProcessesWithContext works like CleanupOrphanedBackgroundProcesses but accepts a context for cancellation and timeout control. PIDs are processed concurrently with a worker pool of 16 goroutines. A 5-second deadline is applied to the entire operation.

func ClearLastVisionUsage

func ClearLastVisionUsage()

ClearLastVisionUsage clears the stored vision usage information. Thread-safe.

func CreateOllamaClient

func CreateOllamaClient(model string) (api.ClientInterface, error)

CreateOllamaClient creates an Ollama client with the specified model

func CreateVisionClient

func CreateVisionClient() (api.ClientInterface, error)

CreateVisionClient creates a client capable of vision analysis

func CreateVisionClientWithModel

func CreateVisionClientWithModel(modelName string) (api.ClientInterface, error)

CreateVisionClientWithModel creates a vision client using a specific model

func CreateVisionClientWithProvider

func CreateVisionClientWithProvider(providerType api.ClientType) (api.ClientInterface, error)

CreateVisionClientWithProvider creates a vision client using the specified provider

func DefaultTaskQueuePath

func DefaultTaskQueuePath() string

DefaultTaskQueuePath returns the default path for the task queue file.

func DoVisionRetry added in v0.16.19

func DoVisionRetry(ctx context.Context, op func(ctx context.Context) error, opts RetryOptions) error

DoVisionRetry runs op with retries, respecting ctx cancellation.

The op function is called with ctx so it can be cancelled independently. Between failed attempts, DoVisionRetry sleeps with an exponential backoff (plus jitter) and checks ctx.Done() before each sleep.

Returns nil on success, or the last error after exhausting all attempts.

func EditFile

func EditFile(ctx context.Context, filePath, oldString, newString string) (string, error)

func EnsureOllamaModelTag

func EnsureOllamaModelTag(model string) string

EnsureOllamaModelTag ensures the model has a tag suffix

func ExecuteGitOperation

func ExecuteGitOperation(ctx context.Context, op GitOperation, sessionID string, commitFlowExecutor GitCommitFlowExecutor, approvalPrompter GitApprovalPrompter) (string, error)

ExecuteGitOperation executes a git operation with approval (all git operations require approval)

func ExecuteShellCommand

func ExecuteShellCommand(ctx context.Context, command string) (string, error)

ExecuteShellCommand executes a shell command with safety checks

func ExecuteShellCommandBackground

func ExecuteShellCommandBackground(ctx context.Context, command string, sessionID string) (string, error)

ExecuteShellCommandBackground runs a command in a background hidden PTY session and returns a JSON result with the session ID. Works in WebUI mode (TerminalManager) and CLI mode (BackgroundProcessManager). This is for commands that should run asynchronously without waiting for completion.

func ExecuteShellCommandWithSafety

func ExecuteShellCommandWithSafety(ctx context.Context, command string, interactiveMode bool, sessionID string, streamOutput bool) (string, error)

ExecuteShellCommandWithSafety executes a shell command with configurable safety checks. The streamOutput parameter controls whether output streams to terminal in real-time (true) or is captured silently (false, for LLM tool calls).

Native builds use os/exec; the js/wasm build routes through pkg/wasmshell. The platform-specific implementation lives in shell_native.go / shell_js.go.

func FetchURL

func FetchURL(url string, cfg *configuration.Manager) (string, error)

FetchURL fetches content from a specific URL using the webcontent fetcher. This provides direct URL access as an agent tool.

func FormatTodoPriorityError

func FormatTodoPriorityError(priority string) string

FormatTodoPriorityError returns a standardized error message for invalid priority values.

func FormatTodoStatusError

func FormatTodoStatusError(status string) string

FormatTodoStatusError returns a standardized error message for invalid status values.

func GeneratePromptForMode

func GeneratePromptForMode(mode string) string

GeneratePromptForMode creates appropriate prompts based on analysis mode

func GenerateRepoMap

func GenerateRepoMap(ctx context.Context, rootDir string, depth int, query string) (string, error)

GenerateRepoMap walks the directory tree rooted at rootDir and produces a lightweight overview of the codebase showing file paths and top-level symbols. For Go files it uses go/ast; for TS/JS/Python it uses tree-sitter via pkg/ast.

depth controls the detail level:

  • 1: directory tree with file counts per dir, no symbols
  • 2: directory tree + symbols in root-level and top-level files only (max 15 symbols per file)
  • 3 (default): full symbol listing

query, when non-empty, filters files to only those whose path or symbol names contain the query string (case-insensitive).

When the codegraph store is available and populated, it reads from the store for near-instant results on warm cache, falling back to the filesystem walk.

func GetBackgroundOutputBaseDir added in v0.16.18

func GetBackgroundOutputBaseDir() string

GetBackgroundOutputBaseDir returns the standard default baseDir path used by BackgroundProcessManager for output and PID files. Callers outside the tools package (e.g., agent startup code) can use this to locate the directory for orphan cleanup without knowing BPM internals.

func GetBaseName

func GetBaseName(path string) string

GetBaseName returns the base name of a file path

func GetCustomProviderConfig

func GetCustomProviderConfig(providerType api.ClientType) (configuration.CustomProviderConfig, bool)

GetCustomProviderConfig returns the custom provider configuration for a given type

func GetCustomVisionFallback

func GetCustomVisionFallback(providerType api.ClientType) (api.ClientType, string, bool)

GetCustomVisionFallback returns the fallback provider and model for vision

func GetCustomVisionProviders

func GetCustomVisionProviders() []api.ClientType

GetCustomVisionProviders returns a list of custom providers that support vision

func GetDefaultModelForProvider

func GetDefaultModelForProvider(providerType api.ClientType) string

GetDefaultModelForProvider returns the default model for a given provider type

func GetFileExtension

func GetFileExtension(path string) string

GetFileExtension returns the file extension (with dot) in lowercase

func GetOCRPrompt

func GetOCRPrompt() string

GetOCRPrompt returns a prompt for OCR text extraction

func GetPDFPythonExecutable

func GetPDFPythonExecutable() (string, error)

GetPDFPythonExecutable ensures a consistent per-user Python environment for PDF extraction.

func GetUIElementPrompt

func GetUIElementPrompt() string

GetUIElementPrompt returns a prompt for extracting UI elements

func GetVisionCacheStats

func GetVisionCacheStats() map[string]interface{}

GetVisionCacheStats returns statistics about vision result caching

func GetVisionModelForProvider

func GetVisionModelForProvider(providerType api.ClientType) string

GetVisionModelForProvider returns the appropriate vision model for a given provider.

Resolution order:

  1. Special-cased providers (OpenAI, Ollama) check their specific config paths, falling back to the provider JSON config's vision_model field.
  2. Custom providers check their explicit vision_model / model_name config.
  3. All other providers read from the provider JSON config via a temporary client's GetVisionModel().

Vision models are configured in the provider JSON config files in pkg/agent_providers/configs/*.json under the "vision_model" field.

func HasVisionCapability

func HasVisionCapability() bool

HasVisionCapability checks if vision processing is available

func IncVisionBatchAttempt added in v0.16.19

func IncVisionBatchAttempt()

IncVisionBatchAttempt bumps the batch attempt counter.

func IncVisionBatchHit added in v0.16.19

func IncVisionBatchHit()

IncVisionBatchHit bumps the batch cache-hit counter.

func IncVisionBatchMiss added in v0.16.19

func IncVisionBatchMiss()

IncVisionBatchMiss bumps the batch cache-miss counter.

func IncVisionBatchPartialFailure added in v0.16.19

func IncVisionBatchPartialFailure()

IncVisionBatchPartialFailure bumps the batch partial-failure counter.

func IncVisionCacheHit added in v0.16.19

func IncVisionCacheHit()

IncVisionCacheHit/Miss track cache outcomes for metrics consumers that only watch the metrics surface (not VisionCacheStats).

func IncVisionCacheMiss added in v0.16.19

func IncVisionCacheMiss()

func IncVisionEmbedCall added in v0.16.19

func IncVisionEmbedCall()

IncVisionEmbedCall bumps the embed-call counter by 1.

func IncVisionFailure added in v0.16.19

func IncVisionFailure(reason string)

IncVisionFailure classifies err into a reason bucket and increments the corresponding counter. Reason buckets:

"http_5xx"         — HTTP 5xx errors
"http_429"         — HTTP 429 Too Many Requests
"http_4xx"         — Other HTTP 4xx errors
"context_cancel"   — context.Canceled or context.DeadlineExceeded
"network"          — net.Error (timeout or temporary)
"timeout"          — syscall.ETIMEDOUT
"invalid_response" — empty or unparseable provider response
"ocr_no_text"      — OCR fallback returned no text
"unknown"          — everything else

func IncVisionFallbackSuccess added in v0.16.19

func IncVisionFallbackSuccess()

IncVisionFallbackSuccess bumps the OCR-fallback success counter.

func IncVisionFallbackTotal added in v0.16.19

func IncVisionFallbackTotal()

IncVisionFallbackTotal bumps the OCR-fallback attempt counter.

func IncVisionImageTokens added in v0.16.19

func IncVisionImageTokens(delta int, deltaCached int)

IncVisionImageTokens adds delta to the image-tokens counter. deltaCached is the portion of delta that was served from cache (so the discounted-rate bucket is updated separately).

func IncVisionOCRCall added in v0.16.19

func IncVisionOCRCall()

IncVisionOCRCall bumps the OCR-call counter by 1.

func IncVisionResizeEvent added in v0.16.19

func IncVisionResizeEvent()

IncVisionResizeEvent records that we resized one image down to embed.

func IncVisionRetry added in v0.16.19

func IncVisionRetry()

IncVisionRetry bumps the retry counter by 1. Called each time DoVisionRetry loops back for another attempt.

func IsFileDeletionCommand

func IsFileDeletionCommand(command string) bool

IsFileDeletionCommand checks if a command will delete files This is used for change tracking (not security validation) Security validation is handled by the static classifier in security_classifier.go

func IsHTMLInput

func IsHTMLInput(path string) bool

IsHTMLInput checks if the input path appears to be HTML content. For URLs, it does a HEAD request to check Content-Type. For local files, it checks the file extension.

func IsRemoteSizeExceededError added in v0.16.19

func IsRemoteSizeExceededError(err error) bool

IsRemoteSizeExceededError reports whether err (or any wrapped error in its chain) is a *remoteSizeExceededError.

func IsValidPriority

func IsValidPriority(priority string) bool

IsValidPriority checks if the given priority string is valid. Empty string is accepted (priority is optional).

func IsValidStatus

func IsValidStatus(status string) bool

IsValidStatus checks if the given status string is valid.

func NormalizeTodoID

func NormalizeTodoID(id interface{}) string

NormalizeTodoID converts various ID formats to the internal "todo_X" format. Accepted inputs:

  • string: "todo_1" -> "todo_1", "1" -> "todo_1"
  • float64: 1.0 -> "todo_1"
  • int: 1 -> "todo_1"

Returns empty string for unsupported types.

func OptimizeImageData

func OptimizeImageData(imagePath string, data []byte) ([]byte, string, error)

func ProcessPDFForTextOnly

func ProcessPDFForTextOnly(ctx context.Context, pdfPath string) (string, error)

ProcessPDFForTextOnly extracts text from a PDF using Go-native extraction. Falls back to page-rasterization OCR if no text is found.

func ProcessPDFWithVision

func ProcessPDFWithVision(ctx context.Context, pdfPath string) (string, error)

ProcessPDFWithVision processes a PDF file. Delegates to ProcessPDFForTextOnly.

func PromptForGitApprovalStdin

func PromptForGitApprovalStdin(command string) (bool, error)

PromptForGitApprovalStdin prompts for git approval using stdin. Fires mid-turn during git tool execution, so it pauses the SteerInputReader to release stdin back to cooked mode (otherwise the bufio.Reader hits EOF immediately while steer holds the raw- mode fd).

func ReadFile

func ReadFile(ctx context.Context, filePath string) (string, error)

func ReadFileWithRange

func ReadFileWithRange(ctx context.Context, filePath string, startLine, endLine int) (string, error)

func RenderTodosForCLI added in v0.16.4

func RenderTodosForCLI(w io.Writer, todos []TodoItem)

RenderTodosForCLI writes a bar-wrapped block summarizing the todo list to w, so CLI users see progress without having to read the LLM's structured tool output. Mirrors the visual treatment used by the ask_user CLI prompt (renderCLIPrompt) so the two surfaces feel like one family. Safe to call with an empty list — prints a "cleared" marker so the user knows the agent intentionally wiped the list.

func ResetTodoManagerForChat added in v0.16.4

func ResetTodoManagerForChat(chatID string)

ResetTodoManagerForChat clears a chat's todo list (used by chat-end / session-reset flows). Safe to call with an unknown chat_id.

func ResolvePDFInputPath

func ResolvePDFInputPath(ctx context.Context, inputPath string) (string, func(), error)

func SetAuditLogger

func SetAuditLogger(l *AuditLogger)

SetAuditLogger sets the package-level audit logger for recording security decisions. Must be called during initialization before concurrent goroutines begin calling ClassifyToolCall.

func SetGlobalAskUserManager deprecated

func SetGlobalAskUserManager(mgr *AskUserManager)

SetGlobalAskUserManager sets the global singleton (called by webui setup).

Deprecated: use dependency injection via Agent.InjectWebUIManagers instead.

func SetGlobalStalenessChecker added in v0.16.18

func SetGlobalStalenessChecker(checker *StalenessChecker)

SetGlobalStalenessChecker installs the global checker. Existing code that doesn't set a checker continues to work (CheckStaleness is a no-op).

func SimplePDFInfo

func SimplePDFInfo(pdfPath string) (map[string]interface{}, error)

func TodoWrite

func TodoWrite(todos []TodoItem) string

TodoWrite is a convenience wrapper that writes todos to the default scope.

func ValidTodoPriorityList

func ValidTodoPriorityList() []string

ValidTodoPriorityList returns a slice of all valid priority values for error messages

func ValidTodoStatuses

func ValidTodoStatuses() []string

ValidTodoStatuses returns a slice of all valid status values for error messages

func ValidateGitArgs

func ValidateGitArgs(args string) error

ValidateGitArgs validates that the provided git arguments string does not contain any dangerous flags or patterns.

It uses a combination of matching strategies:

  • Field prefix matching: splits args into whitespace-delimited tokens and checks if any token starts with a blocklisted prefix. Catches abbreviations.
  • Substring matching: for multi-token patterns like "-c core.", checks containment in the full args string.

Returns nil if all arguments are safe, or an error describing which flag was blocked and why.

func WebSearch

func WebSearch(query string, cfg *configuration.Manager) (string, error)

WebSearch performs a web search and returns raw search results. The agent can then decide which URLs to fetch and process.

func WithBackgroundProcessManager

func WithBackgroundProcessManager(ctx context.Context, bpm *BackgroundProcessManager) context.Context

WithBackgroundProcessManager returns a new context that carries the BackgroundProcessManager. Use BackgroundProcessManagerFromContext to retrieve it.

func WithPasswordPrompter added in v0.16.18

func WithPasswordPrompter(ctx context.Context, pp PasswordPrompter) context.Context

WithPasswordPrompter returns a new context that carries the PasswordPrompter. Use PasswordPrompterFromContext to retrieve it.

func WithTerminalManager

func WithTerminalManager(ctx context.Context, tm TerminalAccess) context.Context

WithTerminalManager returns a new context that carries the TerminalAccess. Use TerminalManagerFromContext to retrieve it.

func WriteFile

func WriteFile(ctx context.Context, filePath, content string) (string, error)

Types

type ApprovalManager

type ApprovalManager interface {
	// RequestApproval asks the user to approve a tool execution.
	// Returns an ApprovalResult with the outcome and optional context.
	RequestApproval(requestID, toolName, riskLevel, prompt string, extras map[string]string) ApprovalResult
}

ApprovalManager handles security approval requests for tool execution.

type ApprovalResult

type ApprovalResult struct {
	Approved    bool   `json:"approved"`
	Reason      string `json:"reason,omitempty"`       // "rejected", "timed_out", "cancelled"
	UserComment string `json:"user_comment,omitempty"` // Optional feedback from user
}

ApprovalResult contains the outcome of an approval request.

type AskUserManager

type AskUserManager struct {
	// contains filtered or unexported fields
}

AskUserManager coordinates ask_user requests between the agent and the webui. It follows the same pattern as security.ApprovalManager but returns string responses instead of bool.

func GetGlobalAskUserManager deprecated

func GetGlobalAskUserManager() *AskUserManager

GetGlobalAskUserManager returns the global singleton.

Deprecated: use dependency injection via Agent.InjectWebUIManagers instead.

func NewAskUserManager

func NewAskUserManager() *AskUserManager

NewAskUserManager creates a new AskUserManager with the default timeout.

func (*AskUserManager) RequestAskUser

func (m *AskUserManager) RequestAskUser(ctx context.Context, eventBus *events.EventBus, req AskUserRequest, clientID, userID, chatID string) (string, error)

RequestAskUser publishes an ask_user_request event and blocks until the webui responds, a timeout elapses, the context is cancelled, or the event bus is nil. Returns the user's text response.

func (*AskUserManager) RespondToAskUser

func (m *AskUserManager) RespondToAskUser(requestID string, response string) bool

RespondToAskUser resolves a pending ask_user request with the user's text response. Returns true if the request existed and was responded to, false otherwise.

func (*AskUserManager) SetTimeout

func (m *AskUserManager) SetTimeout(d time.Duration)

SetTimeout sets the maximum duration requests will block. A zero or negative value resets to the default.

type AskUserOption added in v0.16.4

type AskUserOption struct {
	Label       string `json:"label"`
	Value       string `json:"value,omitempty"`
	Description string `json:"description,omitempty"`
}

AskUserOption is a single selectable choice in a structured ask_user request. When Value is empty the response carries Label verbatim.

type AskUserRequest added in v0.16.4

type AskUserRequest struct {
	Question    string          `json:"question"`
	Header      string          `json:"header,omitempty"`
	Options     []AskUserOption `json:"options,omitempty"`
	MultiSelect bool            `json:"multi_select,omitempty"`
	Default     string          `json:"default,omitempty"`
}

AskUserRequest carries the full prompt payload from the tool layer to the CLI / WebUI renderer. Only Question is required.

type AskUserService added in v0.16.4

type AskUserService interface {
	// Ask presents req to the user and returns their response. Returns
	// ErrAskUserNoChannel when no input channel is available so callers
	// can surface a structured error to the LLM.
	Ask(ctx context.Context, req AskUserRequest) (string, error)
}

AskUserService is the interface ask_user-style tools use to drive an interactive prompt. Implementations decide between WebUI routing (event bus + AskUserManager) and CLI stdin fallback based on whether a browser client is connected. ToolEnv.AskUser is populated by the agent at dispatch time so the tool handler doesn't need *Agent.

type AuditEntry

type AuditEntry struct {
	Timestamp time.Time `json:"timestamp"`
	Tool      string    `json:"tool"`
	Args      string    `json:"args,omitempty"`
	RiskLevel string    `json:"risk_level"`
	Category  string    `json:"category"`
	Action    string    `json:"action"` // "allowed", "denied", "prompted"
	Reasoning string    `json:"reasoning,omitempty"`
	Source    string    `json:"source,omitempty"` // "classifier", "policy", "user_override"
	SessionID string    `json:"session_id,omitempty"`
	Workspace string    `json:"workspace,omitempty"`
}

AuditEntry represents a single security audit log entry.

type AuditLogger

type AuditLogger struct {
	// contains filtered or unexported fields
}

AuditLogger provides thread-safe JSONL audit logging for security decisions.

func NewAuditLogger

func NewAuditLogger(logPath string) (*AuditLogger, error)

NewAuditLogger creates or opens a log file at the given path, automatically creating parent directories as needed.

func (*AuditLogger) Close

func (l *AuditLogger) Close() error

Close closes the underlying log file.

func (*AuditLogger) Log

func (l *AuditLogger) Log(entry AuditEntry) error

Log marshals the entry to JSON and appends it as a single line (JSONL/NDJSON format) followed by a newline.

func (*AuditLogger) LogEntry

func (l *AuditLogger) LogEntry(entry AuditEntry) error

LogEntry is an alias for Log, named for call-site clarity. Nil-receiver safe via Log's internal nil guard.

type BackgroundNotifier added in v0.16.19

type BackgroundNotifier interface {
	NotifyCompletion(sessionID, kind, content string)
}

BackgroundNotifier is the interface tools use to queue a background completion notification. The agent (pkg/agent) implements this so tool handlers don't need *Agent access.

type BackgroundProcess

type BackgroundProcess struct {
	ID         string    // "bg-<sanitized-prefix>-<random-hex>"
	Cmd        *exec.Cmd // the running process (nil after exit)
	Process    *os.Process
	OutputPath string // path to accumulated output temp file
	Dir        string // working directory
	Command    string // original command string
	Kind       string // "shell" (default), "automate", etc.
	StartedAt  time.Time
	LastPolled time.Time
	// contains filtered or unexported fields
}

BackgroundProcess represents a tracked background process for CLI mode. Unlike WebUI background sessions (PTY-based), these use os/exec with output piped to a temp file for polling via check_background.

func (*BackgroundProcess) Done added in v0.16.4

func (p *BackgroundProcess) Done() <-chan struct{}

Done returns a channel that closes when the background process exits. Callers can select on this channel to wait for process completion. If the process has already exited, the returned channel is already closed.

func (*BackgroundProcess) GetExitCode added in v0.16.4

func (p *BackgroundProcess) GetExitCode() int

GetExitCode returns the exit code of the background process. Returns -1 if the process has not yet exited.

func (*BackgroundProcess) GetOutputPath added in v0.16.4

func (p *BackgroundProcess) GetOutputPath() string

GetOutputPath returns the output file path under the lock.

func (*BackgroundProcess) GetPID added in v0.16.4

func (p *BackgroundProcess) GetPID() int

GetPID returns the process PID under the lock. Returns 0 if the process is nil (not yet started or already exited).

type BackgroundProcessManager

type BackgroundProcessManager struct {
	// contains filtered or unexported fields
}

BackgroundProcessManager manages background processes for CLI mode. Provides the same lifecycle as the WebUI's TerminalManager background sessions but without PTY support.

func BackgroundProcessManagerFromContext

func BackgroundProcessManagerFromContext(ctx context.Context) *BackgroundProcessManager

BackgroundProcessManagerFromContext extracts the BackgroundProcessManager from the context. Returns nil if no manager is available.

func NewBackgroundProcessManager

func NewBackgroundProcessManager() *BackgroundProcessManager

NewBackgroundProcessManager creates a new BackgroundProcessManager and starts the cleanup goroutine.

func (*BackgroundProcessManager) AdoptProcess

func (m *BackgroundProcessManager) AdoptProcess(cmd *exec.Cmd, outputPath string, command string, dir string, waitCh <-chan error) (string, error)

AdoptProcess takes an already-started exec.Cmd (from timeout promotion) and registers it into the background process manager. The output file is already created by the caller.

If waitCh is non-nil, AdoptProcess assumes the caller has already started a goroutine calling cmd.Wait() and reads its result from waitCh instead of calling cmd.Wait() itself. Calling cmd.Wait() concurrently from two goroutines on the same exec.Cmd is undefined behavior and trips the race detector. The shell-promotion path uses this to hand off its existing Wait goroutine. Callers that haven't yet started a Wait (e.g. tests) pass nil and AdoptProcess starts one internally.

func (*BackgroundProcessManager) CheckOutput

func (m *BackgroundProcessManager) CheckOutput(sessionID string) (string, string, error)

CheckOutput reads accumulated output from a background session. Returns the raw output string, status ("running" or "exited"), and any error.

func (*BackgroundProcessManager) Close

func (m *BackgroundProcessManager) Close()

Close stops the cleanup goroutine and terminates all background processes.

func (*BackgroundProcessManager) GetBaseDir added in v0.16.18

func (m *BackgroundProcessManager) GetBaseDir() string

GetBaseDir returns the base directory used for output and PID files.

func (*BackgroundProcessManager) GetProcess added in v0.16.4

func (m *BackgroundProcessManager) GetProcess(sessionID string) (*BackgroundProcess, bool)

GetProcess returns a BackgroundProcess by its session ID. Returns the process and true if found, or nil and false otherwise.

The returned pointer must not be accessed without first acquiring proc.mu.Lock() or proc.mu.RLock(). The BackgroundProcessManager does not keep the process in the map permanently — cleanup may remove entries at any time. Acquire proc.mu immediately after calling GetProcess.

func (*BackgroundProcessManager) IsActive

func (m *BackgroundProcessManager) IsActive(sessionID string) bool

IsActive checks whether a session is still running.

func (*BackgroundProcessManager) SessionIDs

func (m *BackgroundProcessManager) SessionIDs() []string

SessionIDs returns all tracked session IDs.

func (*BackgroundProcessManager) Start

func (m *BackgroundProcessManager) Start(ctx context.Context, command string, dir string) (string, error)

Start creates a new background process, pipes its output to a temp file, and returns a session ID for later polling.

func (*BackgroundProcessManager) StartWithKind added in v0.16.4

func (m *BackgroundProcessManager) StartWithKind(ctx context.Context, command string, dir string, kind string) (string, error)

StartWithKind works like Start but allows specifying the process kind (e.g., "automate" vs "shell").

func (*BackgroundProcessManager) StartWithOptions added in v0.16.4

func (m *BackgroundProcessManager) StartWithOptions(ctx context.Context, command string, dir string, kind string, opts *StartOptions) (string, error)

StartWithOptions works like StartWithKind but also accepts options that control output streaming. When kind == "automate" and opts.EventBus is non-nil, output is teed through an OutputChunkPublisher that emits automate.output_chunk events on a coalesced basis (≥250ms or ≥4KB).

func (*BackgroundProcessManager) Stop

func (m *BackgroundProcessManager) Stop(sessionID string, grace time.Duration) error

Stop terminates a background session using a graduated signal sequence: SIGINT → wait for grace period → SIGTERM → wait 5s → SIGKILL if still alive.

func (*BackgroundProcessManager) StopAll

func (m *BackgroundProcessManager) StopAll()

StopAll terminates all managed background processes.

type BatchVisionRequest added in v0.16.19

type BatchVisionRequest struct {
	Images  [][]byte
	Prompts []string
	Mode    string // prompt template mode; ignored if Prompts is non-empty
}

BatchVisionRequest holds the inputs for a batched vision analysis call. Images are raw bytes (not base64); they are encoded internally. Prompts can be one per image (len(Prompts) == len(Images)) or a single shared prompt (len(Prompts) == 1) that applies to all images.

type BatchVisionResult added in v0.16.19

type BatchVisionResult struct {
	Results       []VisionAnalysis
	CombinedUsage *VisionUsageInfo
}

BatchVisionResult holds the per-image analyses from a batched call. Results[i] corresponds to the i-th image in the request.

func AnalyzeImagesBatched added in v0.16.19

func AnalyzeImagesBatched(ctx context.Context, client api.ClientInterface, req BatchVisionRequest) (*BatchVisionResult, error)

AnalyzeImagesBatched sends ONE provider request containing all images, parses the response into N per-image analyses, and caches the result.

Cache key: image hashes (in original order) + prompt hash, prefixed with "batch:". On per-image failure (missing/empty section in response), falls back to single-image processing for that image only.

Returns a TypedError if the client is nil with error code "validation".

type BinaryFetchResult

type BinaryFetchResult struct {
	Images       []api.ImageData // Populated for image URLs (and scanned PDFs)
	Text         string          // Populated for text-based PDFs
	Source       string          // Description of how content was obtained
	EffectiveURL string          // Post-redirect URL (differs from input if redirected)
}

BinaryFetchResult holds the result of fetching binary content from a URL. Exactly one of Images or Text will be meaningfully populated.

func FetchBinaryURL

func FetchBinaryURL(ctx context.Context, url string, kind ResponseKind) (*BinaryFetchResult, error)

FetchBinaryURL downloads binary content from a URL and processes it for multimodal consumption based on the detected content type. The ctx is threaded through the HTTP request and downstream PDF processing so the Stop button can abort in-flight fetches (SP-034-1c).

type ConflictResult added in v0.16.18

type ConflictResult struct {
	// Path is the original file path that had the conflict.
	Path string `json:"path"`
	// TheirsPath is the <path>.theirs sibling file location.
	TheirsPath string `json:"theirs_path"`
	// HashContainer is the SHA-256 hex digest of the container's content.
	HashContainer string `json:"hash_container"`
	// HashBrowser is the SHA-256 hex digest of the browser's current content.
	HashBrowser string `json:"hash_browser"`
	// Message is a human-readable explanation.
	Message string `json:"message"`
}

ConflictResult is returned when a container patch conflicts with unsynced browser edits. The container's content is safely written as a .theirs sibling instead of overwriting the browser's version.

type EventPublisher added in v0.16.18

type EventPublisher interface {
	Publish(eventType string, data any)
}

EventPublisher is the minimal interface satisfied by events.EventBus. Defined locally to avoid an import-cycle dependency from agent_tools → events (agent_tools is used by both the daemon and the WASM browser build).

type FileMetadata added in v0.16.18

type FileMetadata struct {
	// BrowserSeq is the latest browser-originated sequence number for this file.
	// Bumped each time the user makes an edit in the browser editor.
	BrowserSeq int64 `json:"browser_seq"`

	// ContainerSeq is the latest container-originated sequence number for this file.
	// Bumped each time the agent writes to this file via a tool call.
	ContainerSeq int64 `json:"container_seq"`

	// LastSyncedBrowser is the browser_seq value that the container has last
	// observed. When BrowserSeq > LastSyncedBrowser, the browser has unsynced
	// edits from the container's perspective.
	LastSyncedBrowser int64 `json:"last_synced_browser"`

	// LastSyncedContainer is the container_seq value that the browser has last
	// observed. When ContainerSeq > LastSyncedContainer, the container has
	// unsynced writes from the browser's perspective.
	LastSyncedContainer int64 `json:"last_synced_container"`

	// ModifiedAt is the last time any sync-relevant change occurred for this file.
	ModifiedAt time.Time `json:"modified_at"`
}

FileMetadata tracks sync state for a single file between browser (OPFS) and container replicas. Both sides hold their own sequence counters; the last synced counters record what has been reconciled in each direction.

@ts-generated — consumed by the frontend to generate a TypeScript interface.

type GitApprovalPrompter

type GitApprovalPrompter interface {
	PromptForApproval(command string) (bool, error)
}

GitApprovalPrompter is an interface for prompting the user for approval This avoids importing the agent package and creating import cycles

type GitCommitFlowExecutor

type GitCommitFlowExecutor interface {
	ExecuteGitCommitFlow() (string, error)
}

GitCommitFlowExecutor is an interface for executing the commit flow This allows the git tool to delegate commit operations without creating import cycles

type GitOperation

type GitOperation struct {
	Operation GitOperationType `json:"operation"`
	Args      string           `json:"args,omitempty"`
}

GitOperation defines a git operation request

type GitOperationType

type GitOperationType string

GitOperationType defines the type of git operation

const (
	GitOpCommit       GitOperationType = "commit"
	GitOpPush         GitOperationType = "push"
	GitOpAdd          GitOperationType = "add"
	GitOpRm           GitOperationType = "rm"
	GitOpMv           GitOperationType = "mv"
	GitOpReset        GitOperationType = "reset"
	GitOpRebase       GitOperationType = "rebase"
	GitOpMerge        GitOperationType = "merge"
	GitOpCheckout     GitOperationType = "checkout"
	GitOpBranchDelete GitOperationType = "branch_delete"
	GitOpTag          GitOperationType = "tag"
	GitOpClean        GitOperationType = "clean"
	GitOpStash        GitOperationType = "stash"
	GitOpAm           GitOperationType = "am"
	GitOpApply        GitOperationType = "apply"
	GitOpCherryPick   GitOperationType = "cherry_pick"
	GitOpRevert       GitOperationType = "revert"
	GitOpPull         GitOperationType = "pull"
	GitOpFetch        GitOperationType = "fetch"
	GitOpRestore      GitOperationType = "restore"
)

type HeartbeatLostError added in v0.16.18

type HeartbeatLostError struct {
	SessionID     string
	LastHeartbeat time.Time
}

HeartbeatLostError is returned when a heartbeat has been missed for the configured threshold. Used by callers to detect abandonment without relying solely on the event bus.

func (*HeartbeatLostError) Error added in v0.16.18

func (e *HeartbeatLostError) Error() string

type HeartbeatMonitor added in v0.16.18

type HeartbeatMonitor struct {
	// contains filtered or unexported fields
}

HeartbeatMonitor tracks heartbeat pings from browser sessions and automatically terminates abandoned jobs after a configurable threshold.

The monitor runs a background goroutine (started via StartMonitor) that periodically checks all registered sessions. When a session's last heartbeat exceeds the threshold, the monitor:

  1. Publishes an EventTypeWorkspaceHeartbeatLost event (if publisher is set)
  2. Calls the session's JobTerminated callback (if registered)
  3. Removes the session from the map

func NewHeartbeatMonitor added in v0.16.18

func NewHeartbeatMonitor(publisher EventPublisher) *HeartbeatMonitor

NewHeartbeatMonitor creates a new HeartbeatMonitor with the given event publisher. The publisher can be nil (the monitor will still track sessions but won't emit events on timeout).

func (*HeartbeatMonitor) GetActiveCount added in v0.16.18

func (m *HeartbeatMonitor) GetActiveCount() int

GetActiveCount returns the number of currently tracked sessions. Thread-safe.

func (*HeartbeatMonitor) GetSession added in v0.16.18

func (m *HeartbeatMonitor) GetSession(sessionID string) *SessionHeartbeat

GetSession returns a copy of the heartbeat state for a session, or nil if the session is not tracked. Thread-safe.

func (*HeartbeatMonitor) GetSessionIDs added in v0.16.18

func (m *HeartbeatMonitor) GetSessionIDs() []string

GetSessionIDs returns a snapshot of all tracked session IDs. Thread-safe.

func (*HeartbeatMonitor) RecordHeartbeat added in v0.16.18

func (m *HeartbeatMonitor) RecordHeartbeat(sessionID string, ts time.Time)

RecordHeartbeat records a heartbeat timestamp for the given session. If the session doesn't exist yet, it is created with JobTerminated set to nil. Thread-safe via mutex.

func (*HeartbeatMonitor) RegisterJob added in v0.16.18

func (m *HeartbeatMonitor) RegisterJob(sessionID string, terminate func(sessionID string))

RegisterJob registers a job termination callback for a session. When the heartbeat threshold is exceeded for this session, the callback will be invoked with the sessionID. If the session doesn't exist yet, it is created with LastHeartbeat set to the current time. Thread-safe.

func (*HeartbeatMonitor) RemoveSession added in v0.16.18

func (m *HeartbeatMonitor) RemoveSession(sessionID string)

RemoveSession removes a session from the monitor without emitting an event or calling the termination callback. Use this for normal session teardown (e.g., job completes successfully). Thread-safe.

func (*HeartbeatMonitor) StartMonitor added in v0.16.18

func (m *HeartbeatMonitor) StartMonitor(interval, threshold time.Duration)

StartMonitor begins the background goroutine that periodically checks all registered sessions for missed heartbeats. The interval parameter controls how often the check runs (e.g. 15s in production), and the threshold defines how long without a heartbeat before a session is considered abandoned (e.g. 60s).

The goroutine runs until Stop() is called. Calling StartMonitor multiple times is safe — only one monitor goroutine runs at a time.

func (*HeartbeatMonitor) Stop added in v0.16.18

func (m *HeartbeatMonitor) Stop()

Stop signals the monitor goroutine to shut down. Safe to call multiple times — subsequent calls after the first are no-ops.

type HeartbeatPayload added in v0.16.18

type HeartbeatPayload struct {
	// Timestamp is the server-side or client-side time of the heartbeat.
	Timestamp string `json:"timestamp"`
}

HeartbeatPayload carries the data for a heartbeat envelope.

@ts-generated — consumed by the frontend to generate a TypeScript interface.

type ImageAnalysisResponse

type ImageAnalysisResponse struct {
	Success         bool                   `json:"success"`
	ToolInvoked     bool                   `json:"tool_invoked"`
	InputResolved   bool                   `json:"input_resolved"`
	OCRAttempted    bool                   `json:"ocr_attempted"`
	InputType       string                 `json:"input_type"` // "local_file", "remote_url", "unknown"
	InputPath       string                 `json:"input_path"`
	ErrorCode       string                 `json:"error_code,omitempty"`
	ErrorMessage    string                 `json:"error_message,omitempty"`
	ExtractedText   string                 `json:"extracted_text,omitempty"`
	OutputTruncated bool                   `json:"output_truncated,omitempty"`
	OriginalChars   int                    `json:"original_chars,omitempty"`
	ReturnedChars   int                    `json:"returned_chars,omitempty"`
	FullOutputPath  string                 `json:"full_output_path,omitempty"` // Path to full OCR/analysis text when truncated
	Analysis        *VisionAnalysis        `json:"analysis,omitempty"`
	SupportedInput  ImageAnalysisSupported `json:"supported_input"`
}

ImageAnalysisResponse represents a structured response for the analyze_image_content tool

type ImageAnalysisSupported

type ImageAnalysisSupported struct {
	RemoteURL     bool   `json:"remote_url"`
	LocalFile     bool   `json:"local_file"`
	ImageFormats  bool   `json:"image_formats"`  // jpg, png, gif, webp, etc.
	PDFSupport    bool   `json:"pdf_support"`    // PDF support status
	PDFWorkaround string `json:"pdf_workaround"` // Instructions for PDF handling
	MaxFileSizeMB int    `json:"max_file_size_mb"`
}

ImageAnalysisSupported describes what input types are supported

type ImageData

type ImageData struct {
	// URI is the path or data URI of the image
	URI string `json:"uri"`
	// Base64 is the base64-encoded image data (for inline multimodal attachment)
	Base64 string `json:"base64,omitempty"`
	// MIMEType is the image MIME type (e.g., "image/png")
	MIMEType string `json:"mime_type"`
}

ImageData represents an image returned by a vision-capable tool.

type OutputChunkPublisher added in v0.16.4

type OutputChunkPublisher struct {
	// contains filtered or unexported fields
}

OutputChunkPublisher implements io.Writer. It accumulates bytes from a background process's stdout/stderr and publishes automate.output_chunk events on a time-and-size coalesced basis (≥250ms or ≥4KB) so that WebSocket frames aren't overwhelmed by rapid small writes.

func NewOutputChunkPublisher added in v0.16.4

func NewOutputChunkPublisher(sessionID string, eventBus *events.EventBus) *OutputChunkPublisher

NewOutputChunkPublisher creates a publisher that streams output-chunk events for the given session ID via the provided event bus.

func (*OutputChunkPublisher) Flush added in v0.16.4

func (p *OutputChunkPublisher) Flush()

Flush publishes any remaining accumulated bytes. Call this when the backing process exits so the last bits of output reach subscribers. Safe to call when there is nothing to flush (no-op).

func (*OutputChunkPublisher) Write added in v0.16.4

func (p *OutputChunkPublisher) Write(data []byte) (int, error)

Write accumulates bytes from the writer chain. It triggers a publish event when the coalescing thresholds are met (≥250ms since last publish or ≥4KB accumulated since last publish). Implements io.Writer.

type PDFPipelineResult

type PDFPipelineResult struct {
	Text   string
	Images []api.ImageData
	Source string
}

func ProcessPDFForMultimodal

func ProcessPDFForMultimodal(ctx context.Context, pdfPath string) (*PDFPipelineResult, error)

type ParameterDef

type ParameterDef struct {
	Name        string `json:"name"`
	Type        string `json:"type"`
	Required    bool   `json:"required"`
	Description string `json:"description"`
}

ParameterDef defines a single tool parameter's schema.

type PasswordPrompter added in v0.16.18

type PasswordPrompter interface {
	// Prompt asks the user to type a password and returns it without a
	// trailing newline. The reason is a human-readable description shown to
	// the user (e.g., "sudo apt update needs your password").
	//
	// Returns ErrNoInteractiveSurface when there is no way to prompt the
	// user (non-TTY stdin, no WebUI client, etc.).
	Prompt(ctx context.Context, reason string) (string, error)
}

PasswordPrompter handles interactive password prompts during shell command execution. The interface is defined in pkg/agent_tools (the consumer package) so that both pkg/agent_tools (shell tool) and pkg/agent (broker + CLI impl) can reference it without import cycles. Implementors in other packages satisfy it structurally — no explicit import is needed.

func PasswordPrompterFromContext added in v0.16.18

func PasswordPrompterFromContext(ctx context.Context) PasswordPrompter

PasswordPrompterFromContext extracts the PasswordPrompter from the context. Returns nil if no prompter is available.

type PatchEvent added in v0.16.18

type PatchEvent struct {
	// Path is the workspace-relative file path (e.g. "pkg/foo/bar.go").
	Path string `json:"path"`

	// ContainerSeq is the new container sequence number for this file after the
	// agent's write.
	ContainerSeq int64 `json:"container_seq"`

	// Content is the full file content after the agent's write. For the first
	// pass, patches are whole-file replaces.
	Content string `json:"content"`

	// BaseBrowserSeq is the browser_seq value the container observed before
	// applying this write. Used for staleness detection.
	BaseBrowserSeq int64 `json:"base_browser_seq"`
}

PatchEvent represents a file change from one replica to the other.

@ts-generated — consumed by the frontend to generate a TypeScript interface.

type PatchInPayload added in v0.16.18

type PatchInPayload struct {
	// Path is the workspace-relative file path (e.g. "pkg/foo/bar.go").
	Path string `json:"path"`

	// Content is the full file content after the browser edit.
	Content string `json:"content"`

	// BrowserSeq is the browser's sequence number after this edit.
	BrowserSeq int64 `json:"browser_seq"`

	// LastSyncedContainer is the last container_seq the browser has seen for
	// this file. Used for staleness detection on the server side.
	LastSyncedContainer int64 `json:"last_synced_container"`
}

PatchInPayload carries the data for a browser→container patch.

@ts-generated — consumed by the frontend to generate a TypeScript interface.

type ResponseKind

type ResponseKind int

ResponseKind classifies what kind of content a URL serves.

const (
	ResponseKindUnknown ResponseKind = iota // Unable to determine or unsupported
	ResponseKindText                        // HTML, JSON, XML, plain text, etc.
	ResponseKindImage                       // PNG, JPEG, GIF, WebP, BMP, AVIF
	ResponseKindPDF                         // application/pdf
)

func ClassifyContentType

func ClassifyContentType(contentType string, urlPath string) ResponseKind

ClassifyContentType maps a Content-Type header value to a ResponseKind. Falls back to URL path extension when the header is ambiguous.

func ProbeURLContentType

func ProbeURLContentType(url string) (ResponseKind, string)

ProbeURLContentType sends a HEAD request to determine the kind of content a URL serves. Falls back to URL path extension if the HEAD request fails. Returns both the ResponseKind and the effective URL (after redirects).

func (ResponseKind) IsBinary

func (k ResponseKind) IsBinary() bool

IsBinary returns true if the ResponseKind represents binary content that should go through the multimodal pipeline rather than text extraction.

func (ResponseKind) String

func (k ResponseKind) String() string

String returns a human-readable name for debugging.

type RetryOptions added in v0.16.19

type RetryOptions struct {
	MaxAttempts int              // total attempts (including first); 1 disables; 0 falls back to default
	BaseDelay   time.Duration    // base for exponential backoff (200ms default)
	MaxDelay    time.Duration    // cap on backoff (1600ms default)
	JitterPct   int              // ± jitter percent (20 default)
	IsRetryable func(error) bool // optional classifier; uses default if nil
	OpName      string           // for logging
	// Stats is an optional output pointer. If non-nil, DoVisionRetry
	// populates it with per-call retry statistics (retry count, total
	// sleep time, last error). Safe for use by callers that need per-call
	// metrics for JSONL records.
	Stats *RetryStats
}

RetryOptions configures DoVisionRetry.

type RetryStats added in v0.16.19

type RetryStats struct {
	RetryCount    int           // number of retry attempts (0 = first attempt succeeded)
	SleepDuration time.Duration // total time spent sleeping between retries
	LastError     error         // last error (nil on success)
}

RetryStats captures per-call retry statistics populated by DoVisionRetry.

type RetryableHTTPError added in v0.16.19

type RetryableHTTPError struct {
	StatusCode int
	Status     string
	Method     string
	URL        string
	RetryAfter time.Duration // 0 means server didn't provide one
	Err        error         // underlying cause (for HTTP errors wrapping a network failure)
}

RetryableHTTPError describes a retryable HTTP failure with optional server-supplied retry hints (Retry-After header, parsed as a duration).

func IsRetryableHTTPError added in v0.16.19

func IsRetryableHTTPError(err error) (*RetryableHTTPError, bool)

IsRetryableHTTPError reports whether err is a RetryableHTTPError that should be retried. It returns the unwrapped error and true if so.

func (*RetryableHTTPError) Error added in v0.16.19

func (e *RetryableHTTPError) Error() string

func (*RetryableHTTPError) Unwrap added in v0.16.19

func (e *RetryableHTTPError) Unwrap() error

type RiskCategory

type RiskCategory string

RiskCategory represents the specific category of risk for a classified tool call.

const (
	// RiskCategoryReadOnly — commands that only read data (cat, ls, head, grep, etc.)
	RiskCategoryReadOnly RiskCategory = "read-only"
	// RiskCategoryFileWrite — commands that modify files (write_file, edit_file, mkdir, cp, mv)
	RiskCategoryFileWrite RiskCategory = "file-write"
	// RiskCategoryNetwork — commands that access network (curl, wget, fetch)
	RiskCategoryNetwork RiskCategory = "network"
	// RiskCategoryProcessManagement — commands that manage processes (kill, pkill, docker start/stop)
	RiskCategoryProcessManagement RiskCategory = "process-management"
	// RiskCategoryDestructive — commands that destroy data (rm -rf, git reset --hard)
	RiskCategoryDestructive RiskCategory = "destructive"
	// RiskCategoryPrivileged — commands requiring elevated permissions (sudo, chmod, chown)
	RiskCategoryPrivileged RiskCategory = "privileged"
	// RiskCategoryUnknown — default when category cannot be determined
	RiskCategoryUnknown RiskCategory = "unknown"
)

type RollbackResult

type RollbackResult struct {
	Output   string
	Metadata map[string]interface{}
	Success  bool
}

RollbackResult captures the output, metadata, and success state for rollback operations.

func RollbackChanges

func RollbackChanges(revisionID string, filePath string, confirm bool) (RollbackResult, error)

RollbackChanges previews or performs a rollback for a revision or file.

type SearchEngine added in v0.16.18

type SearchEngine interface {
	// Search runs a web search query and returns formatted results.
	Search(ctx context.Context, query string) (string, error)
}

SearchEngine performs web search queries via Google Custom Search API.

type SecurityResult

type SecurityResult struct {
	Risk         SecurityRisk
	Reasoning    string
	ShouldBlock  bool
	ShouldPrompt bool
	IsHardBlock  bool
	RiskType     string       // Deprecated: Use Category instead. Risk category for user-facing messages
	Category     RiskCategory // Granular risk category for the classified operation

	// IntentConfirmation marks a tool call as requiring explicit user
	// confirmation before proceeding, but NOT because it's dangerous.
	// Used for operations that are safe but consequential — like launching
	// a long-running autonomous workflow. The approval prompt uses
	// intent-focused framing instead of security-warning framing.
	IntentConfirmation bool
}

SecurityResult contains the classification result for a tool call

func ClassifyToolCall

func ClassifyToolCall(toolName string, args map[string]interface{}) SecurityResult

ClassifyToolCall classifies a tool call for security purposes based on the tool name and its arguments. It returns a SecurityResult indicating the risk level, reasoning, and whether the operation should be blocked or prompt the user.

Classification is purely string-based (no filesystem access). See the package-level documentation for known limitations of this approach.

Only tools whose arguments carry risk (shell commands, file writes, git ops) need explicit classification. All other registered tools default to SAFE — if a tool is in the registry, it's already vetted. The only real security value is inspecting the *arguments* to those risky tools.

func (SecurityResult) IsDestructive

func (r SecurityResult) IsDestructive() bool

IsDestructive returns true if the operation's risk category is destructive.

type SecurityRisk

type SecurityRisk int

SecurityRisk represents the risk level of a tool call

const (
	SecuritySafe      SecurityRisk = 0
	SecurityCaution   SecurityRisk = 1
	SecurityDangerous SecurityRisk = 2
)

func (SecurityRisk) String

func (r SecurityRisk) String() string

String returns a human-readable risk level

type SessionHeartbeat added in v0.16.18

type SessionHeartbeat struct {
	// SessionID is the unique identifier for this session.
	SessionID string
	// LastHeartbeat is the timestamp of the most recent heartbeat received.
	LastHeartbeat time.Time
	// JobTerminated is called when the heartbeat threshold is exceeded.
	// It receives the sessionID so the caller can clean up resources.
	// If nil, no action is taken on timeout.
	JobTerminated func(sessionID string)
}

SessionHeartbeat tracks the heartbeat state for a single session.

type SkillInfo added in v0.16.18

type SkillInfo struct {
	ID          string `json:"id"`
	Name        string `json:"name"`
	Description string `json:"description"`
	Path        string `json:"path"`
	Content     string `json:"content"`
	Source      string `json:"source"` // "builtin", "user", or "project"
}

SkillInfo is the canonical description of a skill loaded from disk or embedded. It lives here (rather than in pkg/agent) so that pkg/agent_tools can reference it without creating an import cycle.

type SkillLoader added in v0.16.18

type SkillLoader interface {
	// LoadSkill resolves a skill ID and returns its metadata and content.
	LoadSkill(skillID string) (*SkillInfo, error)
}

SkillLoader resolves skill IDs to their on-disk instructions.

type StalenessChecker added in v0.16.18

type StalenessChecker struct {
	// contains filtered or unexported fields
}

StalenessChecker checks whether a file write is stale based on the agent's read tracking and the workspace sync state.

func NewStalenessChecker added in v0.16.18

func NewStalenessChecker(syncState *SyncState, tracker *TurnReadTracker) *StalenessChecker

NewStalenessChecker creates a checker with the given sync state and tracker. Default staleness window is 30 seconds.

func (*StalenessChecker) Check added in v0.16.18

func (sc *StalenessChecker) Check(path string) error

Check returns nil if the write is allowed, or an error if the file may be stale. The error uses the exact format from spec §7.

type StartOptions added in v0.16.4

type StartOptions struct {
	EventBus *events.EventBus // non-nil to enable output-chunk streaming for automate sessions
}

StartOptions configures optional behavior when starting a background process.

type SubtaskInput

type SubtaskInput struct {
	Title      string `json:"title"`
	WorkingDir string `json:"working_dir,omitempty"`
	Persona    string `json:"persona,omitempty"`
	Priority   string `json:"priority,omitempty"`
}

SubtaskInput represents input for creating subtasks

type SymbolEntry added in v0.16.19

type SymbolEntry struct {
	Name string
	Line int
}

SymbolEntry pairs a symbol name with its 1-based line number.

type SymbolWithEdges added in v0.16.19

type SymbolWithEdges struct {
	Symbols []SymbolEntry
	Edges   []codegraph.Edge
}

SymbolWithEdges holds symbols and call edges for a single file.

func ExtractCallsAndSymbols added in v0.16.19

func ExtractCallsAndSymbols(path string, content []byte) (*SymbolWithEdges, error)

ExtractCallsAndSymbols returns both symbols and call edges for a given file.

func (*SymbolWithEdges) ToCodegraphSymbols added in v0.16.19

func (s *SymbolWithEdges) ToCodegraphSymbols(filePath string) ([]codegraph.Symbol, []codegraph.Edge, error)

ToCodegraphSymbols converts the SymbolWithEdges to codegraph Symbol and Edge slices. filePath is the relative path of the source file.

type SyncEnvelope added in v0.16.18

type SyncEnvelope struct {
	// Type is one of the EnvelopeType* constants.
	Type string `json:"type"`

	// Seq is a monotonic sequence number for this direction. The browser and
	// container each maintain their own counters; the counter increments with
	// every envelope sent.
	Seq int64 `json:"seq"`

	// Payload is the structured payload, whose shape depends on Type. For
	// patch_in, it is a PatchInPayload. For patch_out, it is a PatchEvent.
	// For heartbeat, it may be nil or a HeartbeatPayload.
	Payload any `json:"payload"`

	// Error is non-empty when the envelope carries an error response.
	Error string `json:"error,omitempty"`
}

SyncEnvelope wraps a workspace sync message for transport over WebSocket.

@ts-generated — consumed by the frontend to generate a TypeScript interface.

func NewHeartbeatEnvelope added in v0.16.18

func NewHeartbeatEnvelope() *SyncEnvelope

NewHeartbeatEnvelope creates a new heartbeat envelope for keep-alive communication.

func NewPatchInEnvelope added in v0.16.18

func NewPatchInEnvelope(content, path string, browserSeq int64) *SyncEnvelope

NewPatchInEnvelope creates a new patch-in envelope for a browser→container sync operation.

func NewPatchOutEnvelope added in v0.16.18

func NewPatchOutEnvelope(event *PatchEvent) *SyncEnvelope

NewPatchOutEnvelope creates a new patch-out envelope for a container→browser sync operation, wrapping a PatchEvent.

type SyncState added in v0.16.18

type SyncState struct {
	// contains filtered or unexported fields
}

SyncState is the per-file metadata store, protected by a mutex. It lives in-process on the server side to track sequence numbers for each workspace file during a session.

func GetGlobalSyncState added in v0.16.18

func GetGlobalSyncState() *SyncState

GetGlobalSyncState returns the package-level SyncState singleton.

func NewSyncState added in v0.16.18

func NewSyncState() *SyncState

NewSyncState creates a new empty SyncState ready for use.

func (*SyncState) ApplyBrowserOp added in v0.16.18

func (ss *SyncState) ApplyBrowserOp(path string, content string) (*FileMetadata, error)

ApplyBrowserOp applies a browser→container operation (user edit synced to the container). Bumps the browser sequence and acknowledges the container as current.

func (*SyncState) GetAllMetadata added in v0.16.18

func (ss *SyncState) GetAllMetadata() map[string]*FileMetadata

GetAllMetadata returns a snapshot copy of all metadata entries. The returned map and its values are independent copies; mutations will not affect the internal state.

func (*SyncState) GetMetadata added in v0.16.18

func (ss *SyncState) GetMetadata(path string) (*FileMetadata, bool)

GetMetadata looks up metadata for a path. Returns nil and false if not found.

func (*SyncState) HandleContainerPatchWithConflictDetection added in v0.16.18

func (ss *SyncState) HandleContainerPatchWithConflictDetection(
	path string,
	event *PatchEvent,
	browserContent string,
	eventBus EventPublisher,
) (*FileMetadata, *ConflictResult, error)

HandleContainerPatchWithConflictDetection applies a container→browser patch with full conflict detection (SP-046-3).

On clean apply (no conflict): returns (&metadataCopy, nil, nil). On conflict: returns (&metadataCopy, &ConflictResult, nil). On error: returns (nil, nil, err).

func (*SyncState) UpdateContainerPatch added in v0.16.18

func (ss *SyncState) UpdateContainerPatch(path string, event *PatchEvent) (*FileMetadata, error)

UpdateContainerPatch applies a container→browser patch event for the given path. This is called when the agent writes to a file and the server needs to notify the browser of the change.

Returns an error if the browser has unsynced edits (BrowserSeq > LastSyncedBrowser), indicating a conflict that the caller must resolve (e.g., by surfacing a ".theirs" file to the user).

type Task

type Task struct {
	ID           string    `json:"id"`
	Title        string    `json:"title"`
	Description  string    `json:"description,omitempty"`
	Status       string    `json:"status"`   // pending, in_progress, completed, failed, blocked
	Priority     string    `json:"priority"` // high, medium, low
	AssignedTo   string    `json:"assigned_to,omitempty"`
	WorkingDir   string    `json:"working_dir,omitempty"`
	Persona      string    `json:"persona,omitempty"`
	CreatedAt    time.Time `json:"created_at"`
	UpdatedAt    time.Time `json:"updated_at"`
	Result       string    `json:"result,omitempty"`
	ParentTaskID string    `json:"parent_task_id,omitempty"`
}

Task represents a single task in the queue

type TaskQueue

type TaskQueue struct {
	// contains filtered or unexported fields
}

TaskQueue manages a persistent file-based task queue.

Public mutation methods (ReadTasks, PublishTask, AddTask) each acquire an exclusive or shared file lock, read the file fresh from disk, perform their operation, and write back atomically. This prevents the race condition that occurs when callers invoke Load() then PublishTask() as separate steps.

Load() and Save() are retained as convenience wrappers for advanced usage.

func NewTaskQueue

func NewTaskQueue(filePath string) *TaskQueue

NewTaskQueue creates a new TaskQueue instance.

func (*TaskQueue) AddTask

func (tq *TaskQueue) AddTask(ctx context.Context, title, description, priority, workingDir, persona string) (*Task, error)

AddTask atomically adds a new task. Reads fresh from disk under an exclusive file lock, so concurrent adds from separate processes do not overwrite each other.

The exclusive lock is acquired via TryLockContext so the caller's ctx cancels the wait when another process holds the lock.

func (*TaskQueue) Load

func (tq *TaskQueue) Load(ctx context.Context) error

Load reads tasks from disk into the in-memory cache. Useful for initial load followed by multiple operations before Save(), but prefer the self-contained ReadTasks / PublishTask / AddTask methods for safety.

func (*TaskQueue) PublishTask

func (tq *TaskQueue) PublishTask(ctx context.Context, taskID, status, result string, subtasks []SubtaskInput) ([]Task, error)

PublishTask atomically updates a task's status and result and optionally creates subtasks. Reads fresh from disk under an exclusive file lock, so it never overwrites work done by another process between Load and Publish.

The exclusive lock is acquired via TryLockContext so the caller's ctx cancels the wait (tool timeout / user interrupt) instead of blocking indefinitely when another process holds the lock — historically the non-cancellable Lock() call orphaned a goroutine that kept the lock contended long after the tool's outer timeout fired.

func (*TaskQueue) ReadTasks

func (tq *TaskQueue) ReadTasks(ctx context.Context, status string, limit int) ([]Task, error)

ReadTasks reads tasks from disk, filtered by status, sorted by priority then created_at. Each call reads fresh from disk under a shared file lock, so it reflects the latest state without a prior Load().

The shared lock is acquired via TryRLockContext so the caller's ctx cancels the wait (tool timeout / user interrupt) instead of blocking indefinitely when another process holds an exclusive write lock.

func (*TaskQueue) Save

func (tq *TaskQueue) Save(ctx context.Context) error

Save writes the in-memory cache back to disk atomically. Prefer PublishTask / AddTask for mutation since they handle the full atomic cycle.

type TerminalAccess

type TerminalAccess interface {
	// ExecuteCommandInHidden runs a command synchronously on a hidden PTY session
	// and returns the output and exit code.
	ExecuteCommandInHidden(ctx context.Context, sessionID string, command string) (output string, exitCode int, err error)

	// GetOrCreateHiddenSessionForChat returns the session ID of an existing hidden session
	// for the given chat, or creates a new one. Returns the session ID.
	GetOrCreateHiddenSessionForChat(ctx context.Context, chatID string) (sessionID string, err error)

	// ExecuteCommandInBackground writes a command to a new hidden PTY session
	// and returns immediately with the session ID. Does NOT wait for completion.
	// Background sessions get a descriptive name and longer cleanup timeout.
	ExecuteCommandInBackground(ctx context.Context, chatID, command string) (sessionID string, err error)

	// GetBackgroundOutput returns accumulated output for a background session.
	GetBackgroundOutput(sessionID string) (output string, err error)

	// StopBackgroundSession terminates a background session by session ID.
	// Sends Ctrl+C to the PTY and closes the session. Returns an error if the
	// session is not found or is not a background session.
	StopBackgroundSession(sessionID string) error

	// IsSessionActive checks whether a session (by ID) is still active.
	// Returns false if the session doesn't exist or has terminated.
	IsSessionActive(sessionID string) bool
}

TerminalAccess abstracts the operations that shell command execution needs from a terminal manager. This interface is satisfied by the webui's TerminalManager struct (pkg/webui/terminal_types.go) — no explicit import is needed; Go satisfies interfaces structurally.

When a TerminalAccess is available in the context (WebUI mode), shell commands can route through hidden PTY sessions. When absent (CLI mode), commands use the existing os/exec path unchanged.

func TerminalManagerFromContext

func TerminalManagerFromContext(ctx context.Context) TerminalAccess

TerminalManagerFromContext extracts the TerminalAccess from the context. Returns nil if no terminal manager is available (CLI mode).

type TodoItem

type TodoItem struct {
	ID         string `json:"id"`
	Content    string `json:"content"`
	Status     string `json:"status"`               // pending, in_progress, completed, cancelled
	Priority   string `json:"priority,omitempty"`   // high, medium, low
	ActiveForm string `json:"activeForm,omitempty"` // present-continuous phrasing
}

TodoItem represents a single todo item matching Claude Code's TodoWrite/TodoRead schema.

ActiveForm is the present-continuous phrasing surfaced in the activity indicator while Status == "in_progress" (e.g. "Implementing X" vs the imperative Content "Implement X"). Priority drives the colored indicator on the UI; it's accepted from the LLM but is purely presentational.

func TodoRead

func TodoRead() []TodoItem

TodoRead is a convenience wrapper that reads todos from the default scope.

type TodoManager

type TodoManager struct {
	// contains filtered or unexported fields
}

TodoManager manages the todo list for a single conversation scope.

func ManagerForChat added in v0.16.4

func ManagerForChat(chatID string) *TodoManager

ManagerForChat returns the TodoManager for the given chat scope, lazily creating one if needed. A zero scope returns the process-default manager (used by CLI/non-chat tool invocations).

func NewTodoManager

func NewTodoManager() *TodoManager

NewTodoManager creates a new TodoManager instance.

func (*TodoManager) Read

func (tm *TodoManager) Read() []TodoItem

Read returns a copy of the current todo list.

func (*TodoManager) Write

func (tm *TodoManager) Write(todos []TodoItem) string

Write replaces all todo items with the new list and returns a status message.

type ToolDefinition

type ToolDefinition struct {
	Name        string         `json:"name"`
	Description string         `json:"description"`
	Parameters  []ParameterDef `json:"parameters"`
	Required    []string       `json:"required,omitempty"` // Required parameter names
}

ToolDefinition describes a tool's schema for LLM consumption.

type ToolEnv

type ToolEnv struct {
	// EventBus for publishing events (tool_start, tool_end, etc.)
	EventBus *events.EventBus
	// WorkspaceRoot is the working directory root for path resolution
	WorkspaceRoot string
	// OutputWriter for writing tool output (stdout, logs, etc.)
	OutputWriter io.Writer
	// ApprovalManager for security approvals; nil if approvals are not supported
	ApprovalManager ApprovalManager
	// MaxTokensFunc returns the current token budget limit
	MaxTokensFunc func() int
	// ConfigManager provides configuration access for tools that need it (e.g., API keys for web fetching)
	ConfigManager *configuration.Manager
	// EmbeddingMgr is the agent's long-lived embedding manager. When set, tools
	// must reuse it instead of constructing their own — the manager holds the
	// loaded ONNX model and an open HNSW handle, so per-call construction is
	// both slow and unsafe under concurrent writes.
	EmbeddingMgr *embedding.EmbeddingManager
	// AskUser routes ask_user prompts through the active interactive channel
	// (WebUI dialog when a browser is connected, terminal stdin otherwise).
	// Nil means the tool must fall back to the CLI prompt directly.
	AskUser AskUserService
	// TodoManager is the conversation-scoped todo list. When nil, tools
	// should fall back to the package-default scope via ManagerForChat("").
	TodoManager *TodoManager
	// IsInteractiveCLI reports whether the agent is running with a controlling
	// TTY (no WebUI client). Tools use this to decide whether to render
	// rich CLI output (boxes, colors) for the user.
	IsInteractiveCLI bool
	// VisionProcessor, when set, lets vision-dependent tools analyze
	// images and UI screenshots without holding an *Agent reference.
	// Nil means the tool must report "vision unavailable".
	VisionProcessor *VisionProcessor
	// WebBrowser runs headless browser navigation (Playwright/rod wrapper).
	// Nil means the tool must report "browser unavailable".
	WebBrowser WebBrowser
	// SkillLoader resolves skill IDs to their on-disk instructions.
	// Nil means skill loading is not available.
	SkillLoader SkillLoader
	// SearchEngine performs Google Custom Search API queries.
	// Nil means web search is not available.
	SearchEngine SearchEngine
	// SubagentDepth is the nesting depth of subagents (0 = primary agent, 1 = first-level
	// subagent, 2 = second-level, etc.). Used by memory gate and other subagent-specific
	// tool behaviors. Default 0 means not in subagent context.
	SubagentDepth int
	// RawArgsJSON is the raw JSON string of the tool arguments as sent by the
	// LLM. When set, handlers can parse this to recover the original key
	// insertion order of nested maps (e.g., the "data" field in
	// write_structured_file) before Go's map iteration randomizes it.
	RawArgsJSON string
	Notifier    BackgroundNotifier
	// Agent is the *pkg/agent.Agent instance. Only set for tools that explicitly
	// need agent access (e.g., run_subagent, run_parallel_subagents).
	// For all other tools this is nil. Use with care — it creates a tight
	// coupling that should be avoided for new tools.
	Agent interface{} `json:"-"`
}

ToolEnv provides the execution context for a tool without coupling to *Agent.

type ToolHandler

type ToolHandler interface {
	// Name returns the unique tool identifier (e.g., "read_file").
	Name() string
	// Definition returns the JSON schema definition for the LLM to understand the tool.
	Definition() ToolDefinition
	// Validate checks arguments before execution. Returns error if invalid.
	Validate(args map[string]any) error
	// Execute runs the tool with the given context, environment, and arguments.
	Execute(ctx context.Context, env ToolEnv, args map[string]any) (ToolResult, error)

	// Metadata — all optional with sensible defaults. When a metadata method
	// returns its zero value, the ToolRegistry falls back to its own
	// registry-wide defaults for timeout and max result size.
	Aliases() []string      // default: nil (no aliases)
	Timeout() time.Duration // default: 0 (use registry default)
	MaxResultSize() int     // default: 0 (use registry default)
	SafeForParallel() bool  // default: false
	Interactive() bool      // default: false
}

ToolHandler defines the interface for a tool that can be invoked by the agent.

func AllTools

func AllTools() []ToolHandler

AllTools returns all available tool handlers for registration. This is the central registration point for the new interface-based tool system. Currently includes: read_file, list_directory, fetch_url, search_files, repo_map, rollback_changes, view_history, list_skills, embedding_index, write_file, write_structured_file, edit_file, shell_command, manage_memory, manage_settings, task_queue, todo_write, todo_read, ask_user, patch_structured_file, commit, git, activate_skill, web_search, semantic_search, analyze_image_content, analyze_ui_screenshot, list_automate_workflows, list_changes, revert_my_changes, recover_file, create_pull_request, run_automate, mcp_refresh, run_subagent, run_parallel_subagents, request_clarification, and respond_clarification.

browse_url is registered conditionally via registerBrowseURLTool() (build-tagged): it requires a host-side headless browser (rod/Chromium) that is unavailable in WASM builds, so it is excluded from the WASM tool set rather than advertised as a tool that can never succeed.

Memory operations (add/read/list/delete/search) are exposed as the consolidated manage_memory tool registered in pkg/agent/tool_registrations.go. The legacy add_memory / read_memory / list_memories / delete_memory handlers were removed once manage_memory covered the full surface.

Subagent tools (run_subagent / run_parallel_subagents) are registered here using the function-pointer pattern established in Batch A2. Each exports a function pointer (RunSubagentFunc, RunParallelSubagentsFunc) that pkg/agent sets at startup, capturing the *Agent reference in a closure so the handlers don't need direct *Agent access. SP-059 Phase 3b removed earlier stub entries that returned hardcoded errors; these new ToolHandler implementations delegate to the canonical seed-registry dispatch path via the function pointers.

To register all tools with a registry:

registry := tools.NewToolRegistry()
for _, h := range tools.AllTools() {
    registry.Register(h)
}

type ToolRegistry

type ToolRegistry struct {
	// contains filtered or unexported fields
}

ToolRegistry provides thread-safe registration and lookup of ToolHandlers.

func GetNewToolRegistry

func GetNewToolRegistry() *ToolRegistry

GetNewToolRegistry returns the global new-style tool registry singleton.

func NewToolRegistry

func NewToolRegistry() *ToolRegistry

NewToolRegistry creates an empty ToolRegistry.

func (*ToolRegistry) All

func (r *ToolRegistry) All() map[string]ToolHandler

All returns a copy of all registered tools.

func (*ToolRegistry) ForPersona

func (r *ToolRegistry) ForPersona(allowlist []string) map[string]ToolHandler

ForPersona returns the subset of registered tools whose names appear in allowlist. An empty or nil allowlist returns every tool (matching the behavior of unrestricted personas). Tool names present in allowlist but not in the registry are silently skipped — callers shouldn't have to defend against stale allowlists.

The returned map is a copy; callers may mutate it without affecting the registry's state.

func (*ToolRegistry) Lookup

func (r *ToolRegistry) Lookup(name string) (ToolHandler, bool)

Lookup finds a tool by name. Returns (handler, true) if found, (nil, false) otherwise.

func (*ToolRegistry) Names

func (r *ToolRegistry) Names() []string

Names returns a sorted list of all registered tool names.

func (*ToolRegistry) Register

func (r *ToolRegistry) Register(handler ToolHandler) error

Register adds a tool handler. Returns error if name is already registered.

func (*ToolRegistry) Unregister

func (r *ToolRegistry) Unregister(name string) bool

Unregister removes a tool handler by name. Returns true if the tool was found and removed.

type ToolResult

type ToolResult struct {
	// Output is the primary text result of the tool execution.
	Output string `json:"output"`
	// StructuredOut holds optional structured data (maps, slices, etc.)
	StructuredOut any `json:"structured_out,omitempty"`
	// Images contains optional image data for vision-capable tools.
	Images []ImageData `json:"images,omitempty"`
	// TokenUsage tracks tokens consumed during execution.
	TokenUsage int64 `json:"token_usage"`
	// IsError indicates whether this result represents an error state.
	IsError bool `json:"is_error"`
}

ToolResult is the return value from a tool's Execute method.

type TurnReadTracker added in v0.16.18

type TurnReadTracker struct {
	// contains filtered or unexported fields
}

TurnReadTracker tracks per-turn read state for staleness enforcement.

func GetGlobalTurnReadTracker added in v0.16.18

func GetGlobalTurnReadTracker() *TurnReadTracker

GetGlobalTurnReadTracker returns the tracker from the global checker, or nil if no checker has been configured yet.

func NewTurnReadTracker added in v0.16.18

func NewTurnReadTracker() *TurnReadTracker

NewTurnReadTracker creates a fresh tracker ready for a new turn.

func (*TurnReadTracker) GetLastReadSeq added in v0.16.18

func (t *TurnReadTracker) GetLastReadSeq(path string) (int64, bool)

GetLastReadSeq returns the browser_seq captured when the agent last read the given path this turn, along with whether the path was seen.

func (*TurnReadTracker) GetLastReadTime added in v0.16.18

func (t *TurnReadTracker) GetLastReadTime(path string) time.Time

GetLastReadTime returns the time the agent last read the given path this turn, and whether the path was seen.

func (*TurnReadTracker) HasReadThisTurn added in v0.16.18

func (t *TurnReadTracker) HasReadThisTurn(path string) bool

HasReadThisTurn returns true if the agent called read_file on this path during the current turn.

func (*TurnReadTracker) RecordRead added in v0.16.18

func (t *TurnReadTracker) RecordRead(path string, browserSeq int64)

RecordRead records that the agent read the given path at the current turn, capturing the browser_seq from the file's metadata.

type UIElement

type UIElement struct {
	Type        string `json:"type"`             // button, input, text, etc.
	Description string `json:"description"`      // what it looks like
	Position    string `json:"position"`         // approximate location
	Issues      string `json:"issues,omitempty"` // any problems noted
}

UIElement represents a UI element detected in an image

type ViewHistoryResult

type ViewHistoryResult struct {
	Output   string
	Metadata map[string]interface{}
}

ViewHistoryResult captures the output and metadata for history views.

func ViewHistory

func ViewHistory(limit int, fileFilter string, since *time.Time, showContent bool) (ViewHistoryResult, error)

ViewHistory returns a formatted history view based on the provided filters.

type VisionAnalysis

type VisionAnalysis struct {
	ImagePath   string      `json:"image_path"`
	Description string      `json:"description"`
	Elements    []UIElement `json:"elements,omitempty"`
	Issues      []string    `json:"issues,omitempty"`
	Suggestions []string    `json:"suggestions,omitempty"`
}

VisionAnalysis represents the result of vision model analysis

type VisionCacheStats added in v0.16.19

type VisionCacheStats struct {
	Hits       atomic.Uint64
	Misses     atomic.Uint64
	Evictions  atomic.Uint64
	Size       atomic.Int64
	Insertions atomic.Uint64
}

type VisionCacheStatsSnapshot added in v0.16.19

type VisionCacheStatsSnapshot struct {
	Hits       uint64
	Misses     uint64
	Evictions  uint64
	Size       int64
	Insertions uint64
}

type VisionLRUCache added in v0.16.19

type VisionLRUCache struct {
	// contains filtered or unexported fields
}

func NewVisionLRUCache added in v0.16.19

func NewVisionLRUCache(capacity int) *VisionLRUCache

NewVisionLRUCache creates a new LRU cache with the given capacity.

func (*VisionLRUCache) Capacity added in v0.16.19

func (c *VisionLRUCache) Capacity() int

Capacity returns the configured capacity.

func (*VisionLRUCache) CurrentSize added in v0.16.19

func (c *VisionLRUCache) CurrentSize() int64

CurrentSize returns the number of entries currently in the cache.

func (*VisionLRUCache) Get added in v0.16.19

func (c *VisionLRUCache) Get(key string) (string, *VisionUsageInfo, bool)

Get looks up key. On hit the entry is moved to the front (most recently used) and a hit is recorded. On miss a miss is recorded.

func (*VisionLRUCache) Put added in v0.16.19

func (c *VisionLRUCache) Put(key, result string, usage *VisionUsageInfo)

Put inserts or updates key. If the key already exists, the value is updated and the entry is moved to the front. If the cache is at capacity, the least recently used entry (tail.prev) is evicted first.

func (*VisionLRUCache) Reset added in v0.16.19

func (c *VisionLRUCache) Reset()

Reset clears all entries and resets stats, preserving capacity.

func (*VisionLRUCache) Stats added in v0.16.19

Stats returns a snapshot of the current cache statistics.

type VisionMetrics added in v0.16.19

type VisionMetrics struct {
	// ImageTokensTotal is the cumulative count of input prompt image tokens
	// billed across all vision calls (including cached reads).
	ImageTokensTotal atomic.Int64

	// ImageTokensCachedTotal is the cumulative count of cached image
	// tokens — a subset of ImageTokensTotal that hit the cache and so
	// cost only the discounted cached rate.
	ImageTokensCachedTotal atomic.Int64

	// EmbedCallsTotal counts every call to embed a multimodal image into
	// a chat message (processImagesAsMultimodal result).
	EmbedCallsTotal atomic.Int64

	// OCRCallsTotal counts calls to the OCR-via-tool path.
	OCRCallsTotal atomic.Int64

	// ResizeEvents counts every image we resized down to the 1568px cap
	// before embedding (SP-103-B2).
	ResizeEvents atomic.Int64

	// CacheHits and CacheMisses mirror the cache stats on a separate
	// atomic surface so callers can poll metrics cheaply.
	CacheHits   atomic.Int64
	CacheMisses atomic.Int64

	// RetryCount tracks the total number of retry attempts across all
	// vision calls (i.e., the number of times DoVisionRetry re-entered
	// the loop after a failed attempt).
	RetryCount atomic.Int64

	// OCRFallbackTotal counts the number of times OCR fallback was attempted.
	OCRFallbackTotal atomic.Int64

	// OCRFallbackSuccess counts the number of times OCR fallback returned
	// a successful result. The fallback success rate is
	// OCRFallbackSuccess / OCRFallbackTotal.
	OCRFallbackSuccess atomic.Int64

	// LatencyRequestMS accumulates wall-clock time (ms) spent in the
	// provider's SendVisionRequest call (per attempt, not including retries).
	LatencyRequestMS atomic.Int64

	// LatencyRetrySleepMS accumulates wall-clock time (ms) spent sleeping
	// between retry attempts.
	LatencyRetrySleepMS atomic.Int64

	// LatencyFallbackMS accumulates wall-clock time (ms) spent in the
	// OCR fallback path (from entry to result).
	LatencyFallbackMS atomic.Int64

	// LatencyParseMS accumulates wall-clock time (ms) spent parsing the
	// provider response into a VisionAnalysis struct.
	LatencyParseMS atomic.Int64

	FailuresByReason map[string]int64

	// BatchAttempts counts the number of times batched vision analysis was
	// attempted (N>1 images sent together in one provider call).
	BatchAttempts atomic.Int64

	// BatchHits counts the number of times a batched result was served
	// from the cache without a provider call.
	BatchHits atomic.Int64

	// BatchMisses counts the number of times a batched result was NOT in
	// the cache and required a provider call.
	BatchMisses atomic.Int64

	// BatchPartialFailures counts the number of times a batched provider
	// call returned but one or more per-image sections were missing/failed,
	// requiring per-image fallback processing.
	BatchPartialFailures atomic.Int64
	// contains filtered or unexported fields
}

VisionMetrics holds in-memory counters for vision pipeline observability. These are surfaced via GetVisionMetrics() and emitted to the OpenTelemetry metrics sink when one is configured. The hot-path counters are atomic-only (no mutex) — they can be incremented in tight loops without lock contention. The failure-by-reason map uses a sync.RWMutex because (a) writes are rare (only on failures) and (b) we need to iterate the map for snapshots. A sync.Map would work but adds per-entry allocation overhead for no benefit at our write volume.

SP-103-C4: metrics + observability for vision image tokens. VISION-5: structured vision metrics (failure-by-reason, retry count, OCR fallback rate, latency by phase).

type VisionMetricsRecord added in v0.16.19

type VisionMetricsRecord struct {
	Timestamp           string `json:"timestamp"` // RFC3339
	SessionID           string `json:"session_id,omitempty"`
	OpName              string `json:"op_name"`     // e.g. "analyze_image"
	ImageCount          int    `json:"image_count"` // number of images in this call
	Success             bool   `json:"success"`
	FailureReason       string `json:"failure_reason,omitempty"` // classified reason (empty if success)
	RetryCount          int    `json:"retry_count"`              // number of retry attempts (0 = first attempt succeeded)
	UsedOCRFallback     bool   `json:"used_ocr_fallback"`
	OCRFallbackSuccess  bool   `json:"ocr_fallback_success"`
	LatencyRequestMS    int64  `json:"latency_request_ms"`     // total provider call wall time (all attempts)
	LatencyRetrySleepMS int64  `json:"latency_retry_sleep_ms"` // time spent sleeping between retries
	LatencyFallbackMS   int64  `json:"latency_fallback_ms"`    // OCR fallback wall time (0 if not used)
	LatencyParseMS      int64  `json:"latency_parse_ms"`       // response parsing wall time
	ImageTokens         int    `json:"image_tokens"`           // prompt image tokens (including cached)
	ImageTokensCached   int    `json:"image_tokens_cached"`    // cached image tokens
}

VisionMetricsRecord is the per-call entry persisted to ~/.config/sprout/vision_metrics.jsonl. Fire-and-forget — the instrumentation never blocks the agent loop on file IO.

type VisionMetricsSnapshot added in v0.16.19

type VisionMetricsSnapshot struct {
	ImageTokensTotal       int64 `json:"vision_image_tokens_total"`
	ImageTokensCachedTotal int64 `json:"vision_image_tokens_cached_total"`
	EmbedCallsTotal        int64 `json:"vision_embed_calls_total"`
	OCRCallsTotal          int64 `json:"vision_ocr_calls_total"`
	ResizeEvents           int64 `json:"vision_resize_events"`
	CacheHits              int64 `json:"vision_cache_hits"`
	CacheMisses            int64 `json:"vision_cache_misses"`
	// VISION-5: structured metrics
	RetryCount          int64            `json:"vision_retry_count"`
	OCRFallbackTotal    int64            `json:"vision_ocr_fallback_total"`
	OCRFallbackSuccess  int64            `json:"vision_ocr_fallback_success"`
	LatencyRequestMS    int64            `json:"vision_latency_request_ms"`
	LatencyRetrySleepMS int64            `json:"vision_latency_retry_sleep_ms"`
	LatencyFallbackMS   int64            `json:"vision_latency_fallback_ms"`
	LatencyParseMS      int64            `json:"vision_latency_parse_ms"`
	FailuresByReason    map[string]int64 `json:"vision_failures_by_reason"`
	// VISION-4: batch metrics
	BatchAttempts        int64 `json:"vision_batch_attempts"`
	BatchHits            int64 `json:"vision_batch_hits"`
	BatchMisses          int64 `json:"vision_batch_misses"`
	BatchPartialFailures int64 `json:"vision_batch_partial_failures"`
}

VisionMetricsSnapshot is a stable-by-value snapshot of the metrics state.

func GetVisionMetrics added in v0.16.19

func GetVisionMetrics() VisionMetricsSnapshot

GetVisionMetrics returns a stable snapshot of the current vision metrics.

type VisionProcessor

type VisionProcessor struct {
	// contains filtered or unexported fields
}

VisionProcessor handles image analysis using vision-capable models

func NewVisionProcessor

func NewVisionProcessor(client api.ClientInterface, logger *utils.Logger, debug bool) *VisionProcessor

NewVisionProcessor creates a vision processor with the given client

func NewVisionProcessorWithMode

func NewVisionProcessorWithMode(debug bool, _ string) (*VisionProcessor, error)

NewVisionProcessorWithMode creates a vision processor for image/OCR workflows. Client selection is intentionally deterministic and does not vary by mode: provider-vision list first, local Ollama fallback last.

func NewVisionProcessorWithProvider

func NewVisionProcessorWithProvider(debug bool, providerType api.ClientType) (*VisionProcessor, error)

NewVisionProcessorWithProvider creates a vision processor using the specified provider

func (*VisionProcessor) AnalyzeImage

func (vp *VisionProcessor) AnalyzeImage(ctx context.Context, imagePath string, optionalPrompt ...string) (VisionAnalysis, error)

AnalyzeImage processes a single image with the vision model. If optionalPrompt is provided and non-empty, it is used as the prompt; otherwise the default vision prompt for imagePath is created.

func (*VisionProcessor) CreateVisionPrompt

func (vp *VisionProcessor) CreateVisionPrompt(imagePath string) string

CreateVisionPrompt creates an appropriate prompt based on image context

func (*VisionProcessor) DownloadImage

func (vp *VisionProcessor) DownloadImage(ctx context.Context, url string) ([]byte, error)

DownloadImage downloads an image from URL

func (*VisionProcessor) EnhanceTextWithAnalysis

func (vp *VisionProcessor) EnhanceTextWithAnalysis(text, imagePath string, analysis VisionAnalysis) string

EnhanceTextWithAnalysis replaces image references with detailed analysis

func (*VisionProcessor) ExtractPosition

func (vp *VisionProcessor) ExtractPosition(line string) string

ExtractPosition attempts to extract position information from a description

func (*VisionProcessor) ExtractUIElements

func (vp *VisionProcessor) ExtractUIElements(description string) []UIElement

ExtractUIElements attempts to extract structured UI elements from the description

func (*VisionProcessor) GetImageData

func (vp *VisionProcessor) GetImageData(ctx context.Context, imagePath string) (string, string, error)

GetImageData reads image data from file or URL

func (*VisionProcessor) LastUsage added in v0.16.19

func (vp *VisionProcessor) LastUsage() *VisionUsageInfo

LastUsage returns the per-session usage info for this VisionProcessor. Returns nil if no vision call has been made with this processor yet.

func (*VisionProcessor) LooksLikeUI

func (vp *VisionProcessor) LooksLikeUI(description string) bool

LooksLikeUI determines if the description suggests a UI interface

func (*VisionProcessor) ParseUIElementFromLine

func (vp *VisionProcessor) ParseUIElementFromLine(line string) UIElement

ParseUIElementFromLine attempts to extract a UI element from a description line

func (*VisionProcessor) ProcessImagesInText

func (vp *VisionProcessor) ProcessImagesInText(ctx context.Context, text string) (string, []VisionAnalysis, error)

ProcessImagesInText detects images in text and processes them with vision models

func (*VisionProcessor) ProcessPDFForVision

func (vp *VisionProcessor) ProcessPDFForVision(ctx context.Context, pdfPath string) (VisionAnalysis, error)

ProcessPDFForVision processes PDF using the configured vision/OCR model

type VisionProgressFunc added in v0.16.19

type VisionProgressFunc func(completed, total int)

VisionProgressFunc is a callback invoked after each image's OCR completes (success or failure). completed is the number of images processed so far (1-indexed), and total is the total number of images in the batch.

type VisionUsageInfo

type VisionUsageInfo struct {
	PromptTokens     int     `json:"prompt_tokens"`
	CompletionTokens int     `json:"completion_tokens"`
	TotalTokens      int     `json:"total_tokens"`
	EstimatedCost    float64 `json:"estimated_cost"`
}

VisionUsageInfo contains token usage and cost information from vision model calls

func GetLastVisionUsage

func GetLastVisionUsage() *VisionUsageInfo

GetLastVisionUsage returns the usage information from the most recent vision model call across all sessions. Thread-safe.

type WebBrowser added in v0.16.18

type WebBrowser interface {
	// BrowseURL navigates to a URL and returns rendered content.
	// The opts parameter carries tool arguments (action, viewport dimensions,
	// selectors, steps, etc.) as a flexible map. Implementations are expected
	// to convert this map into their internal option struct (e.g.
	// webcontent.BrowseOptions) and perform any action-specific validation.
	BrowseURL(ctx context.Context, url string, opts map[string]any) (string, error)
}

WebBrowser provides headless browser navigation for URL/content analysis.

func NewBrowserAdapter added in v0.16.18

func NewBrowserAdapter() WebBrowser

NewBrowserAdapter creates a browser adapter instance.

Source Files

Directories

Path Synopsis
Package computer_use denylist loader.
Package computer_use denylist loader.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL