Documentation
¶
Index ¶
- Constants
- Variables
- func Handlers() []tools.ToolHandler
- func NewAuditingBackend(inner ComputerBackend, dir, sessionID string) (*auditingBackend, error)
- func NewRateLimitedBackend(inner ComputerBackend, maxPerMin int) *rateLimitedBackend
- func RecordSafetyEvent(action string, args map[string]any)
- func SetBackend(b ComputerBackend)
- func ToolNames() []string
- func TranslateAnthropicAction(action string, params map[string]any) (any, error)
- type AuditRecord
- type ComputerBackend
- type MockBackend
- func (m *MockBackend) KeyboardPress(key string) error
- func (m *MockBackend) KeyboardType(text string) error
- func (m *MockBackend) MouseClick(x, y int, button MouseButton, double bool) error
- func (m *MockBackend) MouseDrag(from, to Point, button MouseButton) error
- func (m *MockBackend) MoveTo(x, y int) error
- func (m *MockBackend) Screenshot(region *Rect) ([]byte, Size, error)
- func (m *MockBackend) Scroll(dir ScrollDir, amount int, at *Point) error
- type MockBackendRecord
- type MouseButton
- type PlatformSupport
- type Point
- type Rect
- type ScrollDir
- type Size
Constants ¶
const AnthropicComputerToolName = "computer_20241022"
AnthropicComputerToolName is the name of Anthropic's native computer use tool.
Variables ¶
var ErrRateLimited = fmt.Errorf("computer-use action rate limit exceeded")
ErrRateLimited is returned when the action-rate cap is exceeded. It exists so callers / tests can distinguish a safety stop from a backend failure.
Functions ¶
func Handlers ¶
func Handlers() []tools.ToolHandler
Handlers returns the computer-use tool handlers in a stable order. The agent registers these into its execution registry (and derives LLM definitions from each handler's Definition()) only when computer use is enabled in config. Kept here so the unexported handler structs don't need to leak.
func NewAuditingBackend ¶
func NewAuditingBackend(inner ComputerBackend, dir, sessionID string) (*auditingBackend, error)
NewAuditingBackend wraps inner, writing audit records to <dir>/<sessionID>.jsonl. The directory is created if missing. If the log cannot be opened, the error is returned and the caller should fall back to the unwrapped backend rather than failing the whole feature.
func NewRateLimitedBackend ¶
func NewRateLimitedBackend(inner ComputerBackend, maxPerMin int) *rateLimitedBackend
NewRateLimitedBackend wraps inner with a cap of maxPerMin actions per rolling 60s window. A maxPerMin <= 0 disables the cap.
func RecordSafetyEvent ¶
RecordSafetyEvent records a safety-related event (e.g. per-session opt-in consent) to the audit log when the active backend is an auditingBackend. It is a no-op when the backend is not audit-wrapped (e.g. MockBackend in tests or when the audit directory could not be created). This lets callers in the agent layer emit opt-in / denial events without needing to know whether the audit decorator is present.
func SetBackend ¶
func SetBackend(b ComputerBackend)
SetBackend sets the active backend for all tool handlers.
func ToolNames ¶
func ToolNames() []string
ToolNames returns the names of all computer-use tools. Used by the agent's dispatch-layer guard to reject these tools for any persona other than computer_user.
func TranslateAnthropicAction ¶
TranslateAnthropicAction converts an Anthropic computer_20241022 action into calls to our ComputerBackend interface.
Anthropic's tool uses a single "action" parameter with sub-params:
action: "screenshot", "mouse_move", "left_click", "right_click", "middle_click", "double_click", "triple_click", "left_click_drag", "type", "key", "hold_key", "scroll", "wait" coordinate: [x, y] for mouse actions to_coordinate: [x, y] for drag end point text: for type action key: for key / hold_key actions scroll_direction: "up", "down", "left", "right" for scroll amount: for scroll milliseconds: for wait
The return value is optional structured output (e.g. screenshot data for the "screenshot" action). Callers should inspect it for action-specific results.
Types ¶
type AuditRecord ¶
type AuditRecord struct {
Time string `json:"time"` // RFC3339, supplied by the caller's clock
Action string `json:"action"`
Args map[string]any `json:"args,omitempty"`
Err string `json:"error,omitempty"`
}
AuditRecord is one line in a session's computer-use audit log. Screenshots are recorded by size only (not contents) to keep the log small and avoid persisting potentially sensitive screen captures.
type ComputerBackend ¶
type ComputerBackend interface {
Screenshot(region *Rect) (image []byte, dims Size, err error)
MouseClick(x, y int, button MouseButton, double bool) error
MouseDrag(from, to Point, button MouseButton) error
MoveTo(x, y int) error
KeyboardType(text string) error
KeyboardPress(key string) error
Scroll(dir ScrollDir, amount int, at *Point) error
}
ComputerBackend is the platform-specific interface for desktop control. Phase 2 (SP-063-2) will implement this for macOS, Linux, Windows.
func NewPlatformBackend ¶
func NewPlatformBackend() (ComputerBackend, error)
NewPlatformBackend returns the best available real backend for the current host, or a descriptive error when the platform/toolchain can't support computer use. Callers that get an error should keep the default MockBackend and surface the message to the user (e.g. "install cliclick").
type MockBackend ¶
type MockBackend struct {
Records []MockBackendRecord
OverrideScreenshotData []byte
OverrideScreenshotDims Size
OverrideError error
// contains filtered or unexported fields
}
MockBackend implements ComputerBackend for testing.
func (*MockBackend) KeyboardPress ¶
func (m *MockBackend) KeyboardPress(key string) error
func (*MockBackend) KeyboardType ¶
func (m *MockBackend) KeyboardType(text string) error
func (*MockBackend) MouseClick ¶
func (m *MockBackend) MouseClick(x, y int, button MouseButton, double bool) error
func (*MockBackend) MouseDrag ¶
func (m *MockBackend) MouseDrag(from, to Point, button MouseButton) error
func (*MockBackend) MoveTo ¶
func (m *MockBackend) MoveTo(x, y int) error
func (*MockBackend) Screenshot ¶
func (m *MockBackend) Screenshot(region *Rect) ([]byte, Size, error)
type MockBackendRecord ¶
MockBackendRecord records a single call for test assertions.
type MouseButton ¶
type MouseButton string
MouseButton specifies which mouse button.
const ( MouseLeft MouseButton = "left" MouseRight MouseButton = "right" MouseMiddle MouseButton = "middle" )
type PlatformSupport ¶
type PlatformSupport struct {
Supported bool `json:"supported"`
OS string `json:"os"`
Reason string `json:"reason,omitempty"` // populated when Supported is false
}
PlatformSupport describes whether the current host can run computer use and, if not, why. Used by the "Test connection" diagnostic and the persona activation check.
func CheckPlatformSupport ¶
func CheckPlatformSupport() PlatformSupport
CheckPlatformSupport reports whether a real backend can be constructed without actually taking control of the desktop.
type Rect ¶
type Rect struct {
X, Y, Width, Height int
}
Rect represents a rectangular region on screen.
Source Files