Documentation
¶
Overview ¶
Package domain holds the computer-use bounded context's contracts: actions the agent can perform, targets they act on, observations they return, and the capabilities the platform supports. It is pure - stdlib imports only.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Action ¶
type Action struct {
Kind ActionKind
Target *Target
Text string // type: the text to type
Label string // press: exact accessibility label
Scope string // accessibility/press: frontmost, dock, menubar, app:<name>, or pid:<n>
Combo string // key: a combo such as "cmd+shift+t"
Button string // click kinds: left (default), right, middle
Direction string // scroll: vertical (default) or horizontal
Amount int // scroll: wheel clicks; negative reverses direction
}
Action is one computer-use request: what to do and, where relevant, on what target and with what input.
type ActionKind ¶
type ActionKind string
ActionKind enumerates what the Computer tool can do.
const ( ActionScreenshot ActionKind = "screenshot" ActionAccessibility ActionKind = "accessibility" ActionCursor ActionKind = "cursor" ActionMove ActionKind = "move" ActionClick ActionKind = "click" ActionDoubleClick ActionKind = "double_click" ActionTripleClick ActionKind = "triple_click" ActionScroll ActionKind = "scroll" ActionType ActionKind = "type" ActionKey ActionKind = "key" ActionPress ActionKind = "press" )
type Image ¶
type Image struct {
Data string // base64-encoded
MimeType string
Path string // on-disk copy, reachable by ImageDecode for non-vision models
}
Image is a captured frame.
type Observation ¶
type Observation struct {
Message string // LLM-facing outcome description
CursorX int
CursorY int
Width int // frame-space dimensions
Height int
Image *Image // set by screenshot actions
Elements []UIElement // set by accessibility observations
}
Observation is what the computer reports back after an action.
type Target ¶
type Target struct {
X int
Y int
// Region, on a screenshot action, captures only this rectangle.
Region *Region
}
Target locates an action on screen, in the frame coordinate space (the same space screenshots and their annotations use).
type UIElement ¶ added in v0.179.0
type UIElement struct {
Role string `json:"role"`
Label string `json:"label"`
State string `json:"state"`
BBox [4]int `json:"bbox"`
}
UIElement is one compact accessibility-tree observation. BBox uses the same frame coordinate space as Computer screenshots and pointer actions.
Source Files