Documentation
¶
Overview ¶
Package computeruse defines the shared model for the session-oriented computer-use server.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ActionResult ¶
type ActionResult struct {
SessionID string `json:"session_id,omitempty"`
StateID string `json:"state_id,omitempty"`
Action string `json:"action,omitempty"`
Target string `json:"target,omitempty"`
Message string `json:"message,omitempty"`
RequiresRefresh bool `json:"requires_refresh,omitempty"`
}
ActionResult reports the outcome of a state-bound interaction.
type AppInfo ¶
type AppInfo struct {
Name string `json:"name,omitempty"`
BundleID string `json:"bundle_id,omitempty"`
PID int `json:"pid,omitempty"`
}
AppInfo identifies a running macOS application.
type AppState ¶
type AppState struct {
ScreenshotMetadata *ScreenshotInfo `json:"screenshot_metadata,omitempty"`
SessionID string `json:"session_id"`
StateID string `json:"state_id"`
App AppInfo `json:"app"`
Window WindowInfo `json:"window"`
Tree []ElementNode `json:"tree"`
ScreenshotPNGBase64 string `json:"screenshot_png_base64,omitempty"`
Instructions string `json:"instructions,omitempty"`
Approval ApprovalState `json:"approval"`
Permissions PermissionState `json:"permissions"`
}
AppState is the canonical snapshot returned by get_app_state.
type ApprovalDecision ¶
type ApprovalDecision string
ApprovalDecision reports how an approval request should be resolved.
const ( ApprovalDecisionRequire ApprovalDecision = "require" ApprovalDecisionApprove ApprovalDecision = "approve" ApprovalDecisionApprovePersistent ApprovalDecision = "approve_persistent" ApprovalDecisionDeny ApprovalDecision = "deny" ApprovalDecisionCancel ApprovalDecision = "cancel" )
type ApprovalOutcome ¶
type ApprovalOutcome string
ApprovalOutcome reports the current approval status or the result of a recent approval request.
const ( ApprovalOutcomeRequired ApprovalOutcome = "required" ApprovalOutcomeApproved ApprovalOutcome = "approved" ApprovalOutcomeDenied ApprovalOutcome = "denied" ApprovalOutcomeCanceled ApprovalOutcome = "canceled" ApprovalOutcomePersistenceFailed ApprovalOutcome = "persistence_failed" )
type ApprovalState ¶
type ApprovalState struct {
Outcome ApprovalOutcome `json:"outcome,omitempty"`
Required bool `json:"required"`
Approved bool `json:"approved"`
Persistent bool `json:"persistent"`
Message string `json:"message,omitempty"`
}
ApprovalState reports the current approval state and, when applicable, the outcome of the decision that produced it.
type ApprovalStore ¶
type ApprovalStore interface {
Resolve(ctx context.Context, bundleID string, decision ApprovalDecision) (ApprovalState, error)
}
ApprovalStore manages app-control approvals.
type CaptureRect ¶ added in v0.3.0
type CaptureRect struct {
X float64 `json:"x"`
Y float64 `json:"y"`
Width float64 `json:"width"`
Height float64 `json:"height"`
}
CaptureRect describes an unrounded rectangle in global macOS logical points.
Example ¶
package main
import (
"fmt"
"github.com/tmc/axmcp/internal/computeruse"
)
func main() {
r := computeruse.CaptureRect{X: -100.5, Y: 20.25, Width: 410, Height: 272}
fmt.Println(r.X, r.Y, r.Width, r.Height)
}
Output: -100.5 20.25 410 272
type DisplayInfo ¶ added in v0.3.0
type DisplayInfo struct {
ID uint32 `json:"id"`
Bounds CaptureRect `json:"bounds"`
PixelWidth int `json:"pixel_width"`
PixelHeight int `json:"pixel_height"`
}
DisplayInfo records a display's logical bounds and physical pixel dimensions.
Example ¶
package main
import (
"fmt"
"github.com/tmc/axmcp/internal/computeruse"
)
func main() {
d := computeruse.DisplayInfo{ID: 1, Bounds: computeruse.CaptureRect{Width: 1280, Height: 800}, PixelWidth: 2560, PixelHeight: 1600}
fmt.Println(d.Bounds.Width, d.PixelWidth)
}
Output: 1280 2560
type ElementNode ¶
type ElementNode struct {
Index int `json:"index"`
ParentIndex int `json:"parent_index"`
Role string `json:"role,omitempty"`
Title string `json:"title,omitempty"`
Value string `json:"value,omitempty"`
Description string `json:"description,omitempty"`
Identifier string `json:"identifier,omitempty"`
X int `json:"x,omitempty"`
Y int `json:"y,omitempty"`
Width int `json:"width,omitempty"`
Height int `json:"height,omitempty"`
Enabled bool `json:"enabled"`
Settable bool `json:"settable"`
SecondaryActions []string `json:"secondary_actions,omitempty"`
}
ElementNode is an indexed AX node in a returned app state.
type InstructionProvider ¶
InstructionProvider returns app-specific guidance for a snapshot.
type PermissionState ¶
type PermissionState struct {
AccessibilityGranted bool `json:"accessibility_granted"`
AccessibilityStatus string `json:"accessibility_status,omitempty"`
ScreenRecordingGranted bool `json:"screen_recording_granted"`
ScreenRecordingStatus string `json:"screen_recording_status,omitempty"`
Pending bool `json:"pending"`
Message string `json:"message,omitempty"`
}
PermissionState reports the current system-permission status needed for computer-use actions.
type ScreenshotInfo ¶ added in v0.3.0
type ScreenshotInfo struct {
ImageID string `json:"image_id"`
Width int `json:"width"`
Height int `json:"height"`
SourceKind string `json:"source_kind"`
TargetWindow uint32 `json:"target_window"`
GlobalRect CaptureRect `json:"global_rect"`
ScaleX float64 `json:"scale_x"`
ScaleY float64 `json:"scale_y"`
Displays []DisplayInfo `json:"displays"`
}
ScreenshotInfo binds raw PNG pixels to the window geometry used at capture. ScaleX and ScaleY are pixels per logical point, not claims about display DPI.
Example ¶
package main
import (
"fmt"
"github.com/tmc/axmcp/internal/computeruse"
)
func main() {
image := computeruse.ScreenshotInfo{
Width: 820, Height: 544,
GlobalRect: computeruse.CaptureRect{X: 150, Y: 100, Width: 410, Height: 272},
ScaleX: 2, ScaleY: 2,
}
// A raw image point maps to global macOS logical points using its capture.
x, y := 188.0, 320.0
fmt.Println(image.GlobalRect.X+x/image.ScaleX, image.GlobalRect.Y+y/image.ScaleY)
}
Output: 244 260
type WindowInfo ¶
type WindowInfo struct {
WindowID uint32 `json:"window_id,omitempty"`
Title string `json:"title,omitempty"`
X int `json:"x,omitempty"`
Y int `json:"y,omitempty"`
Width int `json:"width,omitempty"`
Height int `json:"height,omitempty"`
ScreenshotWidth int `json:"screenshot_width,omitempty"`
ScreenshotHeight int `json:"screenshot_height,omitempty"`
}
WindowInfo describes the active target window for a state snapshot.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package approval coordinates app-control grants and revocation.
|
Package approval coordinates app-control grants and revocation. |
|
Package appstate builds live app snapshots for the computer-use server.
|
Package appstate builds live app snapshots for the computer-use server. |
|
Package coords converts between computer-use coordinate spaces.
|
Package coords converts between computer-use coordinate spaces. |
|
Package input provides reusable input helpers for the computer-use server.
|
Package input provides reusable input helpers for the computer-use server. |
|
Package instruction provides app-specific hints for computer-use sessions.
|
Package instruction provides app-specific hints for computer-use sessions. |
|
Package intervention detects recent physical user input.
|
Package intervention detects recent physical user input. |
|
Package magnify resolves semantic zoom and pinch requests into concrete backend actions.
|
Package magnify resolves semantic zoom and pinch requests into concrete backend actions. |
|
Package overlay draws labeled OCR matches on captured images.
|
Package overlay draws labeled OCR matches on captured images. |
|
Package policy enforces computer-use safety policies.
|
Package policy enforces computer-use safety policies. |
|
Package session keeps native observations alive while actions hold leases.
|
Package session keeps native observations alive while actions hold leases. |
Click to show internal directories.
Click to hide internal directories.