Documentation
¶
Overview ¶
Package desktop captures the screen and sends pointer and keyboard input to the desktop of the current user session.
A Driver builds clicks, drags, key combinations and typing from the few primitives a Backend implements for each operating system. Positions are pixels of the screenshots the driver captures, on the primary display.
Index ¶
- Variables
- func Crop(src *image.RGBA, rect image.Rectangle) *image.RGBA
- func EncodePNG(img image.Image) ([]byte, error)
- func FitSize(width, height, longEdge, maxPixels int) (int, int)
- func Launch(dir, command string, args []string) error
- func RequestPermissions()
- func Resize(src *image.RGBA, width, height int) *image.RGBA
- type Backend
- type Button
- type Diagnostics
- type Driver
- func (d *Driver) AssumeInputSent(at time.Time)
- func (d *Driver) Click(ctx context.Context, at *image.Point, button Button, count int, ...) error
- func (d *Driver) Close() error
- func (d *Driver) CursorPosition() (image.Point, error)
- func (d *Driver) Drag(ctx context.Context, path []image.Point, modifiers []Key) error
- func (d *Driver) HoldKeys(ctx context.Context, keys []Key, duration time.Duration) error
- func (d *Driver) InputSentAt() time.Time
- func (d *Driver) Move(ctx context.Context, at image.Point) error
- func (d *Driver) MoveHolding(ctx context.Context, at image.Point, modifiers []Key) error
- func (d *Driver) PersonInputSince(since time.Time) bool
- func (d *Driver) PressButton(button Button) error
- func (d *Driver) PressKeys(ctx context.Context, keys []Key, repeat int) error
- func (d *Driver) ReleaseButton(button Button) error
- func (d *Driver) Screenshot() (*image.RGBA, error)
- func (d *Driver) Scroll(ctx context.Context, at *image.Point, dx, dy int, modifiers []Key) error
- func (d *Driver) Type(ctx context.Context, text string) error
- func (d *Driver) WaitForIdle(ctx context.Context, idle, poll time.Duration, onWait func()) error
- type Fingerprint
- type Key
- type Problem
Constants ¶
This section is empty.
Variables ¶
var ErrUnsupported = errors.New("desktop automation is supported on macOS and Windows only")
ErrUnsupported reports an operating system without a desktop backend.
Functions ¶
func FitSize ¶
FitSize returns the largest size with the same aspect ratio that fits a long edge and a pixel count; a zero limit is ignored. The size never grows.
func Launch ¶
Launch starts an application in a working directory without waiting for it. The application keeps running after the calling process exits.
func RequestPermissions ¶
func RequestPermissions()
RequestPermissions does nothing where desktop automation is unsupported.
Types ¶
type Backend ¶
type Backend interface {
// Capture returns the primary display in physical pixels.
Capture() (*image.RGBA, error)
// MoveTo moves the pointer to a physical pixel position.
MoveTo(x, y int) error
// Position reports the pointer's physical pixel position.
Position() (x, y int, err error)
// Button presses or releases a button at the pointer. clicks counts
// the clicks of a multi-click so far, starting at 1.
Button(button Button, down bool, clicks int) error
// Wheel scrolls by wheel notches; positive values scroll right and down.
Wheel(dx, dy int) error
// Key presses or releases a key.
Key(key Key, down bool) error
// Type types text at the keyboard focus.
Type(text string) error
// LastInput reports when the desktop last received pointer or keyboard
// input from any source, the driver included, or the zero time when it
// cannot tell.
LastInput() time.Time
// Close releases the backend.
Close() error
}
Backend is the operating system's side of a Driver.
type Button ¶
type Button string
Button is a pointer button.
Pointer buttons.
func ParseButton ¶
ParseButton accepts a button name, defaulting to the left button.
type Diagnostics ¶
type Diagnostics struct {
OS string
Width int
Height int
// Problems are the reasons automation cannot work, such as a missing
// permission; empty when the desktop is usable.
Problems []Problem
}
Diagnostics describes whether the current session can be automated.
func Check ¶
func Check() Diagnostics
Check reports whether the desktop of the current session can be automated.
func (Diagnostics) Err ¶
func (d Diagnostics) Err() error
Err summarizes the problems as one error, or returns nil.
type Driver ¶
type Driver struct {
// contains filtered or unexported fields
}
Driver performs desktop actions on a Backend.
func (*Driver) AssumeInputSent ¶
AssumeInputSent records that input was sent at, such as by an earlier driver on the same desktop, so the desktop reporting it is not taken for a person's.
func (*Driver) Click ¶
func (d *Driver) Click(ctx context.Context, at *image.Point, button Button, count int, modifiers []Key) error
Click clicks a button count times at a position, or at the pointer when at is nil, while holding modifiers.
func (*Driver) CursorPosition ¶
CursorPosition reports the pointer position.
func (*Driver) Drag ¶
Drag holds the left button from the first point of a path to the last, moving in small steps so applications see the drag.
func (*Driver) InputSentAt ¶
InputSentAt reports when the driver last sent input.
func (*Driver) MoveHolding ¶
MoveHolding moves the pointer while holding modifiers, as when hovering with a key pressed.
func (*Driver) PersonInputSince ¶
PersonInputSince reports whether input the driver did not send, such as a person's, reached the desktop after since.
func (*Driver) PressButton ¶
PressButton presses a button at the pointer.
func (*Driver) ReleaseButton ¶
ReleaseButton releases a button at the pointer.
func (*Driver) Screenshot ¶
Screenshot captures the primary display.
func (*Driver) Scroll ¶
Scroll scrolls wheel notches at a position, or at the pointer when at is nil, while holding modifiers.
type Fingerprint ¶
type Fingerprint uint64
Fingerprint is a perceptual hash of an image: similar images have fingerprints a small Distance apart.
func FingerprintAround ¶
FingerprintAround hashes the square of an image centered on a point, which tells apart screens that differ only near where an action lands.
func FingerprintOf ¶
func FingerprintOf(img *image.RGBA) Fingerprint
FingerprintOf hashes an image by whether brightness rises between neighboring cells of a coarse grid.
func (Fingerprint) Distance ¶
func (f Fingerprint) Distance(other Fingerprint) int
Distance counts the bits two fingerprints differ in, from 0 to 64.
type Key ¶
type Key string
Key is a keyboard key: one of the named keys below, a function key "f1" to "f24", or a single printable ASCII character such as "a" or "/".
const ( KeyShift Key = "shift" KeyCtrl Key = "ctrl" KeyAlt Key = "alt" KeyMeta Key = "meta" // Command on macOS, Windows key on Windows. KeyEnter Key = "enter" KeyTab Key = "tab" KeyEscape Key = "escape" KeyBackspace Key = "backspace" KeyDelete Key = "delete" KeySpace Key = "space" KeyInsert Key = "insert" KeyHome Key = "home" KeyEnd Key = "end" KeyPageUp Key = "pageup" KeyPageDown Key = "pagedown" KeyUp Key = "up" KeyDown Key = "down" KeyLeft Key = "left" KeyRight Key = "right" KeyCapsLock Key = "capslock" KeyPrintScreen Key = "printscreen" )
Named keys.
func (Key) FunctionNumber ¶
FunctionNumber returns n for the function key "fn".