agent

package
v0.6.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 22, 2026 License: MIT Imports: 28 Imported by: 0

Documentation

Overview

Package agent provides AI-friendly interaction with FluffyUI applications. It enables automated testing, AI agents, and scripted interactions by exposing a semantic API over the widget tree rather than raw terminal I/O.

Package agent provides AI-friendly interaction with FluffyUI applications. This file contains integration helpers for using the agent server.

The agent server now operates in real-time mode by default, providing:

  • Live UI change notifications via event streaming
  • Bidirectional WebSocket communication
  • Async wait operations for UI conditions
  • Event subscription system

Basic usage:

server, err := agent.EnableFromEnv(app)
if err != nil {
    log.Fatal(err)
}
if server != nil {
    defer server.Stop()
}
app.Run(ctx)

Index

Constants

View Source
const (
	EventWidgetChanged = "widget_changed"
	EventFocusChanged  = "focus_changed"
	EventTextChanged   = "text_changed"
	EventValueChanged  = "value_changed"
	EventStateChanged  = "state_changed"
	EventLayoutChanged = "layout_changed"
	EventSnapshot      = "snapshot"
	EventHeartbeat     = "heartbeat"
)

Event types

Variables

View Source
var (
	ErrWidgetNotFound = errors.New("widget not found")
	ErrWidgetDisabled = errors.New("widget is disabled")
	ErrNotFocusable   = errors.New("widget is not focusable")
	ErrNotInteractive = errors.New("widget is not interactive")
	ErrTimeout        = errors.New("operation timed out")
	ErrNoApp          = errors.New("no app configured")
)

Common errors returned by Agent methods.

View Source
var (
	ErrQueueFull      = errors.New("request queue is full")
	ErrQueueClosed    = errors.New("request queue is closed")
	ErrRequestTimeout = errors.New("request timeout")
)

Request errors

View Source
var (
	ErrSessionNotFound    = errors.New("session not found")
	ErrSessionExpired     = errors.New("session expired")
	ErrSessionRejected    = errors.New("session rejected: server at capacity")
	ErrTooManyRequests    = errors.New("too many pending requests")
	ErrServerShuttingDown = errors.New("server is shutting down")
)

Session errors

Functions

func RealTimeHandler

func RealTimeHandler(opts RealTimeWSOptions) (http.Handler, error)

RealTimeHandler returns an http.Handler for the real-time server

func RunWithAgent

func RunWithAgent(app *runtime.App, ctx context.Context) error

RunWithAgent runs the app with an agent server enabled from environment variables. The agent server is automatically cleaned up when the app exits.

This uses real-time mode by default.

func RunWithRealTimeAgent

func RunWithRealTimeAgent(app *runtime.App, ctx context.Context) error

RunWithRealTimeAgent is an alias for RunWithAgent. Deprecated: Use RunWithAgent instead.

Types

type Agent

type Agent struct {
	// contains filtered or unexported fields
}

Agent provides AI-friendly interaction with a FluffyUI application. It wraps a simulation backend and exposes semantic operations over the widget tree.

func New

func New(cfg Config) *Agent

New creates a new Agent with the given configuration.

func (*Agent) Activate

func (a *Agent) Activate(label string) error

Activate focuses and activates the widget with the given label.

func (*Agent) ActivateWidget

func (a *Agent) ActivateWidget(label string) error

ActivateWidget activates the widget with the given label.

func (*Agent) Backend

func (a *Agent) Backend() *sim.Backend

Backend returns the underlying simulation backend.

func (*Agent) CaptureRegion

func (a *Agent) CaptureRegion(x, y, width, height int) string

CaptureRegion returns raw text for the requested region.

func (*Agent) CaptureText

func (a *Agent) CaptureText() string

CaptureText returns the raw text content of the screen.

func (*Agent) CellAt

func (a *Agent) CellAt(x, y int) (backend.Cell, bool)

CellAt returns the screen cell at the given position.

func (*Agent) ClearFocus

func (a *Agent) ClearFocus() error

ClearFocus clears focus on the current focus scope.

func (*Agent) ContainsText

func (a *Agent) ContainsText(text string) bool

ContainsText checks if the given text appears on screen.

func (*Agent) Dimensions

func (a *Agent) Dimensions() (width, height int)

Dimensions returns the current screen size.

func (*Agent) FindByID

func (a *Agent) FindByID(id string) *WidgetInfo

FindByID finds a widget by its ID.

func (*Agent) FindByLabel

func (a *Agent) FindByLabel(label string) *WidgetInfo

FindByLabel finds the first widget with a matching label (case-insensitive substring).

func (*Agent) FindByRole

func (a *Agent) FindByRole(role accessibility.Role) []WidgetInfo

FindByRole finds all widgets with the given role.

func (*Agent) FindByType

func (a *Agent) FindByType(role accessibility.Role) []WidgetInfo

FindByType is an alias for FindByRole.

func (*Agent) FindText

func (a *Agent) FindText(text string) (x, y int)

FindText returns the position of text on screen, or (-1, -1) if not found.

func (*Agent) Focus

func (a *Agent) Focus(label string) error

Focus moves focus to the widget with the given label.

func (*Agent) FocusByID

func (a *Agent) FocusByID(id string) error

FocusByID focuses a widget by its ID.

func (*Agent) FocusWidget

func (a *Agent) FocusWidget(label string) error

FocusWidget focuses the widget with the given label.

func (*Agent) GetFocused

func (a *Agent) GetFocused() *WidgetInfo

GetFocused returns the currently focused widget.

func (*Agent) GetValue

func (a *Agent) GetValue(label string) (string, error)

GetValue returns the value of an input widget.

func (*Agent) IsChecked

func (a *Agent) IsChecked(label string) bool

IsChecked checks if a checkbox/radio with the given label is checked.

func (*Agent) IsEnabled

func (a *Agent) IsEnabled(label string) bool

IsEnabled checks if a widget with the given label is enabled.

func (*Agent) IsFocused

func (a *Agent) IsFocused(label string) bool

IsFocused checks if a widget with the given label is focused.

func (*Agent) ListWidgets

func (a *Agent) ListWidgets(role accessibility.Role) []WidgetInfo

ListWidgets returns widgets that match the given role.

func (*Agent) Screen

func (a *Agent) Screen() *runtime.Screen

Screen returns the current screen.

func (*Agent) Select

func (a *Agent) Select(label, option string) error

Select focuses the widget and selects the option by label.

func (*Agent) SelectByID

func (a *Agent) SelectByID(id, option string) error

SelectByID focuses the widget by ID and selects the option by label.

func (*Agent) SendKey

func (a *Agent) SendKey(key terminal.Key) error

SendKey injects a key into the app.

func (*Agent) SendKeyMsg

func (a *Agent) SendKeyMsg(msg runtime.KeyMsg) error

SendKeyMsg injects a raw key message into the app.

func (*Agent) SendKeyRune

func (a *Agent) SendKeyRune(key terminal.Key, r rune) error

SendKeyRune injects a key with rune payload.

func (*Agent) SendKeyString

func (a *Agent) SendKeyString(text string) error

SendKeyString injects a string as a sequence of key events.

func (*Agent) SendMouse

func (a *Agent) SendMouse(msg runtime.MouseMsg) error

SendMouse injects a mouse event into the app.

func (*Agent) SendPaste

func (a *Agent) SendPaste(text string) error

SendPaste injects a paste event into the app.

func (*Agent) SendResize

func (a *Agent) SendResize(width, height int) error

SendResize injects a resize event into the app.

func (*Agent) SetScreen

func (a *Agent) SetScreen(screen *runtime.Screen)

SetScreen overrides the screen reference for widget tree access. Most callers can rely on the agent auto-attaching to app.Screen().

func (*Agent) Snapshot

func (a *Agent) Snapshot() Snapshot

Snapshot returns a structured representation of the current UI state.

func (*Agent) SnapshotJSON

func (a *Agent) SnapshotJSON() ([]byte, error)

SnapshotJSON returns the current snapshot serialized to JSON.

func (*Agent) SnapshotWithContext

func (a *Agent) SnapshotWithContext(ctx context.Context, opts SnapshotOptions) (Snapshot, error)

SnapshotWithContext captures a snapshot on the app's event loop when available.

func (*Agent) Tick

func (a *Agent) Tick()

Tick waits for the UI to process pending events.

func (*Agent) Type

func (a *Agent) Type(label, text string) error

Type focuses the widget and types the given text.

func (*Agent) TypeInto

func (a *Agent) TypeInto(label, text string) error

TypeInto focuses the widget and types the given text.

func (*Agent) WaitForEnabled

func (a *Agent) WaitForEnabled(label string, timeout time.Duration) error

WaitForEnabled waits until the widget is enabled.

func (*Agent) WaitForFocus

func (a *Agent) WaitForFocus(label string, timeout time.Duration) error

WaitForFocus waits until the widget with label is focused.

func (*Agent) WaitForIdle

func (a *Agent) WaitForIdle(timeout time.Duration) error

WaitForIdle waits until two consecutive snapshots match.

func (*Agent) WaitForText

func (a *Agent) WaitForText(text string, timeout time.Duration) error

WaitForText waits until text appears on screen or timeout occurs.

func (*Agent) WaitForTextGone

func (a *Agent) WaitForTextGone(text string, timeout time.Duration) error

WaitForTextGone waits until text disappears.

func (*Agent) WaitForValue

func (a *Agent) WaitForValue(label, value string, timeout time.Duration) error

WaitForValue waits until the widget's value matches.

func (*Agent) WaitForWidget

func (a *Agent) WaitForWidget(label string, timeout time.Duration) error

WaitForWidget waits until a widget with the given label is present.

func (*Agent) WaitForWidgetGone

func (a *Agent) WaitForWidgetGone(label string, timeout time.Duration) error

WaitForWidgetGone waits until a widget with the label disappears.

func (*Agent) WithWidgetByID

func (a *Agent) WithWidgetByID(ctx context.Context, id string, fn func(runtime.Widget, accessibility.Accessible) error) error

WithWidgetByID runs fn on the widget with the given ID.

type AgentConfig

type AgentConfig = ServerConfig

AgentConfig is an alias for ServerConfig for backward compatibility. Deprecated: Use ServerConfig instead.

type AppHook

type AppHook struct {
	// contains filtered or unexported fields
}

AppHook integrates the agent server with the app's message/render loop

func NewAppHook

func NewAppHook(app *runtime.App, server *RealTimeServer) *AppHook

NewAppHook creates a new app hook

func (*AppHook) Install

func (h *AppHook) Install()

Install installs the hook into the app

func (*AppHook) OnMessage

func (h *AppHook) OnMessage(fn func(runtime.Message))

OnMessage registers a callback for message events

func (*AppHook) OnRender

func (h *AppHook) OnRender(fn func())

OnRender registers a callback for render events

func (*AppHook) Uninstall

func (h *AppHook) Uninstall()

Uninstall removes the hook

type AppIntegration

type AppIntegration struct {
	App    *runtime.App
	Server *RealTimeServer
	Hook   *AppHook
}

AppIntegration provides high-level integration between App and Agent

func NewAppIntegration

func NewAppIntegration(app *runtime.App, opts EnhancedServerOptions) (*AppIntegration, error)

NewAppIntegration creates and initializes app-agent integration

func (*AppIntegration) Start

func (i *AppIntegration) Start() error

Start starts the integration

func (*AppIntegration) Stop

func (i *AppIntegration) Stop() error

Stop stops the integration

type AsyncResult

type AsyncResult struct {
	// contains filtered or unexported fields
}

AsyncResult represents a future result from an async operation

func (*AsyncResult) IsDone

func (ar *AsyncResult) IsDone() bool

IsDone returns true if the operation is complete

func (*AsyncResult) Wait

func (ar *AsyncResult) Wait(ctx context.Context) (any, error)

Wait blocks until the async operation completes

func (*AsyncResult) WaitTimeout

func (ar *AsyncResult) WaitTimeout(timeout time.Duration) (any, error)

WaitTimeout waits for the async operation with a timeout

type BackgroundJob

type BackgroundJob struct {
	// contains filtered or unexported fields
}

BackgroundJob is a convenience type for simple background jobs

func (*BackgroundJob) Cancel

func (j *BackgroundJob) Cancel()

Cancel stops the job

func (*BackgroundJob) IsRunning

func (j *BackgroundJob) IsRunning() bool

IsRunning returns true if the job is still running

func (*BackgroundJob) Progress

func (j *BackgroundJob) Progress() int

Progress returns the job progress

func (*BackgroundJob) Wait

func (j *BackgroundJob) Wait(ctx context.Context) error

Wait blocks until the job completes

type BackgroundTask

type BackgroundTask struct {
	ID          string
	Name        string
	Description string
	SessionID   string
	// contains filtered or unexported fields
}

BackgroundTask represents a long-running background task

func NewBackgroundTask

func NewBackgroundTask(id, name, description string, sessionID string, fn BackgroundTaskFunc) *BackgroundTask

NewBackgroundTask creates a new background task

func (*BackgroundTask) Cancel

func (t *BackgroundTask) Cancel()

Cancel stops the task

func (*BackgroundTask) CompletedAt

func (t *BackgroundTask) CompletedAt() time.Time

CompletedAt returns when the task completed (zero if not complete)

func (*BackgroundTask) Duration

func (t *BackgroundTask) Duration() time.Duration

Duration returns how long the task has been running

func (*BackgroundTask) Error

func (t *BackgroundTask) Error() error

Error returns the error if the task failed

func (*BackgroundTask) IsDone

func (t *BackgroundTask) IsDone() bool

IsDone returns true if the task has completed, failed, or been cancelled

func (*BackgroundTask) Progress

func (t *BackgroundTask) Progress() int

Progress returns the current progress (0-100)

func (*BackgroundTask) SetProgress

func (t *BackgroundTask) SetProgress(p int)

SetProgress updates the progress (0-100)

func (*BackgroundTask) Start

func (t *BackgroundTask) Start() error

Start begins task execution

func (*BackgroundTask) StartedAt

func (t *BackgroundTask) StartedAt() time.Time

StartedAt returns when the task started

func (*BackgroundTask) Stats

func (t *BackgroundTask) Stats() TaskStats

Stats returns task statistics

func (*BackgroundTask) Status

func (t *BackgroundTask) Status() TaskStatus

Status returns the current task status

func (*BackgroundTask) Wait

func (t *BackgroundTask) Wait(ctx context.Context) error

Wait blocks until the task completes

func (*BackgroundTask) WaitTimeout

func (t *BackgroundTask) WaitTimeout(timeout time.Duration) error

WaitTimeout waits for the task with a timeout

type BackgroundTaskFunc

type BackgroundTaskFunc func(ctx context.Context, task *BackgroundTask) error

BackgroundTaskFunc is the function signature for background tasks

type BackgroundTaskManager

type BackgroundTaskManager struct {
	// contains filtered or unexported fields
}

BackgroundTaskManager manages background tasks

func NewBackgroundTaskManager

func NewBackgroundTaskManager(maxTasks, maxPerSession int) *BackgroundTaskManager

NewBackgroundTaskManager creates a new task manager

func (*BackgroundTaskManager) Cancel

func (m *BackgroundTaskManager) Cancel(id string) bool

Cancel cancels a task by ID

func (*BackgroundTaskManager) CancelSession

func (m *BackgroundTaskManager) CancelSession(sessionID string) int

CancelSession cancels all tasks for a session

func (*BackgroundTaskManager) Count

func (m *BackgroundTaskManager) Count() int

Count returns the number of active tasks

func (*BackgroundTaskManager) Get

Get retrieves a task by ID

func (*BackgroundTaskManager) List

func (m *BackgroundTaskManager) List() []string

List returns all active task IDs

func (*BackgroundTaskManager) ListSession

func (m *BackgroundTaskManager) ListSession(sessionID string) []string

ListSession returns all task IDs for a session

func (*BackgroundTaskManager) SetTaskDoneCallback

func (m *BackgroundTaskManager) SetTaskDoneCallback(fn func(t *BackgroundTask))

SetTaskDoneCallback sets a callback for when tasks complete

func (*BackgroundTaskManager) SetTaskStartCallback

func (m *BackgroundTaskManager) SetTaskStartCallback(fn func(t *BackgroundTask))

SetTaskStartCallback sets a callback for when tasks start

func (*BackgroundTaskManager) Stats

func (m *BackgroundTaskManager) Stats() []TaskStats

Stats returns statistics for all tasks

func (*BackgroundTaskManager) Submit

func (m *BackgroundTaskManager) Submit(id, name, description, sessionID string, fn BackgroundTaskFunc) (*BackgroundTask, error)

Submit creates and starts a new background task

func (*BackgroundTaskManager) SubmitSimple

func (m *BackgroundTaskManager) SubmitSimple(name string, fn func(ctx context.Context) error) (*BackgroundJob, error)

SubmitSimple submits a simple background job

type Capabilities

type Capabilities struct {
	AllowText bool `json:"allow_text"`
	TestMode  bool `json:"test_mode"`
}

Capabilities describes server features exposed to clients.

type Config

type Config struct {
	// App is the FluffyUI application to control.
	// When provided, the agent auto-attaches to the app's screen once available.
	App *runtime.App

	// Sim is the simulation backend. If nil and App is not set, one will be created.
	Sim *sim.Backend

	// PostKey is an optional callback for posting key events.
	// When set, key events are sent through this function instead of the sim backend.
	// This is useful for applications with custom event loops.
	PostKey PostKeyFunc

	// DisableAutoAttach skips automatic App.Screen() attachment.
	// Useful when the caller wants to manage SetScreen explicitly.
	DisableAutoAttach bool

	// Width and Height set the terminal dimensions (default 80x24).
	Width, Height int

	// TickRate is how long to wait between operations for UI to settle.
	// Default is 50ms.
	TickRate time.Duration

	// IncludeText controls whether snapshots include raw screen text.
	// Default is false.
	IncludeText bool
}

Config configures an Agent.

type EnhancedServer

type EnhancedServer struct {
	// contains filtered or unexported fields
}

EnhancedServer exposes an out-of-process JSONL API with session management, request queuing, and background task support.

func NewEnhancedServer

func NewEnhancedServer(opts EnhancedServerOptions) (*EnhancedServer, error)

NewEnhancedServer validates options and constructs an enhanced server.

func (*EnhancedServer) Close

func (s *EnhancedServer) Close() error

Close implements io.Closer

func (*EnhancedServer) Health

func (s *EnhancedServer) Health() HealthStatus

Health returns the current health status

func (*EnhancedServer) Start

func (s *EnhancedServer) Start() error

Start begins accepting connections.

func (*EnhancedServer) Stats

func (s *EnhancedServer) Stats() ServerStats

Stats returns comprehensive server statistics

func (*EnhancedServer) Stop

func (s *EnhancedServer) Stop() error

Stop gracefully shuts down the server.

func (*EnhancedServer) SubmitBackgroundTask

func (s *EnhancedServer) SubmitBackgroundTask(name, description, sessionID string, fn BackgroundTaskFunc) (*BackgroundTask, error)

SubmitBackgroundTask submits a background task

type EnhancedServerOptions

type EnhancedServerOptions struct {
	Addr            string
	App             *runtime.App
	Agent           *Agent
	AllowText       bool
	TestMode        bool
	Token           string
	SnapshotTimeout time.Duration

	// Session management
	SessionPoolLimits PoolLimits
	SessionLimits     SessionLimits

	// Request queue
	QueueOptions QueueOptions

	// Background tasks
	MaxBackgroundTasks int
	MaxTasksPerSession int

	// Connection handling
	MaxConnections        int           // Max concurrent connections (0 = unlimited)
	ConnectionIdleTimeout time.Duration // Timeout for idle connections
	RequestTimeout        time.Duration // Max time to process a request

	// TLS
	TLSConfig   *tls.Config
	TLSCertFile string
	TLSKeyFile  string

	// Health and monitoring
	EnableHealthCheck bool
	HealthInterval    time.Duration
}

EnhancedServerOptions configures the enhanced agent interaction server.

func DefaultEnhancedServerOptions

func DefaultEnhancedServerOptions() EnhancedServerOptions

DefaultEnhancedServerOptions returns reasonable default options

type EventFilters

type EventFilters struct {
	WidgetChanges bool // Widget tree changes
	FocusChanges  bool // Focus changes
	TextChanges   bool // Screen text changes
	ValueChanges  bool // Widget value changes
	StateChanges  bool // Widget state changes (enabled, checked, etc.)
	LayoutChanges bool // Layout/bounds changes
	AllEvents     bool // Receive all events
}

EventFilters controls which events a subscriber receives

func AllEventsFilter

func AllEventsFilter() EventFilters

AllEventsFilter returns filters that capture all events

func DefaultEventFilters

func DefaultEventFilters() EventFilters

DefaultEventFilters returns filters that capture common events

type HealthStatus

type HealthStatus struct {
	Healthy        bool      `json:"healthy"`
	Message        string    `json:"message,omitempty"`
	ActiveConns    int64     `json:"active_connections"`
	ActiveSessions int       `json:"active_sessions"`
	QueueSize      int       `json:"queue_size"`
	ActiveTasks    int       `json:"active_tasks"`
	Timestamp      time.Time `json:"timestamp"`
}

HealthStatus represents the current health of the server

type PoolLimits

type PoolLimits struct {
	MaxSessions        int           // Max total sessions
	MaxBackgroundTasks int           // Max background sessions
	GlobalRateLimit    int           // Global requests per second (0 = unlimited)
	GlobalBurstLimit   int           // Global burst allowance
	CleanupInterval    time.Duration // How often to check for expired sessions
}

PoolLimits defines global pool limits

func DefaultPoolLimits

func DefaultPoolLimits() PoolLimits

DefaultPoolLimits returns reasonable default pool limits

type PoolStats

type PoolStats struct {
	TotalSessions        int   `json:"total_sessions"`
	MaxSessions          int   `json:"max_sessions"`
	NormalSessions       int   `json:"normal_sessions"`
	BackgroundSessions   int   `json:"background_sessions"`
	InteractiveSessions  int   `json:"interactive_sessions"`
	TotalPendingRequests int   `json:"total_pending_requests"`
	RateLimitedSessions  int   `json:"rate_limited_sessions"`
	RateLimitedGlobal    int64 `json:"rate_limited_global"`
}

PoolStats contains pool statistics

type PostKeyFunc

type PostKeyFunc func(msg runtime.KeyMsg) error

PostKeyFunc is a callback for injecting key events into a custom event loop. Used when the application has its own message loop instead of runtime.App.

type QueueOptions

type QueueOptions struct {
	MaxSize        int // Total max requests across all priorities
	MaxPerPriority int // Max per individual priority queue
	Workers        int // Number of concurrent workers
}

QueueOptions configures the request queue

func DefaultQueueOptions

func DefaultQueueOptions() QueueOptions

DefaultQueueOptions returns reasonable defaults

type QueueStats

type QueueStats struct {
	CriticalSize   int `json:"critical_size"`
	HighSize       int `json:"high_size"`
	NormalSize     int `json:"normal_size"`
	LowSize        int `json:"low_size"`
	BackgroundSize int `json:"background_size"`
	Active         int `json:"active"`
	TotalQueued    int `json:"total_queued"`
	TotalDone      int `json:"total_done"`
}

QueueStats contains queue statistics

type RateLimitError

type RateLimitError struct {
	RetryAfter time.Duration
}

RateLimitError is returned when rate limit is exceeded

func (*RateLimitError) Error

func (e *RateLimitError) Error() string

type RealTimeConfig

type RealTimeConfig = ServerConfig

RealTimeConfig is an alias for ServerConfig. Deprecated: Use ServerConfig instead.

type RealTimeNotifier

type RealTimeNotifier struct {
	// contains filtered or unexported fields
}

RealTimeNotifier handles real-time UI change notifications

func NewRealTimeNotifier

func NewRealTimeNotifier(agent *Agent) *RealTimeNotifier

NewRealTimeNotifier creates a new real-time notifier

func (*RealTimeNotifier) BroadcastSnapshot

func (n *RealTimeNotifier) BroadcastSnapshot()

BroadcastSnapshot sends a snapshot to all subscribers

func (*RealTimeNotifier) Notify

func (n *RealTimeNotifier) Notify(event UIEvent)

Notify forces a notification to all subscribers

func (*RealTimeNotifier) Start

func (n *RealTimeNotifier) Start()

Start begins the notifier loop

func (*RealTimeNotifier) Stop

func (n *RealTimeNotifier) Stop()

Stop stops the notifier

func (*RealTimeNotifier) Subscribe

func (n *RealTimeNotifier) Subscribe(sessionID string, filters EventFilters) *RealTimeSubscriber

Subscribe creates a new subscriber

func (*RealTimeNotifier) Unsubscribe

func (n *RealTimeNotifier) Unsubscribe(id string)

Unsubscribe removes a subscriber

func (*RealTimeNotifier) UnsubscribeSession

func (n *RealTimeNotifier) UnsubscribeSession(sessionID string)

UnsubscribeSession removes all subscribers for a session

type RealTimeServer

type RealTimeServer struct {
	*EnhancedServer
	// contains filtered or unexported fields
}

RealTimeServer wraps an enhanced server with real-time capabilities

func EnableEnhancedServerFromEnv

func EnableEnhancedServerFromEnv(app *runtime.App) (*RealTimeServer, error)

EnableEnhancedServerFromEnv is an alias for EnableFromEnv for backward compatibility. Deprecated: Use EnableFromEnv instead.

func EnableFromEnv

func EnableFromEnv(app *runtime.App) (*RealTimeServer, error)

EnableFromEnv enables the agent server from environment variables. This is the primary entry point for agent integration.

The server operates in real-time mode by default, streaming UI events to connected clients.

Environment variables:

  • FLUFFYUI_AGENT: Server address (e.g., "unix:/tmp/agent.sock" or "tcp::8716")
  • FLUFFYUI_AGENT_WS: WebSocket server address (e.g., ":8765")
  • FLUFFYUI_AGENT_TOKEN: Optional authentication token
  • FLUFFYUI_AGENT_ALLOW_TEXT: Set to "1" or "true" to allow text capture
  • FLUFFYUI_AGENT_MAX_SESSIONS: Maximum concurrent sessions (default: 100)
  • FLUFFYUI_AGENT_RATE_LIMIT: Requests per second limit (default: 1000)
  • FLUFFYUI_AGENT_DISABLE_HEALTH: Set to "1" or "true" to disable health checks
  • FLUFFYUI_AGENT_TLS_CERT: TLS certificate file for TCP/WS
  • FLUFFYUI_AGENT_TLS_KEY: TLS key file for TCP/WS
  • FLUFFYUI_AGENT_ALLOWED_ORIGINS: Comma-separated allowed WS origins

Returns nil if FLUFFYUI_AGENT is not set or is set to "0" or "false".

func EnableServerFromEnv

func EnableServerFromEnv(app *runtime.App) (*RealTimeServer, error)

EnableServerFromEnv is an alias for EnableFromEnv for backward compatibility. Deprecated: Use EnableFromEnv instead.

func NewRealTimeServer

func NewRealTimeServer(opts EnhancedServerOptions) (*RealTimeServer, error)

NewRealTimeServer creates a new real-time capable server

func (*RealTimeServer) Start

func (rts *RealTimeServer) Start() error

Start begins the real-time server

func (*RealTimeServer) Stop

func (rts *RealTimeServer) Stop() error

Stop stops the real-time server

func (*RealTimeServer) Subscribe

func (rts *RealTimeServer) Subscribe(sessionID string, filters EventFilters) *RealTimeSubscriber

Subscribe creates a real-time subscription

func (*RealTimeServer) Unsubscribe

func (rts *RealTimeServer) Unsubscribe(id string)

Unsubscribe removes a subscription

func (*RealTimeServer) WaitForCondition

func (rts *RealTimeServer) WaitForCondition(ctx context.Context, condition func(Snapshot) bool, timeout time.Duration) (Snapshot, error)

WaitForCondition waits for a UI condition to be met

func (*RealTimeServer) WaitForFocus

func (rts *RealTimeServer) WaitForFocus(ctx context.Context, widgetID string, timeout time.Duration) error

WaitForFocus waits for a widget to become focused

func (*RealTimeServer) WaitForText

func (rts *RealTimeServer) WaitForText(ctx context.Context, text string, timeout time.Duration) error

WaitForText waits for text to appear on screen

func (*RealTimeServer) WaitForValue

func (rts *RealTimeServer) WaitForValue(ctx context.Context, widgetID string, value string, timeout time.Duration) error

WaitForValue waits for a widget to have a specific value

func (*RealTimeServer) WaitForWidget

func (rts *RealTimeServer) WaitForWidget(ctx context.Context, label string, timeout time.Duration) (WidgetInfo, error)

WaitForWidget waits for a widget to appear

type RealTimeSubscriber

type RealTimeSubscriber struct {
	ID        string
	SessionID string
	Filters   EventFilters

	// Channels
	Events chan UIEvent
	// contains filtered or unexported fields
}

RealTimeSubscriber represents a subscriber to real-time updates

type RealTimeWSOptions

type RealTimeWSOptions struct {
	EnhancedServerOptions
	AllowedOrigins []string
}

RealTimeWSOptions configures the real-time WebSocket server

type RealTimeWebSocketServer

type RealTimeWebSocketServer struct {
	// contains filtered or unexported fields
}

RealTimeWebSocketServer provides bidirectional real-time communication

func NewRealTimeWebSocketServer

func NewRealTimeWebSocketServer(opts RealTimeWSOptions) (*RealTimeWebSocketServer, error)

NewRealTimeWebSocketServer creates a new real-time WebSocket server

func (*RealTimeWebSocketServer) Broadcast

func (s *RealTimeWebSocketServer) Broadcast(msg any)

Broadcast sends a message to all connected clients

func (*RealTimeWebSocketServer) ConnectionCount

func (s *RealTimeWebSocketServer) ConnectionCount() int

ConnectionCount returns the number of active connections

func (*RealTimeWebSocketServer) ServeHTTP

ServeHTTP implements http.Handler

func (*RealTimeWebSocketServer) Start

func (s *RealTimeWebSocketServer) Start() error

Start begins the WebSocket server

func (*RealTimeWebSocketServer) Stop

func (s *RealTimeWebSocketServer) Stop() error

Stop stops the WebSocket server

type Request

type Request struct {
	ID        string
	SessionID string
	Priority  RequestPriority
	CreatedAt time.Time
	Deadline  time.Time // Optional deadline

	// Execution
	Execute   func(ctx context.Context) error
	OnSuccess func(result any)
	OnError   func(err error)
	// contains filtered or unexported fields
}

Request represents a queued operation

func (*Request) IsExpired

func (r *Request) IsExpired(now time.Time) bool

IsExpired returns true if the request has exceeded its deadline

func (*Request) Wait

func (r *Request) Wait(ctx context.Context) (any, error)

Wait blocks until the request is completed or the context is cancelled

type RequestPriority

type RequestPriority int

RequestPriority defines the priority of a request

const (
	RequestPriorityLow RequestPriority = iota
	RequestPriorityNormal
	RequestPriorityHigh
	RequestPriorityCritical
)

type RequestQueue

type RequestQueue struct {
	// contains filtered or unexported fields
}

RequestQueue manages prioritized request queuing

func NewRequestQueue

func NewRequestQueue(opts QueueOptions) *RequestQueue

NewRequestQueue creates a new request queue

func (*RequestQueue) ActiveCount

func (q *RequestQueue) ActiveCount() int

ActiveCount returns the number of active (processing) requests

func (*RequestQueue) AsyncExecute

func (q *RequestQueue) AsyncExecute(fn func(ctx context.Context) (any, error), priority RequestPriority) *AsyncResult

AsyncExecute submits a function for async execution and returns a handle to the result

func (*RequestQueue) AsyncExecuteBackground

func (q *RequestQueue) AsyncExecuteBackground(fn func(ctx context.Context) (any, error)) *AsyncResult

AsyncExecuteBackground submits a function for background execution

func (*RequestQueue) Enqueue

func (q *RequestQueue) Enqueue(req *Request) error

Enqueue adds a request to the queue

func (*RequestQueue) EnqueueBackground

func (q *RequestQueue) EnqueueBackground(req *Request) error

EnqueueBackground adds a request to the background queue

func (*RequestQueue) SetQueueFullCallback

func (q *RequestQueue) SetQueueFullCallback(fn func(req *Request))

SetQueueFullCallback sets a callback for when the queue is full

func (*RequestQueue) SetRequestDoneCallback

func (q *RequestQueue) SetRequestDoneCallback(fn func(req *Request, duration time.Duration, err error))

SetRequestDoneCallback sets a callback for when a request completes

func (*RequestQueue) SetRequestStartCallback

func (q *RequestQueue) SetRequestStartCallback(fn func(req *Request))

SetRequestStartCallback sets a callback for when a request starts

func (*RequestQueue) Size

func (q *RequestQueue) Size() int

Size returns the current queue size

func (*RequestQueue) Stats

func (q *RequestQueue) Stats() QueueStats

Stats returns queue statistics

func (*RequestQueue) Stop

func (q *RequestQueue) Stop()

Stop shuts down the queue and waits for all workers

func (*RequestQueue) TryEnqueue

func (q *RequestQueue) TryEnqueue(req *Request) bool

TryEnqueue attempts to enqueue without blocking, returns immediately

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server exposes an out-of-process JSONL API for agent interaction.

func NewServer

func NewServer(opts ServerOptions) (*Server, error)

NewServer validates options and constructs a server.

func (*Server) Close

func (s *Server) Close() error

Close stops the listener and cleans up any unix socket file.

func (*Server) Serve

func (s *Server) Serve(ctx context.Context) error

Serve starts listening and blocks until the context is done or the listener closes.

type ServerConfig

type ServerConfig struct {
	// contains filtered or unexported fields
}

ServerConfig provides a fluent API for configuring the agent server

func NewAgentConfig

func NewAgentConfig() *ServerConfig

NewAgentConfig is an alias for NewConfig for backward compatibility. Deprecated: Use NewConfig instead.

func NewConfig

func NewConfig() *ServerConfig

NewConfig creates a new agent configuration

func NewRealTimeConfig

func NewRealTimeConfig() *ServerConfig

NewRealTimeConfig is an alias for NewConfig. Deprecated: Use NewConfig instead.

func (*ServerConfig) Build

func (c *ServerConfig) Build(app *runtime.App) (*RealTimeServer, error)

Build creates a RealTimeServer from the configuration

func (*ServerConfig) BuildWebSocket

func (c *ServerConfig) BuildWebSocket(app *runtime.App) (*RealTimeWebSocketServer, error)

BuildWebSocket creates a RealTimeWebSocketServer from the configuration

func (*ServerConfig) WithAddress

func (c *ServerConfig) WithAddress(addr string) *ServerConfig

WithAddress sets the server address

func (*ServerConfig) WithAllowedOrigins

func (c *ServerConfig) WithAllowedOrigins(origins ...string) *ServerConfig

WithAllowedOrigins sets allowed origins for WebSocket connections

func (*ServerConfig) WithBackgroundMode

func (c *ServerConfig) WithBackgroundMode() *ServerConfig

WithBackgroundMode enables background processing mode

func (*ServerConfig) WithEventFilters

func (c *ServerConfig) WithEventFilters(filters EventFilters) *ServerConfig

WithEventFilters sets the event filters for real-time notifications

func (*ServerConfig) WithHealthChecks

func (c *ServerConfig) WithHealthChecks() *ServerConfig

WithHealthChecks enables health monitoring

func (*ServerConfig) WithMaxConnections

func (c *ServerConfig) WithMaxConnections(n int) *ServerConfig

WithMaxConnections sets the maximum number of connections

func (*ServerConfig) WithMaxSessions

func (c *ServerConfig) WithMaxSessions(n int) *ServerConfig

WithMaxSessions sets the maximum number of sessions

func (*ServerConfig) WithRequestTimeout

func (c *ServerConfig) WithRequestTimeout(d time.Duration) *ServerConfig

WithRequestTimeout sets the request timeout

func (*ServerConfig) WithTLSConfig

func (c *ServerConfig) WithTLSConfig(cfg *tls.Config) *ServerConfig

WithTLSConfig sets the TLS configuration for TCP listeners.

func (*ServerConfig) WithTLSFiles

func (c *ServerConfig) WithTLSFiles(certFile, keyFile string) *ServerConfig

WithTLSFiles sets the TLS certificate and key files for TCP listeners.

func (*ServerConfig) WithTestMode

func (c *ServerConfig) WithTestMode() *ServerConfig

WithTestMode enables test mode

func (*ServerConfig) WithTextAccess

func (c *ServerConfig) WithTextAccess() *ServerConfig

WithTextAccess enables text capture

func (*ServerConfig) WithToken

func (c *ServerConfig) WithToken(token string) *ServerConfig

WithToken sets the authentication token

func (*ServerConfig) WithWebSocketAddress

func (c *ServerConfig) WithWebSocketAddress(addr string) *ServerConfig

WithWebSocketAddress sets the WebSocket server address

func (*ServerConfig) WithoutHealthChecks

func (c *ServerConfig) WithoutHealthChecks() *ServerConfig

WithoutHealthChecks disables health monitoring

type ServerOptions

type ServerOptions struct {
	Addr            string
	App             *runtime.App
	Agent           *Agent
	AllowText       bool
	TestMode        bool
	Token           string
	SnapshotTimeout time.Duration
	TLSConfig       *tls.Config
	TLSCertFile     string
	TLSKeyFile      string
}

ServerOptions configures the agent interaction server. TestMode should only be enabled in tests; it bypasses text gating.

type ServerStats

type ServerStats struct {
	Running      bool         `json:"running"`
	ActiveConns  int64        `json:"active_connections"`
	SessionStats PoolStats    `json:"sessions"`
	QueueStats   QueueStats   `json:"queue"`
	ActiveTasks  int          `json:"active_tasks"`
	Health       HealthStatus `json:"health"`
}

ServerStats contains comprehensive server statistics

type Session

type Session struct {
	// contains filtered or unexported fields
}

Session represents an active agent session

func NewSession

func NewSession(id string, mode SessionMode, limits SessionLimits) *Session

NewSession creates a new session with the given parameters

func (*Session) Auth

func (s *Session) Auth()

Auth marks the session as authenticated

func (*Session) CanAcceptRequest

func (s *Session) CanAcceptRequest() error

CanAcceptRequest checks if the session can accept a new request

func (*Session) Close

func (s *Session) Close()

Close closes the session

func (*Session) Context

func (s *Session) Context() context.Context

Context returns the session context

func (*Session) EndRequest

func (s *Session) EndRequest(success bool)

EndRequest marks the end of a request

func (*Session) ID

func (s *Session) ID() string

ID returns the session ID

func (*Session) IsAuthed

func (s *Session) IsAuthed() bool

IsAuthed returns true if the session is authenticated

func (*Session) IsClosed

func (s *Session) IsClosed() bool

IsClosed returns true if the session is closed

func (*Session) IsExpired

func (s *Session) IsExpired(now time.Time) bool

IsExpired returns true if the session has expired

func (*Session) IsRejected

func (s *Session) IsRejected() bool

IsRejected returns true if the session was rejected

func (*Session) LastSeen

func (s *Session) LastSeen() time.Time

LastSeen returns the last seen timestamp

func (*Session) Mode

func (s *Session) Mode() SessionMode

Mode returns the session mode

func (*Session) Priority

func (s *Session) Priority() SessionPriority

Priority returns the session priority

func (*Session) Reject

func (s *Session) Reject()

Reject marks the session as rejected (server at capacity)

func (*Session) SetPriority

func (s *Session) SetPriority(p SessionPriority)

SetPriority updates the session priority

func (*Session) StartRequest

func (s *Session) StartRequest() error

StartRequest marks the beginning of a request

func (*Session) Stats

func (s *Session) Stats() SessionStats

Stats returns session statistics

func (*Session) Touch

func (s *Session) Touch()

Touch updates the last seen timestamp

type SessionLimits

type SessionLimits struct {
	MaxPendingRequests int           // Max requests in queue (0 = unlimited)
	MaxRequestsPerSec  int           // Rate limit (0 = unlimited)
	BurstLimit         int           // Burst allowance (0 = MaxRequestsPerSec * 2)
	IdleTimeout        time.Duration // Session idle timeout
	MaxRequestDuration time.Duration // Max duration for a single request
}

SessionLimits defines resource limits for a session

func BackgroundSessionLimits

func BackgroundSessionLimits() SessionLimits

BackgroundSessionLimits returns limits optimized for background sessions

func DefaultSessionLimits

func DefaultSessionLimits() SessionLimits

DefaultSessionLimits returns reasonable default limits

type SessionMode

type SessionMode int

SessionMode defines how the session operates

const (
	ModeNormal      SessionMode = iota
	ModeBackground              // Lower priority, can be throttled more aggressively
	ModeInteractive             // Higher priority for user-interactive sessions
)

type SessionPool

type SessionPool struct {
	// contains filtered or unexported fields
}

SessionPool manages a pool of sessions with resource limits

func NewSessionPool

func NewSessionPool(limits PoolLimits) *SessionPool

NewSessionPool creates a new session pool

func (*SessionPool) CheckGlobalRate

func (p *SessionPool) CheckGlobalRate() error

CheckGlobalRate returns nil if the global rate limit allows the request

func (*SessionPool) CreateSession

func (p *SessionPool) CreateSession(id string, mode SessionMode, limits SessionLimits) (*Session, error)

CreateSession creates a new session in the pool

func (*SessionPool) GetSession

func (p *SessionPool) GetSession(id string) *Session

GetSession retrieves a session by ID

func (*SessionPool) ListSessions

func (p *SessionPool) ListSessions() []string

ListSessions returns all active session IDs

func (*SessionPool) RemoveSession

func (p *SessionPool) RemoveSession(id string)

RemoveSession removes a session from the pool

func (*SessionPool) Start

func (p *SessionPool) Start()

Start begins background housekeeping

func (*SessionPool) Stats

func (p *SessionPool) Stats() PoolStats

Stats returns pool statistics

func (*SessionPool) Stop

func (p *SessionPool) Stop()

Stop stops background housekeeping

type SessionPriority

type SessionPriority int

SessionPriority defines the priority level for a session

const (
	PriorityLow SessionPriority = iota
	PriorityNormal
	PriorityHigh
	PriorityBackground
)

type SessionStats

type SessionStats struct {
	ID                string          `json:"id"`
	CreatedAt         time.Time       `json:"created_at"`
	LastSeen          time.Time       `json:"last_seen"`
	PendingRequests   int             `json:"pending_requests"`
	CompletedRequests int             `json:"completed_requests"`
	FailedRequests    int             `json:"failed_requests"`
	RateLimited       int             `json:"rate_limited"`
	Authed            bool            `json:"authed"`
	Mode              SessionMode     `json:"mode"`
	Priority          SessionPriority `json:"priority"`
}

SessionStats contains session statistics

type Snapshot

type Snapshot struct {
	Timestamp  time.Time    `json:"timestamp"`
	Width      int          `json:"width"`
	Height     int          `json:"height"`
	LayerCount int          `json:"layer_count,omitempty"`
	Text       string       `json:"text,omitempty"`
	Widgets    []WidgetInfo `json:"widgets,omitempty"`
	FocusedID  string       `json:"focused_id,omitempty"`
	Focused    *WidgetInfo  `json:"focused,omitempty"`
}

Snapshot captures a structured view of the current UI state.

type SnapshotOptions

type SnapshotOptions struct {
	IncludeText bool
}

SnapshotOptions configures snapshot output.

type TaskStats

type TaskStats struct {
	ID          string        `json:"id"`
	Name        string        `json:"name"`
	Description string        `json:"description,omitempty"`
	SessionID   string        `json:"session_id,omitempty"`
	Status      string        `json:"status"`
	Progress    int           `json:"progress"`
	StartedAt   time.Time     `json:"started_at"`
	CompletedAt *time.Time    `json:"completed_at,omitempty"`
	Duration    time.Duration `json:"duration"`
	Error       string        `json:"error,omitempty"`
}

TaskStats returns task statistics

type TaskStatus

type TaskStatus int

TaskStatus represents the status of a background task

const (
	TaskPending TaskStatus = iota
	TaskRunning
	TaskPaused
	TaskCompleted
	TaskFailed
	TaskCancelled
)

func (TaskStatus) String

func (s TaskStatus) String() string

type UIEvent

type UIEvent struct {
	Type      string          `json:"type"`
	Timestamp time.Time       `json:"timestamp"`
	SessionID string          `json:"session_id,omitempty"`
	Data      json.RawMessage `json:"data,omitempty"`
}

UIEvent represents a real-time UI event

type WebSocketOptions

type WebSocketOptions struct {
	ServerOptions
	// AllowedOrigins restricts which origins can connect.
	// Empty slice allows only non-browser clients (no Origin header).
	// Use []string{"*"} explicitly to allow all origins.
	AllowedOrigins []string
}

WebSocketOptions configures the WebSocket server.

type WebSocketServer

type WebSocketServer struct {
	// contains filtered or unexported fields
}

WebSocketServer exposes the agent JSON API over WebSockets.

func NewWebSocketServer

func NewWebSocketServer(opts ServerOptions) (*WebSocketServer, error)

NewWebSocketServer builds a WebSocket server with the provided options.

func NewWebSocketServerWithOptions

func NewWebSocketServerWithOptions(opts WebSocketOptions) (*WebSocketServer, error)

NewWebSocketServerWithOptions builds a WebSocket server with full options.

func (*WebSocketServer) Broadcast

func (s *WebSocketServer) Broadcast(msg any)

Broadcast sends a JSON message to all active connections.

func (*WebSocketServer) NotifyChange

func (s *WebSocketServer) NotifyChange(changeType string, data any)

NotifyChange broadcasts a structured change message.

func (*WebSocketServer) ServeHTTP

func (s *WebSocketServer) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP upgrades the connection and processes JSON messages.

type WidgetChange

type WidgetChange struct {
	ID      string         `json:"id"`
	Changes map[string]any `json:"changes"`
}

WidgetChange represents a change to a specific widget

type WidgetDiff

type WidgetDiff struct {
	Added    []WidgetInfo   `json:"added,omitempty"`
	Removed  []WidgetInfo   `json:"removed,omitempty"`
	Modified []WidgetChange `json:"modified,omitempty"`
}

WidgetDiff represents changes to the widget tree

type WidgetInfo

type WidgetInfo struct {
	ID          string                   `json:"id"`
	Role        accessibility.Role       `json:"type"`
	Label       string                   `json:"label,omitempty"`
	Description string                   `json:"description,omitempty"`
	Value       string                   `json:"value,omitempty"`
	ValueInfo   *accessibility.ValueInfo `json:"value_info,omitempty"`
	State       accessibility.StateSet   `json:"state,omitempty"`
	Bounds      runtime.Rect             `json:"bounds"`
	Children    []WidgetInfo             `json:"children,omitempty"`
	Actions     []string                 `json:"actions,omitempty"`
	Focusable   bool                     `json:"focusable,omitempty"`
	Focused     bool                     `json:"focused,omitempty"`

	// ARIA-like live region, landmark, and relationship fields
	Live        string `json:"live,omitempty"`
	Relevant    string `json:"relevant,omitempty"`
	Atomic      bool   `json:"atomic,omitempty"`
	Landmark    string `json:"landmark,omitempty"`
	LabelledBy  string `json:"labelled_by,omitempty"`
	DescribedBy string `json:"described_by,omitempty"`
	Controls    string `json:"controls,omitempty"`
	Owns        string `json:"owns,omitempty"`
	FlowTo      string `json:"flow_to,omitempty"`

	// WAI-ARIA 1.2/1.3 properties
	Level            int    `json:"level,omitempty"`
	Orientation      string `json:"orientation,omitempty"`
	ActiveDescendant string `json:"active_descendant,omitempty"`
	PosInSet         int    `json:"pos_in_set,omitempty"`
	SetSize          int    `json:"set_size,omitempty"`
	HasPopup         string `json:"has_popup,omitempty"`
	ErrorMessage     string `json:"error_message,omitempty"`
	Current          string `json:"current,omitempty"`
	Autocomplete     string `json:"autocomplete,omitempty"`
	Placeholder      string `json:"placeholder,omitempty"`
	Sort             string `json:"sort,omitempty"`
	KeyShortcuts     string `json:"key_shortcuts,omitempty"`
	Details          string `json:"details,omitempty"`
	RoleDescription  string `json:"role_description,omitempty"`
}

WidgetInfo describes a widget in the UI tree.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL