Documentation
¶
Overview ¶
Package browser implements the Chrome DevTools Protocol accessibility backend (registered as "cdp") for the Pando Desktop Controller (internal/uiauto): when the UI in question is a Chrome/Edge/Chromium page, it serves the normalized core.Element model straight from the CDP Accessibility/DOM domains of an already-running browser session, instead of the OS accessibility API. One core.Element model, multiple sources.
It never launches a browser itself. RegisterSession/UnregisterSession (session.go) let the existing browser_* agent tools (internal/llm/tools/browser_session.go) publish/retract the chromedp session this backend rides on; Available reports honestly (all-false Capabilities/PLATFORM_NOT_SUPPORTED) when none is registered, matching the NullBackend/Manager contract every other platform backend follows. Manager is a process-wide singleton (see internal/uiauto.Manager.Shared), so a single registered-session slot -- rather than a per-pando-session table -- is sufficient: the backend simply operates against whichever browser session the browser_* tools most recently opened or reused.
The CDP wire access (conn.go) sits behind the axConn interface, so the traversal (traverse.go), role/property mapping (element.go), action dispatch (actions.go) and error mapping in backend.go are all unit-tested against a fake in-memory CDP responder (fake_conn_test.go) without a real browser; backend_integration_test.go additionally drives a real, locally detected browser end to end when one is available, skipping otherwise.
Index ¶
- Constants
- func ActiveSession() (context.Context, bool)
- func NewBackend() (core.Backend, error)
- func RegisterSession(id string, ctx context.Context)
- func UnregisterSession(id string)
- type CdpBackend
- func (b *CdpBackend) Apps(ctx context.Context) ([]core.AppInfo, error)
- func (b *CdpBackend) Available(ctx context.Context) (core.Capabilities, error)
- func (b *CdpBackend) Children(ctx context.Context, el *core.Element) ([]*core.Element, error)
- func (b *CdpBackend) Close() error
- func (b *CdpBackend) Find(ctx context.Context, scope core.Scope, sel *core.Selector, limit int) ([]*core.Element, error)
- func (b *CdpBackend) Name() string
- func (b *CdpBackend) Perform(ctx context.Context, el *core.Element, action core.Action) error
- func (b *CdpBackend) Properties(ctx context.Context, el *core.Element, props []string) (map[string]any, error)
- func (b *CdpBackend) Subscribe(ctx context.Context, scope core.Scope) (<-chan events.Event, func(), error)
- func (b *CdpBackend) Windows(ctx context.Context, appIDFilter string) ([]core.WindowInfo, error)
Constants ¶
const AppID = "browser"
AppID is the fixed identifier of the single virtual "application" the CDP backend reports: the connected browser itself (see Apps/Windows in backend.go). Individual pages are windows within it, keyed by CDP target id. Exported so internal/uiauto.Manager can recognize a browser-scoped operation (app_id == AppID) and route it to this backend without ever having to probe/launch a browser to find out.
Variables ¶
This section is empty.
Functions ¶
func ActiveSession ¶
ActiveSession returns the currently registered chromedp context, if any, and whether it is still usable (registered and not already canceled/closed). It never blocks and never creates a browser.
func NewBackend ¶
NewBackend constructs a CdpBackend. It never fails and never launches a browser: it only wires up the axConn that reads whatever session RegisterSession has published.
func RegisterSession ¶
RegisterSession makes an already-running chromedp session (opened by the browser_* agent tools, internal/llm/tools/browser_session.go, for pando session id) visible to the CDP accessibility backend. Call it whenever a browser session is created or reused. It never triggers a browser launch itself -- it only records a context that already exists.
func UnregisterSession ¶
func UnregisterSession(id string)
UnregisterSession clears the active session when it matches id (a no-op otherwise, e.g. closing a session that was already superseded by a newer one).
Types ¶
type CdpBackend ¶
type CdpBackend struct {
// contains filtered or unexported fields
}
CdpBackend implements core.Backend by serving Element data from the Chrome DevTools Protocol Accessibility/DOM domains of an already-running browser session opened by the browser_* agent tools. It never launches a browser itself: Available/Apps/Windows/Find/Children all report honestly when no session is registered, rather than spawning one, so resolving "auto" (which includes "cdp" in its preference order) can never launch Chrome as a side effect.
func (*CdpBackend) Apps ¶
Apps implements core.Backend: the connected browser is reported as a single application (see appID in element.go), named from its CDP Browser.getVersion product string.
func (*CdpBackend) Available ¶
func (b *CdpBackend) Available(ctx context.Context) (core.Capabilities, error)
Available implements core.Backend. It never launches a browser: absent a registered session it reports an all-false Capabilities plus an APP_NOT_FOUND DesktopError suggesting the agent open a page with browser_navigate first (Manager tolerates Available erroring and simply degrades to empty Capabilities, per its documented contract). A registered-but-unreachable session (e.g. the tab was closed underneath us) degrades the same way.
func (*CdpBackend) Children ¶
Children implements core.Backend: the direct AX children of el, fetched with a single Accessibility.getChildAXNodes call.
func (*CdpBackend) Close ¶
func (b *CdpBackend) Close() error
Close implements core.Backend: it releases this backend's own per-target attachments, never the externally-owned registered browser session.
func (*CdpBackend) Find ¶
func (b *CdpBackend) Find(ctx context.Context, scope core.Scope, sel *core.Selector, limit int) ([]*core.Element, error)
Find implements core.Backend with the selector-driven, depth-capped, limit-capped, ctx-aware traversal in traverse.go -- it never walks a whole page's tree.
func (*CdpBackend) Perform ¶
Perform implements core.Backend, preferring semantic/DOM actions over synthetic mouse input: dom.Focus for focus, a real chromedp click on the resolved node for invoke/toggle/select/expand/collapse, dom.SetValue / Input.insertText for setvalue/type, and ScrollIntoView (+ dispatch) for scroll. An action this backend cannot express returns ACTION_FAILED so core.ActionResolver falls back to the Phase 3 physical layer.
func (*CdpBackend) Properties ¶
func (b *CdpBackend) Properties(ctx context.Context, el *core.Element, props []string) (map[string]any, error)
Properties implements core.Backend. The cheap default (props empty) returns whatever was already decoded into Native.Data by toElement, plus "bounds" is fetched (an extra dom.GetBoxModel round trip) only on explicit request, matching the on-demand pattern the other backends use for their expensive extras (AT-SPI's "text"/"actions", for example).
func (*CdpBackend) Subscribe ¶
func (b *CdpBackend) Subscribe(ctx context.Context, scope core.Scope) (<-chan events.Event, func(), error)
Subscribe implements events.Subscriber for CdpBackend. scope is currently not used to filter server-side (CDP's DOM/Accessibility domain events are not natively scopable to an arbitrary selector); events.WaitFor always re-evaluates the actual locator/condition against the backend on every received event, so an unrelated event only costs one harmless extra Find call, never an incorrect result.
func (*CdpBackend) Windows ¶
func (b *CdpBackend) Windows(ctx context.Context, appIDFilter string) ([]core.WindowInfo, error)
Windows implements core.Backend: every CDP target of type "page" is reported as a window, keyed by its target id.
Source Files