browser

package
v0.227.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 6, 2026 License: Apache-2.0 Imports: 14 Imported by: 0

README

browser

What - the browser bounded context: navigating, reading, clicking, typing, taking screenshots and listing tabs in a real browser. Why - browser automation pulls in a heavy engine and a foreign event loop, so it sits behind an anti-corruption layer instead of leaking into the agent. How - domain/ defines the browser port, and infrastructure/ implements it with Playwright or with a thin ExtensionDriver adapter over the opentask extension.

How it plugs in

  • NewTools(...) builds the six Browser* tools from the manifests in tools/, and the container registers them into the tools registry.
  • Swapping Playwright for the extension adapter, or adding another engine, must not change the tool contract the agent calls.
  • ExtensionDriver builds browser_command frames, owns the per-action timeout and maps browser_result payloads to BrowserToolResult. Its transport is the injected ExtensionRequest.
  • This context owns the browser wire contract and uses internal/protocols/agui as its transport. agui knows nothing about browsers, so the frame names, the client kinds, the handshake and the accepted extension origins live in infrastructure/extension_wire.go.
  • ExtensionRelay is the host end. Behind the binding infer daemon hosts, it keeps the one extension connection, replaces it when the extension reconnects, and drives every browser_command through it one at a time, because one browser serves every source. It also reports the extension's state to every other connection as browser_extension_status frames, once when a connection joins and again on attach and detach.
  • ExtensionClient is the client end and the ExtensionRequest of infer chat and a standalone infer headless. It dials the binding as a browser client after its EnsureHost hook made a host listen. The container passes daemon.EnsureRunning, which starts the daemon on demand.
  • A headless --serve worker reroutes the seam through RouteBrowserRequests to a stdio relay: browser_command lines go out on stdout and browser_result lines come back on stdin, so the worker reaches the extension through its host (the daemon, which routes and serializes) and binds no port.

Documentation

Index

Constants

View Source 
const (
	ToolClick      = "BrowserClick"
	ToolNavigate   = "BrowserNavigate"
	ToolRead       = "BrowserRead"
	ToolScreenshot = "BrowserScreenshot"
	ToolTabs       = "BrowserTabs"
	ToolType       = "BrowserType"
)

Names of the browser tools, as their manifests declare them.

Variables

This section is empty.

Functions

func NewTools 

func NewTools(cfg *config.Config, driver browserdomain.BrowserDriver) map[string]agentdomain.Tool

NewTools builds the browser tool set against one shared driver and rate limiter. The caller (the container) owns the driver's lifecycle.

func ToolNames added in v0.216.0 

func ToolNames() []string

ToolNames returns the names of the browser tools, whether or not config enables them, so a custom tool can never take one.

Types

type BrowserClickTool 

type BrowserClickTool struct {
	// contains filtered or unexported fields
}

BrowserClickTool clicks an element in the shared browser session

func NewBrowserClickTool 

func NewBrowserClickTool(cfg *config.Config, rateLimiter rateLimiter, driver browserdomain.BrowserDriver) *BrowserClickTool

NewBrowserClickTool creates a new browser click tool

func (*BrowserClickTool) Definition 

func (t *BrowserClickTool) Definition() sdk.ChatCompletionTool

Definition returns the tool definition for the LLM

func (*BrowserClickTool) Execute 

func (t *BrowserClickTool) Execute(ctx context.Context, args map[string]any) (*agentdomain.ToolExecutionResult, error)

Execute runs the browser click tool with given arguments

func (*BrowserClickTool) FormatForLLM 

func (b *BrowserClickTool) FormatForLLM(result *agentdomain.ToolExecutionResult) string

FormatForLLM formats the result for LLM consumption

func (*BrowserClickTool) FormatPreview 

func (b *BrowserClickTool) FormatPreview(result *agentdomain.ToolExecutionResult) string

FormatPreview returns a short preview of the result for UI display

func (*BrowserClickTool) FormatResult 

func (b *BrowserClickTool) FormatResult(result *agentdomain.ToolExecutionResult, formatType agentdomain.FormatterType) string

FormatResult formats tool execution results for different contexts

func (*BrowserClickTool) IsEnabled 

func (b *BrowserClickTool) IsEnabled() bool

IsEnabled returns whether this tool is enabled

func (*BrowserClickTool) Manifest added in v0.216.0 

func (t *BrowserClickTool) Manifest() agentdomain.ToolManifest

Manifest returns the tool's manifest.

func (*BrowserClickTool) ShouldAlwaysExpand 

func (b *BrowserClickTool) ShouldAlwaysExpand() bool

ShouldAlwaysExpand determines if tool results should always be expanded in UI

func (*BrowserClickTool) ShouldCollapseArg 

func (b *BrowserClickTool) ShouldCollapseArg(string) bool

ShouldCollapseArg determines if an argument should be collapsed in display

func (*BrowserClickTool) Validate 

func (t *BrowserClickTool) Validate(args map[string]any) error

Validate checks if the tool arguments are valid: a selector, or an x/y pair.

type BrowserNavigateTool 

type BrowserNavigateTool struct {
	// contains filtered or unexported fields
}

BrowserNavigateTool opens a URL in the shared browser session

func NewBrowserNavigateTool 

func NewBrowserNavigateTool(cfg *config.Config, rateLimiter rateLimiter, driver browserdomain.BrowserDriver) *BrowserNavigateTool

NewBrowserNavigateTool creates a new browser navigate tool

func (*BrowserNavigateTool) Definition 

func (t *BrowserNavigateTool) Definition() sdk.ChatCompletionTool

Definition returns the tool definition for the LLM

func (*BrowserNavigateTool) Execute 

func (t *BrowserNavigateTool) Execute(ctx context.Context, args map[string]any) (*agentdomain.ToolExecutionResult, error)

Execute runs the browser navigate tool with given arguments

func (*BrowserNavigateTool) FormatForLLM 

func (b *BrowserNavigateTool) FormatForLLM(result *agentdomain.ToolExecutionResult) string

FormatForLLM formats the result for LLM consumption

func (*BrowserNavigateTool) FormatPreview 

func (b *BrowserNavigateTool) FormatPreview(result *agentdomain.ToolExecutionResult) string

FormatPreview returns a short preview of the result for UI display

func (*BrowserNavigateTool) FormatResult 

func (b *BrowserNavigateTool) FormatResult(result *agentdomain.ToolExecutionResult, formatType agentdomain.FormatterType) string

FormatResult formats tool execution results for different contexts

func (*BrowserNavigateTool) IsEnabled 

func (b *BrowserNavigateTool) IsEnabled() bool

IsEnabled returns whether this tool is enabled

func (*BrowserNavigateTool) Manifest added in v0.216.0 

func (t *BrowserNavigateTool) Manifest() agentdomain.ToolManifest

Manifest returns the tool's manifest.

func (*BrowserNavigateTool) ShouldAlwaysExpand 

func (b *BrowserNavigateTool) ShouldAlwaysExpand() bool

ShouldAlwaysExpand determines if tool results should always be expanded in UI

func (*BrowserNavigateTool) ShouldCollapseArg 

func (b *BrowserNavigateTool) ShouldCollapseArg(string) bool

ShouldCollapseArg determines if an argument should be collapsed in display

func (*BrowserNavigateTool) Validate 

func (t *BrowserNavigateTool) Validate(args map[string]any) error

Validate checks if the tool arguments are valid

type BrowserReadTool 

type BrowserReadTool struct {
	// contains filtered or unexported fields
}

BrowserReadTool reads page content from the shared browser session

func NewBrowserReadTool 

func NewBrowserReadTool(cfg *config.Config, rateLimiter rateLimiter, driver browserdomain.BrowserDriver) *BrowserReadTool

NewBrowserReadTool creates a new browser read tool

func (*BrowserReadTool) Definition 

func (t *BrowserReadTool) Definition() sdk.ChatCompletionTool

Definition returns the tool definition for the LLM

func (*BrowserReadTool) Execute 

func (t *BrowserReadTool) Execute(ctx context.Context, args map[string]any) (*agentdomain.ToolExecutionResult, error)

Execute runs the browser read tool with given arguments

func (*BrowserReadTool) FormatForLLM 

func (b *BrowserReadTool) FormatForLLM(result *agentdomain.ToolExecutionResult) string

FormatForLLM formats the result for LLM consumption

func (*BrowserReadTool) FormatPreview 

func (b *BrowserReadTool) FormatPreview(result *agentdomain.ToolExecutionResult) string

FormatPreview returns a short preview of the result for UI display

func (*BrowserReadTool) FormatResult 

func (b *BrowserReadTool) FormatResult(result *agentdomain.ToolExecutionResult, formatType agentdomain.FormatterType) string

FormatResult formats tool execution results for different contexts

func (*BrowserReadTool) IsEnabled 

func (b *BrowserReadTool) IsEnabled() bool

IsEnabled returns whether this tool is enabled

func (*BrowserReadTool) Manifest added in v0.216.0 

func (t *BrowserReadTool) Manifest() agentdomain.ToolManifest

Manifest returns the tool's manifest.

func (*BrowserReadTool) ShouldAlwaysExpand 

func (b *BrowserReadTool) ShouldAlwaysExpand() bool

ShouldAlwaysExpand determines if tool results should always be expanded in UI

func (*BrowserReadTool) ShouldCollapseArg 

func (b *BrowserReadTool) ShouldCollapseArg(string) bool

ShouldCollapseArg determines if an argument should be collapsed in display

func (*BrowserReadTool) Validate 

func (t *BrowserReadTool) Validate(args map[string]any) error

Validate checks if the tool arguments are valid

type BrowserScreenshotTool 

type BrowserScreenshotTool struct {
	// contains filtered or unexported fields
}

BrowserScreenshotTool captures the current browser page as an image and attaches it to the conversation (vision models see it inline; others get a file path for ImageDecode).

func NewBrowserScreenshotTool 

func NewBrowserScreenshotTool(cfg *config.Config, rateLimiter rateLimiter, driver browserdomain.BrowserDriver) *BrowserScreenshotTool

NewBrowserScreenshotTool creates a new browser screenshot tool

func (*BrowserScreenshotTool) Definition 

func (t *BrowserScreenshotTool) Definition() sdk.ChatCompletionTool

Definition returns the tool definition for the LLM

func (*BrowserScreenshotTool) Execute 

func (t *BrowserScreenshotTool) Execute(ctx context.Context, args map[string]any) (*agentdomain.ToolExecutionResult, error)

Execute captures the screenshot and returns it as an attached image

func (*BrowserScreenshotTool) FormatForLLM 

func (b *BrowserScreenshotTool) FormatForLLM(result *agentdomain.ToolExecutionResult) string

FormatForLLM formats the result for LLM consumption

func (*BrowserScreenshotTool) FormatPreview 

func (b *BrowserScreenshotTool) FormatPreview(result *agentdomain.ToolExecutionResult) string

FormatPreview returns a short preview of the result for UI display

func (*BrowserScreenshotTool) FormatResult 

func (b *BrowserScreenshotTool) FormatResult(result *agentdomain.ToolExecutionResult, formatType agentdomain.FormatterType) string

FormatResult formats tool execution results for different contexts

func (*BrowserScreenshotTool) IsEnabled 

func (b *BrowserScreenshotTool) IsEnabled() bool

IsEnabled returns whether this tool is enabled

func (*BrowserScreenshotTool) Manifest added in v0.216.0 

func (t *BrowserScreenshotTool) Manifest() agentdomain.ToolManifest

Manifest returns the tool's manifest.

func (*BrowserScreenshotTool) ShouldAlwaysExpand 

func (b *BrowserScreenshotTool) ShouldAlwaysExpand() bool

ShouldAlwaysExpand determines if tool results should always be expanded in UI

func (*BrowserScreenshotTool) ShouldCollapseArg 

func (b *BrowserScreenshotTool) ShouldCollapseArg(string) bool

ShouldCollapseArg determines if an argument should be collapsed in display

func (*BrowserScreenshotTool) Validate 

func (t *BrowserScreenshotTool) Validate(map[string]any) error

Validate checks if the tool arguments are valid

type BrowserTabsTool 

type BrowserTabsTool struct {
	// contains filtered or unexported fields
}

BrowserTabsTool lists the browser's open tabs and marks the active one, so the agent knows what the user is currently looking at.

func NewBrowserTabsTool 

func NewBrowserTabsTool(cfg *config.Config, rateLimiter rateLimiter, driver browserdomain.BrowserDriver) *BrowserTabsTool

NewBrowserTabsTool creates a new browser tabs tool

func (*BrowserTabsTool) Definition 

func (t *BrowserTabsTool) Definition() sdk.ChatCompletionTool

Definition returns the tool definition for the LLM

func (*BrowserTabsTool) Execute 

func (t *BrowserTabsTool) Execute(ctx context.Context, args map[string]any) (*agentdomain.ToolExecutionResult, error)

Execute lists the open tabs

func (*BrowserTabsTool) FormatForLLM 

func (t *BrowserTabsTool) FormatForLLM(result *agentdomain.ToolExecutionResult) string

FormatForLLM formats the tab list for LLM consumption

func (*BrowserTabsTool) FormatPreview 

func (t *BrowserTabsTool) FormatPreview(result *agentdomain.ToolExecutionResult) string

FormatPreview returns a short preview of the result for UI display

func (*BrowserTabsTool) FormatResult 

func (t *BrowserTabsTool) FormatResult(result *agentdomain.ToolExecutionResult, formatType agentdomain.FormatterType) string

FormatResult formats tool execution results for different contexts

func (*BrowserTabsTool) IsEnabled 

func (b *BrowserTabsTool) IsEnabled() bool

IsEnabled returns whether this tool is enabled

func (*BrowserTabsTool) Manifest added in v0.216.0 

func (t *BrowserTabsTool) Manifest() agentdomain.ToolManifest

Manifest returns the tool's manifest.

func (*BrowserTabsTool) ShouldAlwaysExpand 

func (b *BrowserTabsTool) ShouldAlwaysExpand() bool

ShouldAlwaysExpand determines if tool results should always be expanded in UI

func (*BrowserTabsTool) ShouldCollapseArg 

func (b *BrowserTabsTool) ShouldCollapseArg(string) bool

ShouldCollapseArg determines if an argument should be collapsed in display

func (*BrowserTabsTool) Validate 

func (t *BrowserTabsTool) Validate(map[string]any) error

Validate checks if the tool arguments are valid

type BrowserTypeTool 

type BrowserTypeTool struct {
	// contains filtered or unexported fields
}

BrowserTypeTool types text into an element in the shared browser session

func NewBrowserTypeTool 

func NewBrowserTypeTool(cfg *config.Config, rateLimiter rateLimiter, driver browserdomain.BrowserDriver) *BrowserTypeTool

NewBrowserTypeTool creates a new browser type tool

func (*BrowserTypeTool) Definition 

func (t *BrowserTypeTool) Definition() sdk.ChatCompletionTool

Definition returns the tool definition for the LLM

func (*BrowserTypeTool) Execute 

func (t *BrowserTypeTool) Execute(ctx context.Context, args map[string]any) (*agentdomain.ToolExecutionResult, error)

Execute runs the browser type tool with given arguments

func (*BrowserTypeTool) FormatForLLM 

func (b *BrowserTypeTool) FormatForLLM(result *agentdomain.ToolExecutionResult) string

FormatForLLM formats the result for LLM consumption

func (*BrowserTypeTool) FormatPreview 

func (b *BrowserTypeTool) FormatPreview(result *agentdomain.ToolExecutionResult) string

FormatPreview returns a short preview of the result for UI display

func (*BrowserTypeTool) FormatResult 

func (b *BrowserTypeTool) FormatResult(result *agentdomain.ToolExecutionResult, formatType agentdomain.FormatterType) string

FormatResult formats tool execution results for different contexts

func (*BrowserTypeTool) IsEnabled 

func (b *BrowserTypeTool) IsEnabled() bool

IsEnabled returns whether this tool is enabled

func (*BrowserTypeTool) Manifest added in v0.216.0 

func (t *BrowserTypeTool) Manifest() agentdomain.ToolManifest

Manifest returns the tool's manifest.

func (*BrowserTypeTool) ShouldAlwaysExpand 

func (b *BrowserTypeTool) ShouldAlwaysExpand() bool

ShouldAlwaysExpand determines if tool results should always be expanded in UI

func (*BrowserTypeTool) ShouldCollapseArg 

func (b *BrowserTypeTool) ShouldCollapseArg(string) bool

ShouldCollapseArg determines if an argument should be collapsed in display

func (*BrowserTypeTool) Validate 

func (t *BrowserTypeTool) Validate(args map[string]any) error

Validate checks if the tool arguments are valid

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.