web

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 14, 2026 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package web contains shared contracts and utilities for the reusable web stack.

Index

Constants

View Source 
const (
	// RenderModeAuto lets the fetch core choose the cheapest backend.
	RenderModeAuto = "auto"
	// RenderModeHTTP forces the HTTP fast path.
	RenderModeHTTP = "http"
	// RenderModeBrowser forces JS-rendered browser extraction.
	RenderModeBrowser = "browser"
)
View Source 
const (
	// ActionFetch identifies HTTP or browser-backed page retrieval.
	ActionFetch = "fetch"
	// ActionSearch identifies outbound search requests.
	ActionSearch = "search"
	// ActionMap identifies bounded website discovery requests.
	ActionMap = "map"
	// ActionCrawl identifies bounded multi-page crawl requests.
	ActionCrawl = "crawl"
	// ActionOpen identifies browser page creation.
	ActionOpen = "open"
	// ActionNavigate identifies browser navigation.
	ActionNavigate = "navigate"
	// ActionSnapshot identifies browser read-only extraction.
	ActionSnapshot = "snapshot"
	// ActionClick identifies browser interaction through a click.
	ActionClick = "click"
	// ActionType identifies browser interaction through typing.
	ActionType = "type"
	// ActionPress identifies browser interaction through keyboard presses.
	ActionPress = "press"
	// ActionScroll identifies viewport movement on the current page.
	ActionScroll = "scroll"
	// ActionWait identifies browser stabilization waits.
	ActionWait = "wait"
	// ActionScreenshot identifies visual page capture.
	ActionScreenshot = "screenshot"
	// ActionExtract identifies read-oriented extraction from the current browser page.
	ActionExtract = "extract"
	// ActionListPages identifies page enumeration inside the local browser session.
	ActionListPages = "list_pages"
	// ActionNetworkList identifies read-only browser network inspection.
	ActionNetworkList = "network_list"
	// ActionDownloadList identifies read-only browser download inspection.
	ActionDownloadList = "list_downloads"
	// ActionNetworkPolicy identifies read-only browser network policy inspection.
	ActionNetworkPolicy = "network_policy"
	// ActionSetNetworkPolicy identifies local browser request-policy mutation.
	ActionSetNetworkPolicy = "set_network_policy"
	// ActionSearchContent identifies read-only search over stored browser snapshots.
	ActionSearchContent = "search_content"
	// ActionSelectPage identifies active page switching inside the local browser session.
	ActionSelectPage = "select_page"
	// ActionClosePage identifies page closure inside the local browser session.
	ActionClosePage = "close_page"
)
View Source 
const (
	// CategoryRead is a non-mutating content retrieval action.
	CategoryRead = "read"
	// CategorySearch is a query-oriented discovery action.
	CategorySearch = "search"
	// CategoryNavigate changes browser location or opens a page.
	CategoryNavigate = "navigate"
	// CategoryInteract performs a user-like page interaction.
	CategoryInteract = "interact"
	// CategorySession mutates only ephemeral local browser session state.
	CategorySession = "session"
)

Variables

View Source 
var PreapprovedHosts = map[string]bool{}/* 445 elements not displayed */

PreapprovedHosts enumerates hosts that are safe to access without extra prompts.

View Source 
var PreapprovedPathPrefixes = map[string][]string{
	"github.com": {"/anthropics"},
}

PreapprovedPathPrefixes contains host/path combinations that are allowed more narrowly than the full host.

Functions

func EnrichBrowserPermissionInput 

func EnrichBrowserPermissionInput(input map[string]any, action string, currentURL string) map[string]any

EnrichBrowserPermissionInput normalizes browser tool inputs and optionally attaches the current page URL.

func EnrichFetchPermissionInput 

func EnrichFetchPermissionInput(input map[string]any) map[string]any

EnrichFetchPermissionInput normalizes fetch tool inputs into a stable permission shape.

func EnrichSearchPermissionInput 

func EnrichSearchPermissionInput(input map[string]any) map[string]any

EnrichSearchPermissionInput normalizes search tool inputs into a stable permission shape.

func EvaluatePermission 

func EvaluatePermission(input map[string]any) types.PermissionResult

EvaluatePermission applies lightweight shared web policy before the global permission pipeline runs.

func HostMatchesDomain 

func HostMatchesDomain(host string, domain string) bool

HostMatchesDomain centralizes host/domain matching for filtering and permission logic.

func IsPreapproved 

func IsPreapproved(hostname string) bool

IsPreapproved reports whether the hostname is globally preapproved.

func IsPreapprovedPath 

func IsPreapprovedPath(hostname, pathname string) bool

IsPreapprovedPath reports whether a hostname/path pair is preapproved by either host-wide or path-scoped rules.

func NormalizeHost 

func NormalizeHost(host string) string

NormalizeHost strips superficial hostname differences before policy checks.

func PermissionMatcher 

func PermissionMatcher(input map[string]any) func(string) bool

PermissionMatcher compiles a shared matcher used by content-specific permission rules.

func RejectLocalDialTarget 

func RejectLocalDialTarget(ctx context.Context, address string, resolver HostResolver) error

RejectLocalDialTarget validates a host:port dial target before a transport opens a socket. This closes the gap where a public hostname resolves to a private address after initial URL validation.

func RejectLocalNetworkTarget 

func RejectLocalNetworkTarget(parsed *url.URL) error

RejectLocalNetworkTarget blocks obvious SSRF-style targets before any network work starts. This fast path still checks only the literal host so lightweight normalization can reject obviously local targets before any DNS work starts.

func ResolveAndRejectLocalNetworkTarget 

func ResolveAndRejectLocalNetworkTarget(ctx context.Context, parsed *url.URL, resolver HostResolver) error

ResolveAndRejectLocalNetworkTarget hardens SSRF protection by resolving the hostname and rejecting any target that resolves to loopback, private, or local network addresses.

Types

type ArtifactStore 

type ArtifactStore = storage.ArtifactStore

ArtifactStore exposes persisted blob storage to reusable web services without coupling them to a provider.

type BrowserManager 

type BrowserManager = browsercore.Manager

BrowserManager exposes the shared browser runtime without tying callers to a concrete implementation.

type CrawlPage 

type CrawlPage struct {
	URL           string
	FinalURL      string
	Depth         int
	Mode          string
	Content       string
	PersistedPath string
	PersistedSize int
}

CrawlPage represents one crawled page.

type CrawlRequest 

type CrawlRequest struct {
	URL               string
	SessionID         types.SessionID
	MaxPages          int
	MaxDepth          int
	RenderMode        string
	IncludeSubdomains bool
	IncludePatterns   []string
	ExcludePatterns   []string
}

CrawlRequest describes a bounded multi-page collection request.

type CrawlResponse 

type CrawlResponse struct {
	RootURL string
	Pages   []CrawlPage
}

CrawlResponse is the normalized output shape shared by crawl callers.

type CrawlService 

type CrawlService interface {
	Crawl(ctx context.Context, request CrawlRequest) (CrawlResponse, error)
}

CrawlService represents the reusable crawl core exposed from internal/web.

type DomainCategory 

type DomainCategory struct {
	ID      string   `json:"id"`
	Label   string   `json:"label"`
	Icon    string   `json:"icon"`
	Domains []string `json:"domains"`
}

DomainCategory groups related domains for the frontend domain catalog UI.

func DomainCatalog 

func DomainCatalog() []DomainCategory

DomainCatalog returns the curated pre-approved domain list organized by category.

type FetchRequest 

type FetchRequest struct {
	URL        string
	RenderMode string
	SessionID  types.SessionID
}

FetchRequest is the common input shape for reusable web fetch services.

type FetchResponse 

type FetchResponse struct {
	Content            string
	Bytes              int
	Code               int
	CodeText           string
	ContentType        string
	FinalURL           string
	PersistedPath      string
	PersistedSize      int
	BrowserRecommended bool
	Mode               string
	Redirect           *RedirectInfo
}

FetchResponse is the normalized output shape shared by fetch callers.

type FetchService 

type FetchService interface {
	Fetch(ctx context.Context, request FetchRequest) (FetchResponse, error)
}

FetchService represents the reusable fetch core exposed from internal/web.

type HostResolver 

type HostResolver interface {
	LookupNetIP(ctx context.Context, network string, host string) ([]netip.Addr, error)
}

HostResolver abstracts hostname resolution so network policy can be tested and reused by HTTP fetch, crawl, and future browser transports.

func DefaultResolver 

func DefaultResolver() HostResolver

DefaultResolver is the process-wide DNS resolver used when no explicit resolver is supplied by a web subsystem.

type MapEntry 

type MapEntry struct {
	URL    string
	Source string
}

MapEntry is one discovered canonical URL.

type MapRequest 

type MapRequest struct {
	URL               string
	MaxURLs           int
	IncludeSubdomains bool
}

MapRequest describes a bounded website discovery request.

type MapResponse 

type MapResponse struct {
	RootURL string
	URLs    []MapEntry
}

MapResponse is the normalized output shape shared by map callers.

type MapService 

type MapService interface {
	Map(ctx context.Context, request MapRequest) (MapResponse, error)
}

MapService represents the reusable map core exposed from internal/web.

type RedirectInfo 

type RedirectInfo struct {
	Type        string `json:"type"`
	OriginalURL string `json:"originalUrl"`
	RedirectURL string `json:"redirectUrl"`
	StatusCode  int    `json:"statusCode"`
}

RedirectInfo signals that a host-changing redirect should be explicitly followed by the caller.

type SearchRequest 

type SearchRequest struct {
	Query          string
	AllowedDomains []string
	BlockedDomains []string
}

SearchRequest is the common input shape for reusable web search services.

type SearchResponse 

type SearchResponse struct {
	Query           string
	Results         []SearchResult
	Provider        string
	DurationSeconds float64
}

SearchResponse is the normalized output shape shared by search callers.

type SearchResult 

type SearchResult struct {
	Title       string
	URL         string
	Description string
	Source      string
}

SearchResult is a provider-agnostic search hit used across wrappers and services.

type SearchService 

type SearchService interface {
	Search(ctx context.Context, request SearchRequest) (SearchResponse, error)
}

SearchService represents the reusable search core exposed from internal/web.

type TTLCache 

type TTLCache[V any] struct {
	// contains filtered or unexported fields
}

TTLCache is a generic in-memory cache with simple oldest-expiry eviction.

func NewTTLCache 

func NewTTLCache[V any](config TTLCacheConfig) *TTLCache[V]

NewTTLCache creates a new shared cache with pragmatic defaults if values are omitted.

func (*TTLCache[V]) Clear 

func (c *TTLCache[V]) Clear()

Clear removes every entry from the cache.

func (*TTLCache[V]) Get 

func (c *TTLCache[V]) Get(key string) (V, bool)

Get retrieves a cached value when it is still fresh.

func (*TTLCache[V]) Set 

func (c *TTLCache[V]) Set(key string, value V)

Set stores a cached value and applies basic eviction if the cache is full.

type TTLCacheConfig 

type TTLCacheConfig struct {
	TTL      time.Duration
	MaxItems int
}

TTLCacheConfig configures a lightweight in-memory TTL cache shared by web subsystems.

Source Files

View all Source files

Directories

Path Synopsis
Package browser provides the shared local browser runtime used by Seshat tools and sessions.
 Package browser provides the shared local browser runtime used by Seshat tools and sessions.
Package fetch contains the reusable web fetch core shared by browser- and tool-facing adapters.
 Package fetch contains the reusable web fetch core shared by browser- and tool-facing adapters.
Package search contains the reusable web search core shared by tool wrappers.
 Package search contains the reusable web search core shared by tool wrappers.
providers
Package searxng provides a full-featured SearXNG search client for Go, mirroring the mcp-searxng TypeScript server (src/search.ts).
 Package searxng provides a full-featured SearXNG search client for Go, mirroring the mcp-searxng TypeScript server (src/search.ts).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.