browsertransport

package
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 31, 2026 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

Package browsertransport provides reusable browser and HTTP transport helpers for proxy-aware scraping runtimes.

Index

Constants

View Source 
const DefaultStealthScript = `` /* 4877-byte string literal not displayed */

DefaultStealthScript removes common browser automation markers before page JavaScript runs. It covers navigator properties, WebGL renderer masking, chrome API stubs, screen/window dimensions, and CDP artifact cleanup.

Variables

This section is empty.

Functions

func DefaultUserAgent added in v0.11.0 

func DefaultUserAgent(execPath string) string

DefaultUserAgent returns a realistic Chrome User-Agent string whose major version matches the installed Chrome binary. Falls back to a recent hardcoded version when detection fails.

func DetectChromeVersion added in v0.11.0 

func DetectChromeVersion(execPath string) string

DetectChromeVersion returns the major version of the Chrome binary at execPath, or tries common platform paths when execPath is empty. Returns an empty string if detection fails.

func NewHTTPClient 

func NewHTTPClient(httpProfile HTTPProfile, timeout time.Duration) (*http.Client, error)

NewHTTPClient builds an HTTP client bound to one transport profile.

Types

type BrowserMode 

type BrowserMode string

BrowserMode describes how a browser should reach the upstream network. 

const (
	// BrowserModeDirect uses Chrome's native proxy handling or a direct network path.
	BrowserModeDirect BrowserMode = "direct"
	// BrowserModeHTTPFetchAuth strips inline auth from an HTTP proxy URL and
	// supplies credentials through the Fetch domain.
	BrowserModeHTTPFetchAuth BrowserMode = "http_fetch_auth"
	// BrowserModeSOCKSForwarder bridges an authenticated SOCKS proxy through a
	// local unauthenticated forwarder Chrome can consume.
	BrowserModeSOCKSForwarder BrowserMode = "socks_forwarder"
)

type BrowserProfile 

type BrowserProfile struct {
	ID               string
	Provider         string
	URL              string
	Mode             BrowserMode
	IgnoreCertErrors bool
}

BrowserProfile describes how to launch a browser transport.

func InferBrowserProfile 

func InferBrowserProfile(rawProxyURL string, ignoreCertErrors bool) (BrowserProfile, error)

InferBrowserProfile derives a browser profile from a raw proxy URL.

type Config 

type Config struct {
	Timeout                    time.Duration
	WaitSelector               string
	ProxyURL                   string
	UserAgent                  string
	IgnoreCertErrors           bool
	ExecPath                   string
	StealthScript              string
	AdditionalAllocatorOptions []chromedp.ExecAllocatorOption
}

Config keeps the historical one-shot render surface used by jseval callers.

type HTTPProfile 

type HTTPProfile struct {
	ID               string
	Provider         string
	URL              string
	IgnoreCertErrors bool
}

HTTPProfile describes how to build an HTTP client transport.

func InferHTTPProfile 

func InferHTTPProfile(rawProxyURL string, ignoreCertErrors bool) (HTTPProfile, error)

InferHTTPProfile derives an HTTP transport profile from a raw proxy URL.

type LaunchOptions 

type LaunchOptions struct {
	ExecPath                   string
	UserAgent                  string
	AdditionalAllocatorOptions []chromedp.ExecAllocatorOption
}

LaunchOptions controls browser process launch behavior.

type PageRequest 

type PageRequest struct {
	TargetURL     string
	Timeout       time.Duration
	WaitSelector  string
	StealthScript string
}

PageRequest describes a generic "navigate, wait, capture" render.

type Result 

type Result struct {
	HTML     string
	Title    string
	FinalURL string
}

Result holds the rendered page content.

func RenderPage 

func RenderPage(ctx context.Context, targetURL string, config Config) (*Result, error)

RenderPage launches a one-shot browser session and captures the rendered page.

func RenderPages 

func RenderPages(ctx context.Context, targetURLs []string, config Config) ([]*Result, []error)

RenderPages renders multiple URLs concurrently and returns results in input order.

type Session 

type Session struct {
	// contains filtered or unexported fields
}

Session owns a browser instance bound to one browser transport profile.

func NewSession 

func NewSession(ctx context.Context, browserProfile BrowserProfile, launchOptions LaunchOptions) (*Session, error)

NewSession launches a reusable browser session for the given profile.

func (*Session) Close 

func (session *Session) Close()

Close releases the browser process and any local proxy forwarder.

func (*Session) RenderPage 

func (session *Session) RenderPage(ctx context.Context, pageRequest PageRequest) (*Result, error)

RenderPage renders a page through an existing session.

func (*Session) WithTab 

func (session *Session) WithTab(ctx context.Context, tabOptions TabOptions, run func(context.Context) error) error

WithTab opens a fresh render tab on the session, applies proxy auth and optional preparation actions, then runs the caller callback.

type TabOptions 

type TabOptions struct {
	Timeout        time.Duration
	PreludeActions []chromedp.Action
}

TabOptions controls one render tab opened on an existing browser session.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.