browsertransport

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 28, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package browsertransport provides reusable browser and HTTP transport helpers for proxy-aware scraping runtimes.

Index

Constants

View Source 
const DefaultStealthScript = `` /* 522-byte string literal not displayed */

DefaultStealthScript removes common browser automation markers before page JavaScript runs.

Variables

This section is empty.

Functions

func NewHTTPClient 

func NewHTTPClient(httpProfile HTTPProfile, timeout time.Duration) (*http.Client, error)

NewHTTPClient builds an HTTP client bound to one transport profile.

Types

type BrowserMode 

type BrowserMode string

BrowserMode describes how a browser should reach the upstream network. 

const (
	// BrowserModeDirect uses Chrome's native proxy handling or a direct network path.
	BrowserModeDirect BrowserMode = "direct"
	// BrowserModeHTTPFetchAuth strips inline auth from an HTTP proxy URL and
	// supplies credentials through the Fetch domain.
	BrowserModeHTTPFetchAuth BrowserMode = "http_fetch_auth"
	// BrowserModeSOCKSForwarder bridges an authenticated SOCKS proxy through a
	// local unauthenticated forwarder Chrome can consume.
	BrowserModeSOCKSForwarder BrowserMode = "socks_forwarder"
)

type BrowserProfile 

type BrowserProfile struct {
	ID               string
	Provider         string
	URL              string
	Mode             BrowserMode
	IgnoreCertErrors bool
}

BrowserProfile describes how to launch a browser transport.

func InferBrowserProfile 

func InferBrowserProfile(rawProxyURL string, ignoreCertErrors bool) (BrowserProfile, error)

InferBrowserProfile derives a browser profile from a raw proxy URL.

type Config 

type Config struct {
	Timeout                    time.Duration
	WaitSelector               string
	ProxyURL                   string
	UserAgent                  string
	IgnoreCertErrors           bool
	ExecPath                   string
	StealthScript              string
	AdditionalAllocatorOptions []chromedp.ExecAllocatorOption
}

Config keeps the historical one-shot render surface used by jseval callers.

type HTTPProfile 

type HTTPProfile struct {
	ID               string
	Provider         string
	URL              string
	IgnoreCertErrors bool
}

HTTPProfile describes how to build an HTTP client transport.

func InferHTTPProfile 

func InferHTTPProfile(rawProxyURL string, ignoreCertErrors bool) (HTTPProfile, error)

InferHTTPProfile derives an HTTP transport profile from a raw proxy URL.

type LaunchOptions 

type LaunchOptions struct {
	ExecPath                   string
	UserAgent                  string
	AdditionalAllocatorOptions []chromedp.ExecAllocatorOption
}

LaunchOptions controls browser process launch behavior.

type PageRequest 

type PageRequest struct {
	TargetURL     string
	Timeout       time.Duration
	WaitSelector  string
	StealthScript string
}

PageRequest describes a generic "navigate, wait, capture" render.

type Result 

type Result struct {
	HTML     string
	Title    string
	FinalURL string
}

Result holds the rendered page content.

func RenderPage 

func RenderPage(ctx context.Context, targetURL string, config Config) (*Result, error)

RenderPage launches a one-shot browser session and captures the rendered page.

func RenderPages 

func RenderPages(ctx context.Context, targetURLs []string, config Config) ([]*Result, []error)

RenderPages renders multiple URLs concurrently and returns results in input order.

type Session 

type Session struct {
	// contains filtered or unexported fields
}

Session owns a browser instance bound to one browser transport profile.

func NewSession 

func NewSession(ctx context.Context, browserProfile BrowserProfile, launchOptions LaunchOptions) (*Session, error)

NewSession launches a reusable browser session for the given profile.

func (*Session) Close 

func (session *Session) Close()

Close releases the browser process and any local proxy forwarder.

func (*Session) RenderPage 

func (session *Session) RenderPage(ctx context.Context, pageRequest PageRequest) (*Result, error)

RenderPage renders a page through an existing session.

func (*Session) WithTab 

func (session *Session) WithTab(ctx context.Context, tabOptions TabOptions, run func(context.Context) error) error

WithTab opens a fresh render tab on the session, applies proxy auth and optional preparation actions, then runs the caller callback.

type TabOptions 

type TabOptions struct {
	Timeout        time.Duration
	PreludeActions []chromedp.Action
}

TabOptions controls one render tab opened on an existing browser session.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.