Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrAuthWall = fmt.Errorf("hermai: page requires authentication")
ErrAuthWall is returned when a page requires authentication.
Functions ¶
func DetectAuth ¶
func DetectAuth(signals AuthSignals) bool
DetectAuth returns true if the given signals indicate the page is behind an authentication wall.
Checks (any match returns true):
- HTTP 401 or 403
- FinalURL contains a login-related path (case-insensitive)
- DOMSnapshot contains login-related content (case-insensitive)
Types ¶
type AuthSignals ¶
AuthSignals holds the signals used to detect auth walls.
type CaptureOpts ¶
type CaptureOpts struct {
ProxyURL string
BrowserPath string
Timeout time.Duration
WaitAfterLoad time.Duration
Cookies []string // name=value pairs to inject before navigation
// Headful launches a visible Chrome window so the operator can
// perform real-user interactions (save draft, add to cart, etc.)
// while the capture runs. Default false — headless for CI/automated
// discovery.
Headful bool
}
CaptureOpts holds options for a browser capture session.
type CaptureResult ¶
CaptureResult holds the output of a browser capture session.
type HAREntry ¶
type HAREntry struct {
Request HARRequest `json:"request"`
Response HARResponse `json:"response"`
}
HAREntry represents a single HTTP request/response pair.
type HARLog ¶
type HARLog struct {
Entries []HAREntry `json:"entries"`
}
HARLog represents a simplified HAR log containing captured HTTP entries.
func FilterHAR ¶
FilterHAR returns a new HARLog with noise removed. The input is not mutated.
Stage 1: Keep only entries whose response content-type matches an allowed API type. Stage 2: Drop entries whose URL contains known analytics/tracking patterns. Stage 3: Deduplicate by Method + URL, keeping the first occurrence.
type HARRequest ¶
type HARRequest struct {
Method string `json:"method"`
URL string `json:"url"`
Headers map[string]string `json:"headers"`
Body string `json:"body,omitempty"`
}
HARRequest represents an HTTP request.
type HARResponse ¶
type HARResponse struct {
Status int `json:"status"`
StatusText string `json:"status_text"`
ContentType string `json:"content_type"`
Headers map[string]string `json:"headers"`
Body string `json:"body,omitempty"`
}
HARResponse represents an HTTP response.
type RodBrowser ¶
type RodBrowser struct {
// contains filtered or unexported fields
}
RodBrowser implements the Service interface using CDP protocol. Supports two backends:
- Lightpanda (preferred): 9x less memory, 11x faster, instant startup
- Chromium via Rod launcher (fallback): full browser compatibility
When Lightpanda returns thin/empty HTML (SPA not rendered), the browser automatically falls back to Chromium for a second capture attempt. Domains that required Chromium fallback are recorded in ~/.hermai/spa_domains.txt so subsequent visits skip Lightpanda entirely (avoiding double-latency).
func NewRodBrowser ¶
func NewRodBrowser(browserPath string) (*RodBrowser, error)
NewRodBrowser creates a browser instance. Connection priority:
- If cdpURL is set, connect to that CDP endpoint directly (Lightpanda or remote browser)
- If Lightpanda is running on default port 9222, connect to it
- Fall back to launching Chromium via Rod
func NewRodBrowserWithCDP ¶
func NewRodBrowserWithCDP(cdpURL, browserPath string) (*RodBrowser, error)
NewRodBrowserWithCDP connects to an explicit CDP WebSocket URL. Use this for Lightpanda, remote browsers, or Browserless.io. browserPath is stored for Chromium fallback when the CDP backend returns thin HTML (SPA not rendered).
func (*RodBrowser) Backend ¶
func (rb *RodBrowser) Backend() string
Backend returns which browser engine is in use ("lightpanda" or "chromium").
func (*RodBrowser) Capture ¶
func (rb *RodBrowser) Capture(ctx context.Context, targetURL string, opts CaptureOpts) (*CaptureResult, error)
Capture navigates to the target URL, captures network traffic as HAR, and extracts a simplified DOM snapshot. Uses CDP protocol events for passive network observation. Works with both Lightpanda and Chromium backends.
When Lightpanda returns thin rendered HTML (SPA not rendered), Capture automatically retries with a temporary Chromium instance for full JavaScript execution. This ensures SPAs (React, Next.js, Vue) still work.
func (*RodBrowser) Close ¶
func (rb *RodBrowser) Close() error
Close shuts down the browser and launcher (if any).
func (*RodBrowser) SetSPADomainsFile ¶
func (rb *RodBrowser) SetSPADomainsFile(path string)
SetSPADomainsFile enables auto-learning of SPA domains that require Chromium. The file records domains where Lightpanda returned thin HTML. On subsequent visits to these domains, Lightpanda is skipped entirely (saves 5-10s).
type Service ¶
type Service interface {
Capture(ctx context.Context, targetURL string, opts CaptureOpts) (*CaptureResult, error)
Close() error
}
Service defines the browser capture interface.
Source Files