probe

package
v0.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 24, 2026 License: AGPL-3.0 Imports: 16 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IsZillowSearch

func IsZillowSearch(targetURL string) bool

IsZillowSearch returns true if the URL is a Zillow search page (not a detail page, not a tool page).

func NewClient added in v0.1.5

func NewClient(opts Options) httpclient.Doer

NewClient builds an HTTP client from probe options. Follows up to 10 redirects so callers see the final response instead of 301/302 hops.

func NormalizeZillowSearchURL

func NormalizeZillowSearchURL(targetURL string) (string, error)

NormalizeZillowSearchURL rewrites a Zillow search URL to include searchQueryState in the query string. Zillow slug URLs like /san-francisco-ca/rentals/5-_beds/ do NOT filter data — only searchQueryState does.

If searchQueryState is already present, the URL is returned unchanged.

func SetBrowserHeaders added in v0.1.5

func SetBrowserHeaders(req *http.Request)

SetBrowserHeaders sets headers that match a real Chrome browser navigation. Cloudflare and similar WAFs check Sec-Fetch-* headers to distinguish real browsers from HTTP libraries.

Types

type Candidate

type Candidate struct {
	Schema   *schema.Schema
	Strategy string
	Score    int
}

Candidate is one successful probe-discovered schema candidate.

type Options

type Options struct {
	ProxyURL string
	Timeout  time.Duration
	Insecure bool // skip TLS certificate verification
	Stealth  bool // use TLS+HTTP/2 fingerprinting (Chrome profile)
}

Options configures probe behavior.

type Result

type Result struct {
	Schema          *schema.Schema
	Strategy        string // best-ranked candidate strategy
	Candidates      []Candidate
	HTMLBody        string // If probe got HTML instead of JSON, the body is stored here for reuse
	RequiresStealth bool   // true if stealth TLS was needed to bypass anti-bot
}

Result contains what the probe found.

func Probe

func Probe(ctx context.Context, targetURL string, opts Options) (*Result, error)

Probe tries to detect direct JSON access patterns for a URL. Successful candidates are scored so deterministic, reusable endpoints beat whichever request happens to finish first.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL