Documentation
¶
Overview ¶
Package firecrawl provides a client for the Firecrawl REST API.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client calls the Firecrawl scrape API.
func NewClient ¶
func NewClient(cfg config.FirecrawlConfig) *Client
NewClient constructs a Firecrawl client from runtime configuration.
func (*Client) Scrape ¶
Scrape converts a source URL into markdown through Firecrawl with Firecrawl's default freshness options.
func (*Client) ScrapeWithOptions ¶ added in v0.0.11
func (client *Client) ScrapeWithOptions(ctx context.Context, sourceURL string, opts ScrapeOptions) (*ScrapeResult, error)
ScrapeWithOptions is Scrape with explicit freshness options; only the options that are set are sent.
type ScrapeOptions ¶ added in v0.0.11
type ScrapeOptions struct {
// MaxAge is the maximum cache age in milliseconds; 0 bypasses the cache.
MaxAge *int64
// WaitFor is the delay in milliseconds before capture (client-side rendering).
WaitFor int
// OnlyMainContent toggles Firecrawl's main-content filter.
OnlyMainContent *bool
}
ScrapeOptions are the optional freshness knobs of a scrape request (spec §7 stage 4). A nil pointer or zero WaitFor leaves the field out of the request so Firecrawl applies its default.
func RefetchOptions ¶ added in v0.0.11
func RefetchOptions(cfg config.FirecrawlConfig) ScrapeOptions
RefetchOptions returns the fresh-scrape options used before a quality gate judges a thin or broken capture: maxAge 0, waitFor refetch_wait_ms and onlyMainContent refetch_only_main_content.
type ScrapeResult ¶
type ScrapeResult struct {
Markdown string
Title string
SourceURL string
// StatusCode is the HTTP status Firecrawl reports for the scraped page
// (metadata.statusCode); 0 when absent.
StatusCode int
// FinalURL is the page URL after redirects (metadata.url) when it differs
// from SourceURL; empty otherwise.
FinalURL string
// SiteName is the site name the page declares (metadata.ogSiteName,
// og:site_name or siteName); empty when absent. The ingest gate uses it
// for the "title equals the site name" rule.
SiteName string
}
ScrapeResult contains the normalized Firecrawl scrape payload used by ingest.
Source Files