discoverpage

func CaptureScreenshot ¶

func CaptureScreenshot(ctx context.Context, browser *headless.Requester, sendHTTPRequestConfig *common.SendHttpRequestConfig) ([]byte, error)

CaptureScreenshot uses a headless browser to capture a screenshot of the given URL and returns the image bytes.

func ComputeScreenshotPerceptualHash ¶ added in v0.0.278

func ComputeScreenshotPerceptualHash(pngBytes []byte) (string, error)

ComputeScreenshotPerceptualHash decodes PNG bytes and returns a perceptual hash string in the form "p:<hex>" (the default goimagehash format, compatible with gowitness).

func ExtractFaviconURL ¶ added in v0.0.278

func ExtractFaviconURL(html, finalURL string) string

ExtractFaviconURL parses rendered HTML for a favicon link tag and resolves the href as an absolute URL against finalURL. Falls back to <scheme>://<host>/favicon.ico if no link tag is found.

func ExtractSensitiveContentsFromWebContent ¶ added in v0.0.207

func ExtractSensitiveContentsFromWebContent(ctx context.Context, content string, fingerprints *discover.SensitiveContentFingerprints) ([]*discover.SensitiveContent, []string)

ExtractSensitiveContentsFromWebContent searches for sensitive content in web content

func FetchFavicon ¶ added in v0.0.278

func FetchFavicon(ctx context.Context, faviconURL string, timeout int, verifyTLS bool, userAgent string) ([]byte, string, error)

FetchFavicon downloads the favicon at faviconURL using a fresh HTTP client that respects verifyTLS and timeout. The caller supplies userAgent (typically the resolved discover-page UA) so favicon traffic matches what the headless browser advertised — servers that vary their favicon on UA work correctly.

IMPORTANT: a fresh context is derived from `ctx` instead of using `ctx` as a deadline-bearing parent. The combined headless capture exhausts most of its own `requestCtx` window during navigation + DOM stabilization; if we reused that deadline here, the favicon fetch could see ~0s of slack and always time out. We still inherit cancellation by piggy-backing on `ctx.Done()` (so a parent cancel propagates) but reset the timeout to a fresh budget.

Returns the raw bytes and a Shodan-compatible mmh3-32 hash (signed int32, decimal string). On non-2xx, network error, or empty body, it returns (nil, "", nil) — caller treats this as not found.

func LoadSensitiveConentFingerprints ¶ added in v0.0.207

func LoadSensitiveConentFingerprints(ctx context.Context, configPath *string) (*discover.SensitiveContentFingerprints, error)

LoadSensitiveConentFingerprints loads fingerprints from a custom file path or falls back to the embedded default fingerprints baked into the binary.

func PerformHTMLPageCapture ¶

func PerformHTMLPageCapture(ctx context.Context, config *common.SendHttpRequestConfig) (*common.HttpRequestResponse, error)

PerformHTMLPageCapture sends an HTTP request using the provided config and returns the response.