websearch

Overview ¶

Package websearch exposes Hanzo-native Web Search + Scrape on the unified cloud-api /v1 plane, so hanzo.chat's web_search agent tool runs entirely on Hanzo infrastructure with NO external SaaS provider, per HIP-0106.

hanzo.chat (LibreChat fork) implements web_search as a fixed 3-stage pipeline whose provider contracts are frozen by the upstream client (@librechat/agents tools/search). The only self-hostable, key-less-to-a-SaaS providers it accepts are:

• search provider "searxng" → GET {searxngInstanceUrl}/search?q=&format=json ← {results:[{url,title,content,img_src?}]}
• scraper provider "firecrawl" → POST {firecrawlApiUrl}/{version}/scrape body {url,formats} ← {success,data:{markdown,metadata}} (reranker is optional; we omit it — provider+scraper is sufficient.)

This subsystem serves BOTH contracts under /v1/websearch, backed by Hanzo's own services — never a third-party search API:

• GET /v1/websearch/search SearXNG-shaped. Proxied to a Hanzo-operated metasearch instance (WEBSEARCH_UPSTREAM).
• POST /v1/websearch/v1/scrape Firecrawl-shaped. Backed by Hanzo Crawl (also /v1/websearch/scrape) (crawl.hanzo.svc, Crawl4AI): fetch the URL, return {success,data:{markdown,metadata}}.

The chat server calls these SERVER-SIDE in-cluster, so point searxngInstanceUrl / firecrawlApiUrl at this surface (public api.hanzo.ai/v1 or the internal cloud-api svc DNS — same binary either way).

AUTH: firecrawl requires a Bearer key; searxng accepts an optional X-API-Key. Both are the same shared service key WEBSEARCH_API_KEY (KMS-sourced, synced into the pod env). An unset key fails closed on the authed scrape path.