websearch

Overview ¶

Package websearch is a web search and a page fetch your agents can call.

It exposes Hanzo-native Web Search + Scrape on the unified cloud-api /v1 plane, so hanzo.chat's web_search agent tool runs entirely on Hanzo infrastructure with NO external SaaS provider, per HIP-0106.

hanzo.chat (LibreChat fork) implements web_search as a fixed 3-stage pipeline whose provider contracts are frozen by the upstream client (@librechat/agents tools/search). The only self-hostable, key-less-to-a-SaaS providers it accepts are:

• search provider "searxng" → GET {searxngInstanceUrl}/search?q=&format=json ← {results:[{url,title,content,img_src?}]}
• scraper provider "firecrawl" → POST {firecrawlApiUrl}/{version}/scrape body {url,formats} ← {success,data:{markdown,metadata}} (reranker is optional; we omit it — provider+scraper is sufficient.)

This subsystem serves BOTH contracts, backed by Hanzo's own services — never a third-party search API:

• GET /v1/websearch/search SearXNG-shaped. Served NATIVELY in-process by a keyless Go meta-search (search.go) — no SearXNG pod, no search SaaS.
• POST /v1/scrape Firecrawl-shaped. Served NATIVELY in-process by clients/crawl — fetch, extract, render — returning {success,data:{markdown,metadata}}.

Both halves are now in-process Go, for the same reason and by the same shape: a keyless meta-search here, a fetch-and-extract in clients/crawl. Neither has a pod to be down. Scrape previously dialled a separate crawler at crawl.hanzo.svc that did NOT exist — the name was NXDOMAIN — so this surface answered 200 while every scrape inside it returned success:false. clients/crawl is the same capability with no network hop and no second deployment to keep alive.

The chat server calls these SERVER-SIDE in-cluster, so point searxngInstanceUrl / firecrawlApiUrl at this surface (public api.hanzo.ai/v1 or the internal cloud-api svc DNS — same binary either way).

AUTH: two callers, two ONE-WAY-equivalent gates, never an open proxy —

• SEARCH (/v1/websearch/search) admits EITHER a validated principal (principal.Validated — X-User-Id minted by the identity middleware from a verified JWT: the signed-in console user via the /cloud bearer proxy) OR the shared service key WEBSEARCH_API_KEY as X-API-Key (the hanzo.chat server, which reaches cloud service-to-service with no user principal). A caller with neither is refused.
• SCRAPE (/v1/scrape) requires the shared key as a Bearer (the chat server path only; the console surfaces scrape read-only, does not drive it).

An unset key 503s and any missing/mismatched key 401s on the key path; a request with a validated principal never needs the key. So neither surface is ever an open proxy, and the signed-in console user reaches search without the shared key.

WHY NOTHING HERE IS A TYPED OP (re-verified at zip v1.18.12), so the next sweep does not re-litigate it. Both surfaces exist to be BYTE-COMPATIBLE with a client this repo does not own — LibreChat's frozen searxng and firecrawl contracts — and each is compatible in a way a typed op structurally cannot be:

• /v1/websearch/search is registered with All (Mount, below), so it answers every method in the router's set — today delete, get, options, patch, post, put and trace. zip has no typed `All`, and declaring the five named verbs instead would DROP options and trace from the path: a routing change, not a description. The POST/PUT/PATCH arms also read their query string and IGNORE the body entirely, while a typed op 400s on any unparseable non-empty body (typed.go op.invoke) — so those arms cannot be typed even one at a time.
• /v1/scrape deliberately answers 200 {"success":false,"error":"missing url"} to a malformed or oversized body (scrapeScoped, below): firecrawl clients read data.success, not the status line, and it caps the read at 1 MiB with an io.LimitReader rather than refusing. A typed op cannot express either — the 400 is raised before the handler runs, and the cap is invisible to it.

The route that unblocks the first is a typed `All` in zip; the second needs a body-TOLERANT op. Until then this subsystem is honestly untyped: eight operations, no MCP tool, no SDK method. Their PROSE is not part of that cost — it is declared through openapi.Describe beside the route table (Mount, below), which is the seam for exactly the operations the wire refuses to type.