websearch

package
v0.7.10-dev.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 2, 2026 License: MIT Imports: 14 Imported by: 0

Documentation

Overview

Package websearch is web search behind one small interface. The registry brokers the Drafter's `web_search` through the deployment's provider with the platform's key, so no sandbox host holds one; a person's local draft uses the same clients with their own key. Every result is scanned as model input before a model reads it (pkg/judge), which is the caller's job, not this package's: here a result is only fetched and bounded. Both are held to text a reader sees as it is written: no control character and no format character (a bidirectional override, a zero-width character, the soft hyphen, the tag block), which draw nothing or reorder what is drawn. What a result says is the scan's to judge; its shape is Clean's.

Index

Constants

View Source
const (
	BraveName           = "brave"
	BraveDefaultBaseURL = "https://api.search.brave.com"
)

Brave's LLM Context API: the registry's own provider, a search whose results are page chunks extracted for a model to read, in Brave's plain Search plan. Brave's published terms want a plan with storage rights to store results "in part or whole". The route stores none; whether what a draft keeps of its sources counts is an open question.

The call is GET /res/v1/llm/context with the key in X-Subscription-Token. Brave considers its default pool of results and returns at most count URLs, within a small token budget with a per-URL cap, so one page cannot take all of it: every result is scanned before a model reads it, and Clean cuts a snippet to a thousand characters anyway. Only grounding.generic is read, each result's snippets joined into its snippet; local results (grounding.poi, grounding.map, never asked for: enable_local is off) and the sources metadata map are ignored.

View Source
const (
	ExaName           = "exa"
	ExaDefaultBaseURL = "https://api.exa.ai"
)

Exa's search API: a provider a person may name for their own key. The registry's own provider stays Brave. Exa's terms license it to use queries to improve its products, which the platform's own provider must not allow; a person who names Exa chooses those terms for their own searches.

The call is POST /search with the key in x-api-key. It asks for Exa's default search type, auto (not instant, cheaper and shallower: a draft is research), and each result's highlights, the passages most relevant to the query, cut by Exa to the snippet's own bound. Never a summary (billed per page), a deep search, full text or subpages, and never more than ten results, past which Exa bills extra. Only results[] is read; the cost, the images and the favicons are ignored.

View Source
const (
	PerplexityName           = "perplexity"
	PerplexityDefaultBaseURL = "https://api.perplexity.ai"
)

Perplexity's Search API: the raw results endpoint, not its chat completions, and a provider a person may name for their own key. The registry's own provider stays Brave. Perplexity's published no-training statement covers its chat completions, not this endpoint, so the platform provider's no-training condition is not shown to hold for it; a person who names Perplexity chooses its terms for their own searches.

The call is POST /search with the key as a bearer token, one query (an array would be billed as several), the standard web search type (not fast, cheaper and shallower: a draft is research), and a small per-page budget of extracted text, since Clean cuts a snippet to a thousand characters anyway. Only results[] is read.

View Source
const (
	TavilyName           = "tavily"
	TavilyDefaultBaseURL = "https://api.tavily.com"
)

Tavily's Search API: a provider a person may name for their own key. The registry's own provider stays Brave. Tavily's terms keep a right to use queries to train models, which the platform's own provider must not allow; a person who names Tavily chooses those terms for their own searches.

The call is POST /search with the key as a bearer token. It asks for the basic depth (one credit a search), two chunks of each page, and for none of the paid or unscanned extras: no generated answer, no raw page content, no images, and no automatic parameters, which can raise the depth. Only results[] is read.

View Source
const (
	// MaxQuery is the longest query, in characters.
	MaxQuery = 400
	// MaxCount is the most results one search returns; DefaultCount is
	// what a search asks for when the caller names none.
	MaxCount     = 10
	DefaultCount = 5
)

The bounds every result and query is held to.

View Source
const (
	YouComName           = "youcom"
	YouComDefaultBaseURL = "https://ydc-index.io"
)

You.com's Web Search API: a provider a deployment may name, with terms that suit brokering it — a plain REST call with a key, the online MSA's "only use Customer Data as necessary to provide … the Services" and no training on queries, outputs assigned to the customer, no display or attribution rule. The registry's own provider is Brave.

The call is POST /v1/search with the key in X-API-Key. Only the `web` results are read; `news` is ignored. Neither `extraction` (full page text, billed per page) nor `knowledge` (results that carry a display credit) is ever asked for, and the thumbnail and favicon addresses a result carries are dropped, never fetched.

Variables

View Source
var ErrBadQuery = errors.New("websearch: the query must be 1 to 400 characters of text")

ErrBadQuery is a query that is empty, too long, or not text.

View Source
var ErrNoKey = errors.New("websearch: a search provider key is required")

ErrNoKey is a provider built without its key.

Functions

func ClampCount

func ClampCount(n int) int

ClampCount holds a requested result count to 1..MaxCount, with 0 or less meaning DefaultCount.

func CleanQuery

func CleanQuery(q string) (string, error)

CleanQuery drops a query's format characters, trims it and checks it: 1 to MaxQuery characters of valid UTF-8, with no control characters. A format character is dropped, not refused, because some scripts use one in plain writing (a zero-width non-joiner in Persian); a control character is refused.

func Known

func Known(provider string) bool

Known reports whether a configured name is one of Providers; "" is none.

func Origin

func Origin(address string) string

Origin is an address's scheme and host alone, what a caller shows of a result that did not pass its scan: a path or a query can carry an instruction as readily as a snippet can.

func Providers

func Providers() []string

Providers lists the names a deployment may configure, "none" first.

Types

type Brave

type Brave struct {
	// contains filtered or unexported fields
}

Brave is the Brave LLM Context API client. It is safe for concurrent use.

func NewBrave

func NewBrave(cfg BraveConfig) (*Brave, error)

NewBrave builds the client, refusing a missing key.

func (*Brave) Name

func (*Brave) Name() string

Name implements Provider.

func (*Brave) Search

func (b *Brave) Search(ctx context.Context, query string, count int) ([]Result, error)

Search implements Provider.

type BraveConfig

type BraveConfig = ClientConfig

BraveConfig builds a Brave client.

type ClientConfig

type ClientConfig struct {
	// BaseURL is the API root; empty is the provider's own.
	BaseURL string
	// APIKey is required, never logged, and redacted from every error.
	APIKey string
	// HTTPClient carries the requests; nil is a client with a 20-second
	// timeout.
	HTTPClient *http.Client
}

ClientConfig builds a provider's client.

type Exa

type Exa struct {
	// contains filtered or unexported fields
}

Exa is the Exa search API client. It is safe for concurrent use.

func NewExa

func NewExa(cfg ClientConfig) (*Exa, error)

NewExa builds the client, refusing a missing key.

func (*Exa) Name

func (*Exa) Name() string

Name implements Provider.

func (*Exa) Search

func (e *Exa) Search(ctx context.Context, query string, count int) ([]Result, error)

Search implements Provider.

type Fake

type Fake struct {
	Results []Result
	Err     error
	// contains filtered or unexported fields
}

Fake answers every search from a fixed list, for tests. No deployment configuration can name it.

func (*Fake) Name

func (*Fake) Name() string

Name implements Provider.

func (*Fake) Queries

func (f *Fake) Queries() []string

Queries is what was asked, in order.

func (*Fake) Search

func (f *Fake) Search(_ context.Context, query string, count int) ([]Result, error)

Search implements Provider.

type Perplexity

type Perplexity struct {
	// contains filtered or unexported fields
}

Perplexity is the Perplexity Search API client. It is safe for concurrent use.

func NewPerplexity

func NewPerplexity(cfg ClientConfig) (*Perplexity, error)

NewPerplexity builds the client, refusing a missing key.

func (*Perplexity) Name

func (*Perplexity) Name() string

Name implements Provider.

func (*Perplexity) Search

func (p *Perplexity) Search(ctx context.Context, query string, count int) ([]Result, error)

Search implements Provider.

type Provider

type Provider interface {
	// Name is the provider's short id, as configuration names it.
	Name() string
	// Search asks for count results for query. A provider returns what its
	// upstream sends, which may be more than it was asked for, so the bound
	// on the results a caller uses is Clean's, not the provider's. An error
	// is the provider's failure; its text never carries the key.
	Search(ctx context.Context, query string, count int) ([]Result, error)
}

Provider searches the web.

func New

func New(provider, baseURL, apiKey string) (Provider, error)

New builds the provider a deployment names: "" or "none" is no provider (nil, nil), "youcom" is You.com, "brave" is Brave's LLM Context API, and "tavily", "exa" and "perplexity" are those providers' search APIs. An unknown name, or a named provider without its key, is an error the server refuses to start on.

type Result

type Result struct {
	Title   string
	URL     string
	Snippet string
}

Result is one web search result: what a model is shown of it.

func Clean

func Clean(rs []Result, count int) []Result

Clean bounds what a provider returned: an absolute http or https address with a host and no user information, at most maxURL bytes; a title and a snippet of valid UTF-8 with white space folded, control and format characters dropped, cut to their bounds with no space at either end; no address twice; at most count results. A result without a usable address is dropped, since nothing could name it.

type StatusError

type StatusError struct {
	Provider string
	Status   int
	Snippet  string
}

StatusError is a provider's non-200 answer: its status and a bounded snippet of the body with the key redacted. The snippet can echo the query, so a caller that must not log queries logs Status alone.

func (*StatusError) Error

func (e *StatusError) Error() string

type Tavily

type Tavily struct {
	// contains filtered or unexported fields
}

Tavily is the Tavily Search API client. It is safe for concurrent use.

func NewTavily

func NewTavily(cfg ClientConfig) (*Tavily, error)

NewTavily builds the client, refusing a missing key.

func (*Tavily) Name

func (*Tavily) Name() string

Name implements Provider.

func (*Tavily) Search

func (t *Tavily) Search(ctx context.Context, query string, count int) ([]Result, error)

Search implements Provider.

type YouCom

type YouCom struct {
	// contains filtered or unexported fields
}

YouCom is the You.com Web Search API client. It is safe for concurrent use.

func NewYouCom

func NewYouCom(cfg YouComConfig) (*YouCom, error)

NewYouCom builds the client, refusing a missing key.

func (*YouCom) Name

func (*YouCom) Name() string

Name implements Provider.

func (*YouCom) Search

func (y *YouCom) Search(ctx context.Context, query string, count int) ([]Result, error)

Search implements Provider.

type YouComConfig

type YouComConfig = ClientConfig

YouComConfig builds a YouCom client.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL