Documentation
¶
Overview ¶
Package web defines provider-neutral web search and page-fetching tools.
Searcher and Fetcher are independent capabilities because not every provider supports both. A provider client may implement either or both; each provider owns one package and one transport client. NewSearchTool and NewFetchTool adapt those capabilities to the core tool contract.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( ErrMissingFetchRequest = errors.New("web: fetch request must not be nil") ErrEmptyURL = errors.New("web: fetch URL must not be empty") ErrInvalidURL = errors.New("web: fetch URL must be an absolute http(s) URL") ErrInvalidFormat = errors.New("web: content format must be markdown, html, or text") ErrMissingFetcher = errors.New("web: fetcher is required") ErrMissingFetchResponse = errors.New("web: fetch response must not be nil") ErrInvalidFetchResponse = errors.New("web: fetch response is invalid") ErrUnsupportedFormat = errors.New("web: content format is not supported by provider") )
var ( ErrMissingSearchRequest = errors.New("web: search request must not be nil") ErrEmptyQuery = errors.New("web: search query must not be empty") ErrDomainsBothSides = errors.New("web: allowed_domains and blocked_domains are mutually exclusive") ErrInvalidMaxResults = errors.New("web: max_results must be between 1 and 20 when set") ErrTooManyDomains = errors.New("web: at most 20 allowed_domains or blocked_domains may be set") ErrInvalidDomain = errors.New("web: domain filter must be a bare hostname without scheme, port, path, query, or fragment") ErrInvalidRecency = errors.New("web: recency must be hour, day, week, month, or year") ErrMissingSearcher = errors.New("web: searcher is required") ErrMissingSearchResponse = errors.New("web: search response must not be nil") ErrInvalidSearchResponse = errors.New("web: search response is invalid") ErrUnsupportedFilter = errors.New("web: search filter is not supported by provider") )
Functions ¶
This section is empty.
Types ¶
type ContentFormat ¶
type ContentFormat string
ContentFormat selects the representation of fetched page content. Providers map this to their native format setting.
const ( // FormatMarkdown returns the page rendered to Markdown. This is // the default and usually the most LLM-friendly format. FormatMarkdown ContentFormat = "markdown" // FormatHTML returns the page's HTML (or a cleaned variant). FormatHTML ContentFormat = "html" // FormatText returns plain text — no markup, no structure. FormatText ContentFormat = "text" )
func (ContentFormat) Resolve ¶
func (c ContentFormat) Resolve() ContentFormat
func (ContentFormat) Validate ¶
func (c ContentFormat) Validate() error
type FetchRequest ¶
type FetchRequest struct {
// URL is the page to fetch. Required.
URL string `json:"url" jsonschema:"minLength=1" jsonschema_description:"Absolute http(s) URL of the page to fetch."`
// Format selects the response format. "" defaults to markdown.
Format ContentFormat `` /* 178-byte string literal not displayed */
}
FetchRequest is both the provider-neutral fetch contract and the LLM-facing argument shape.
func (*FetchRequest) Prepare ¶
func (f *FetchRequest) Prepare() (*FetchRequest, error)
Prepare returns a normalized and validated request without mutating f.
func (*FetchRequest) Validate ¶
func (f *FetchRequest) Validate() error
type FetchResponse ¶
type FetchResponse struct {
Content string `json:"content"`
Format ContentFormat `json:"format"`
}
FetchResponse is the normalized scrape result. Used as both the SPI return type and the LLM-facing serialization shape.
func (*FetchResponse) Validate ¶
func (f *FetchResponse) Validate() error
type FetchTool ¶
type FetchTool struct {
// contains filtered or unexported fields
}
FetchTool is the LLM-facing adapter for a Fetcher. Construct with NewFetchTool — there is no nil-default fallback because rendering modern web pages reliably requires an upstream API.
func NewFetchTool ¶
NewFetchTool requires a fetcher for the same reason the shell tool requires an executor: retrieving a model-supplied URL is an SSRF boundary, and defaulting it would grant network reach the caller never chose.
func (FetchTool) Call ¶
func (r FetchTool) Call(ctx context.Context, invocation toolcontract.Invocation) (chat.ToolOutput, error)
func (FetchTool) ConcurrencyKey ¶
func (FetchTool) ConcurrencyKey(toolcontract.Invocation) (key string, concurrent bool)
Network reads have no local resource conflict, so independent calls may run concurrently under the tool executor's optional scheduling contract.
func (FetchTool) Definition ¶
func (r FetchTool) Definition() chat.ToolDefinition
type Fetcher ¶
type Fetcher interface {
// Fetch retrieves and renders exactly request.URL in the requested format
// without mutating or retaining request. Implementations must honor ctx,
// preserve network error causes, and transfer response ownership to the
// caller.
Fetch(ctx context.Context, request *FetchRequest) (*FetchResponse, error)
}
Fetcher is the provider boundary behind the model-facing page fetch tool. Network authority, authentication, redirects, and provider defaults are frozen in the implementation rather than supplied by model arguments.
type Recency ¶
type Recency string
Recency is a coarse "last N period" filter. Providers map this to their native syntax (e.g. Tavily's time_range, Serper's tbs=qdr:).
type SearchRequest ¶
type SearchRequest struct {
// Query is the search string. Required.
Query string `` /* 155-byte string literal not displayed */
// MaxResults caps the number of returned results. 0 = use the
// provider's default (typically 5-10).
MaxResults int `` /* 188-byte string literal not displayed */
// AllowedDomains restricts results to these domains. Mutually
// exclusive with BlockedDomains on most providers.
AllowedDomains []string `` /* 202-byte string literal not displayed */
// BlockedDomains drops results from these domains.
BlockedDomains []string `` /* 197-byte string literal not displayed */
// Recency filters to a coarse time-window. "" = no time filter.
Recency Recency `` /* 167-byte string literal not displayed */
}
SearchRequest is both the provider-neutral search contract and the LLM-facing argument shape.
func (*SearchRequest) Prepare ¶
func (s *SearchRequest) Prepare() (*SearchRequest, error)
Prepare returns an independently owned, normalized request after validating it. The receiver is never mutated.
func (*SearchRequest) QueryWithSiteOperators ¶
func (s *SearchRequest) QueryWithSiteOperators() string
QueryWithSiteOperators returns Query with Google-style site:/-site: operators for the request's domain filters. Providers without native domain fields use this projection; empty domain entries are ignored.
func (*SearchRequest) Validate ¶
func (s *SearchRequest) Validate() error
type SearchResponse ¶
type SearchResponse struct {
Query string `json:"query"`
Results []*SearchResult `json:"results"`
}
SearchResponse carries the executed query plus normalized results. Used as both the SPI return type and the LLM-facing serialization shape.
func (*SearchResponse) Validate ¶
func (s *SearchResponse) Validate() error
type SearchResult ¶
type SearchResult struct {
Title string `json:"title"`
URL string `json:"url"`
Snippet string `json:"snippet"`
FaviconURL string `json:"favicon_url,omitempty"`
PublishedTime time.Time `json:"published_time,omitzero"`
Source string `json:"source,omitempty"`
}
SearchResult is one normalized search hit.
type SearchTool ¶
type SearchTool struct {
// contains filtered or unexported fields
}
SearchTool is the LLM-facing adapter for a Searcher. Construct with NewSearchTool — there is no nil-default fallback because web search inherently requires an upstream API.
func NewSearchTool ¶
func NewSearchTool(searcher Searcher) (*SearchTool, error)
NewSearchTool requires a searcher so the provider, credential, and quota are the caller's explicit choice rather than an implicit dependency on whichever backend happened to be compiled in.
func (SearchTool) Call ¶
func (r SearchTool) Call(ctx context.Context, invocation toolcontract.Invocation) (chat.ToolOutput, error)
func (SearchTool) ConcurrencyKey ¶
func (SearchTool) ConcurrencyKey(toolcontract.Invocation) (key string, concurrent bool)
Network reads have no local resource conflict, so independent calls may run concurrently under the tool executor's optional scheduling contract.
func (SearchTool) Definition ¶
func (r SearchTool) Definition() chat.ToolDefinition
type Searcher ¶
type Searcher interface {
// Search performs one request without mutating or retaining it and transfers
// ownership of a normalized response to the caller. Implementations must
// honor ctx, preserve provider error causes, and reject unsupported explicit
// fields instead of silently ignoring them.
Search(ctx context.Context, request *SearchRequest) (*SearchResponse, error)
}
Searcher is the provider boundary behind the model-facing search tool. It receives only the normalized provider-neutral contract; authentication, endpoint selection, and provider defaults are frozen in the implementation.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package brave wires Brave's Web Search API into web.Searcher.
|
Package brave wires Brave's Web Search API into web.Searcher. |
|
Package exa integrates Exa's Search and Contents APIs with the provider-neutral web contracts.
|
Package exa integrates Exa's Search and Contents APIs with the provider-neutral web contracts. |
|
Package firecrawl integrates Firecrawl's Search and Scrape APIs with the provider-neutral web contracts.
|
Package firecrawl integrates Firecrawl's Search and Scrape APIs with the provider-neutral web contracts. |
|
Package jina integrates Jina Search and Jina Reader with the provider-neutral web contracts.
|
Package jina integrates Jina Search and Jina Reader with the provider-neutral web contracts. |
|
Package perplexity wires Perplexity's Search API into web.Searcher.
|
Package perplexity wires Perplexity's Search API into web.Searcher. |
|
Package serper wires Serper's Google Search API into web.Searcher.
|
Package serper wires Serper's Google Search API into web.Searcher. |
|
Package tavily integrates Tavily Search and Extract with the provider-neutral web contracts.
|
Package tavily integrates Tavily Search and Extract with the provider-neutral web contracts. |