Documentation
¶
Overview ¶
Package web defines provider-neutral web search and page-fetching tools.
Searcher and Fetcher are independent capabilities because not every provider supports both. A provider client may implement either or both; each provider owns one package and one transport client. NewSearchTool and NewFetchTool adapt those capabilities to the core tool contract.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( ErrMissingFetchRequest = errors.New("web: fetch request must not be nil") ErrEmptyURL = errors.New("web: fetch URL must not be empty") ErrInvalidURL = errors.New("web: fetch URL must be an absolute http(s) URL") ErrInvalidFormat = errors.New("web: content format must be markdown, html, or text") ErrMissingFetcher = errors.New("web: fetcher is required") ErrMissingFetchResponse = errors.New("web: fetch response must not be nil") ErrInvalidFetchResponse = errors.New("web: fetch response is invalid") ErrUnsupportedFormat = errors.New("web: content format is not supported by provider") )
var ( ErrMissingSearchRequest = errors.New("web: search request must not be nil") ErrEmptyQuery = errors.New("web: search query must not be empty") ErrDomainsBothSides = errors.New("web: allowed_domains and blocked_domains are mutually exclusive") ErrInvalidMaxResults = errors.New("web: max_results must be between 1 and 20 when set") ErrTooManyDomains = errors.New("web: at most 20 allowed_domains or blocked_domains may be set") ErrInvalidDomain = errors.New("web: domain filter must be a bare hostname without scheme, port, path, query, or fragment") ErrInvalidRecency = errors.New("web: recency must be hour, day, week, month, or year") ErrMissingSearcher = errors.New("web: searcher is required") ErrMissingSearchResponse = errors.New("web: search response must not be nil") ErrInvalidSearchResponse = errors.New("web: search response is invalid") ErrUnsupportedFilter = errors.New("web: search filter is not supported by provider") )
Functions ¶
This section is empty.
Types ¶
type ContentFormat ¶
type ContentFormat string
ContentFormat selects the representation of fetched page content. Providers map this to their native format setting.
const ( // FormatMarkdown returns the page rendered to Markdown. This is // the default and usually the most LLM-friendly format. FormatMarkdown ContentFormat = "markdown" // FormatHTML returns the page's HTML (or a cleaned variant). FormatHTML ContentFormat = "html" // FormatText returns plain text — no markup, no structure. FormatText ContentFormat = "text" )
func (ContentFormat) Normalize ¶ added in v0.24.0
func (c ContentFormat) Normalize() (ContentFormat, error)
Normalize applies the default and rejects unsupported formats.
func (ContentFormat) Validate ¶
func (c ContentFormat) Validate() error
type FetchRequest ¶
type FetchRequest struct {
// URL is the page to fetch. Required.
URL string `json:"url" jsonschema:"minLength=1" jsonschema_description:"Absolute http(s) URL of the page to fetch."`
// Format selects the response format. "" defaults to markdown.
Format ContentFormat `` /* 178-byte string literal not displayed */
}
FetchRequest is both the provider-neutral fetch contract and the LLM-facing argument shape.
func (*FetchRequest) Prepare ¶
func (f *FetchRequest) Prepare() (*FetchRequest, error)
Prepare returns a normalized and validated request without mutating f.
func (*FetchRequest) Validate ¶
func (f *FetchRequest) Validate() error
type FetchResponse ¶
type FetchResponse struct {
Content string `json:"content"`
Format ContentFormat `json:"format"`
}
FetchResponse is the normalized scrape result. Used as both the SPI return type and the LLM-facing serialization shape.
func (*FetchResponse) Validate ¶
func (f *FetchResponse) Validate() error
type FetchTool ¶
type FetchTool struct {
// contains filtered or unexported fields
}
FetchTool is the LLM-facing adapter for a Fetcher. Construct with NewFetchTool — there is no nil-default fallback because rendering modern web pages reliably requires an upstream API.
func NewFetchTool ¶
NewFetchTool requires a fetcher for the same reason the shell tool requires an executor: retrieving a model-supplied URL is an SSRF boundary, and defaulting it would grant network reach the caller never chose.
func (FetchTool) Call ¶
func (r FetchTool) Call(ctx context.Context, invocation toolcontract.Invocation) (chat.ToolOutput, error)
func (FetchTool) ConcurrencyPolicy ¶ added in v0.19.0
func (FetchTool) ConcurrencyPolicy() func(toolcontract.Invocation) (string, bool)
ConcurrencyPolicy declares independent calls to the concurrent-safe backend.
func (FetchTool) Definition ¶
func (r FetchTool) Definition() chat.ToolDefinition
func (FetchTool) Unwrap ¶ added in v0.19.0
func (r FetchTool) Unwrap() toolcontract.Tool
Unwrap exposes the typed input contract through tool decorators.
type Fetcher ¶
type Fetcher interface {
// Fetch retrieves and renders exactly request.URL in the requested format
// without mutating or retaining request. Implementations must honor ctx,
// preserve network error causes, and transfer response ownership to the
// caller.
Fetch(ctx context.Context, request *FetchRequest) (*FetchResponse, error)
}
Fetcher is the provider boundary behind the model-facing page fetch tool. Network authority, authentication, redirects, and provider defaults are frozen in the implementation rather than supplied by model arguments. Implementations must support concurrent calls, including calls through other tools sharing the same backend; fetch tools advertise parallel use.
type Recency ¶
type Recency string
Recency is a coarse "last N period" filter. Providers map this to their native syntax (e.g. Tavily's time_range, Serper's tbs=qdr:).
type SearchRequest ¶
type SearchRequest struct {
// Query is the search string. Required.
Query string `` /* 155-byte string literal not displayed */
// MaxResults caps the number of returned results. 0 = use the
// provider's default (typically 5-10).
MaxResults int `` /* 188-byte string literal not displayed */
// AllowedDomains restricts results to any of these domains. Mutually
// exclusive with BlockedDomains on most providers.
AllowedDomains []string `` /* 202-byte string literal not displayed */
// BlockedDomains drops results from these domains.
BlockedDomains []string `` /* 197-byte string literal not displayed */
// Recency filters to a coarse time-window. "" = no time filter.
Recency Recency `` /* 167-byte string literal not displayed */
}
SearchRequest is both the provider-neutral search contract and the LLM-facing argument shape.
func (*SearchRequest) Prepare ¶
func (s *SearchRequest) Prepare() (*SearchRequest, error)
Prepare returns an independently owned, normalized request after validating it. The receiver is never mutated.
func (*SearchRequest) QueryWithSiteOperators ¶
func (s *SearchRequest) QueryWithSiteOperators() string
QueryWithSiteOperators returns Query with Google-style site:/-site: operators for the request's domain filters. Providers without native domain fields use this projection. Allowed domains form an OR group; every blocked domain is excluded. Empty domain entries are ignored.
func (*SearchRequest) Validate ¶
func (s *SearchRequest) Validate() error
type SearchResponse ¶
type SearchResponse struct {
Query string `json:"query"`
Results []*SearchResult `json:"results"`
}
SearchResponse carries the executed query plus normalized results. Used as both the SPI return type and the LLM-facing serialization shape.
func (*SearchResponse) Validate ¶
func (s *SearchResponse) Validate() error
type SearchResult ¶
type SearchResult struct {
Title string `json:"title"`
URL string `json:"url"`
Snippet string `json:"snippet"`
FaviconURL string `json:"favicon_url,omitempty"`
PublishedTime time.Time `json:"published_time,omitzero"`
Source string `json:"source,omitempty"`
}
SearchResult is one normalized search hit.
type SearchTool ¶
type SearchTool struct {
// contains filtered or unexported fields
}
SearchTool is the LLM-facing adapter for a Searcher. Construct with NewSearchTool — there is no nil-default fallback because web search inherently requires an upstream API.
func NewSearchTool ¶
func NewSearchTool(searcher Searcher) (*SearchTool, error)
NewSearchTool requires a searcher so the provider, credential, and quota are the caller's explicit choice rather than an implicit dependency on whichever backend happened to be compiled in.
func (SearchTool) Call ¶
func (r SearchTool) Call(ctx context.Context, invocation toolcontract.Invocation) (chat.ToolOutput, error)
func (SearchTool) ConcurrencyPolicy ¶ added in v0.19.0
func (SearchTool) ConcurrencyPolicy() func(toolcontract.Invocation) (string, bool)
ConcurrencyPolicy declares independent calls to the concurrent-safe backend.
func (SearchTool) Definition ¶
func (r SearchTool) Definition() chat.ToolDefinition
func (SearchTool) Unwrap ¶ added in v0.19.0
func (r SearchTool) Unwrap() toolcontract.Tool
Unwrap exposes the typed input contract through tool decorators.
type Searcher ¶
type Searcher interface {
// Search performs one request without mutating or retaining it and transfers
// ownership of a normalized response to the caller. Implementations must
// honor ctx, preserve provider error causes, and reject unsupported explicit
// fields instead of silently ignoring them.
Search(ctx context.Context, request *SearchRequest) (*SearchResponse, error)
}
Searcher is the provider boundary behind the model-facing search tool. It receives only the normalized provider-neutral contract; authentication, endpoint selection, and provider defaults are frozen in the implementation. Implementations must support concurrent calls, including calls through other tools sharing the same backend; search tools advertise parallel use.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package brave wires Brave's Web Search API into web.Searcher.
|
Package brave wires Brave's Web Search API into web.Searcher. |
|
Package exa integrates Exa's Search and Contents APIs with the provider-neutral web contracts.
|
Package exa integrates Exa's Search and Contents APIs with the provider-neutral web contracts. |
|
Package firecrawl integrates Firecrawl's Search and Scrape APIs with the provider-neutral web contracts.
|
Package firecrawl integrates Firecrawl's Search and Scrape APIs with the provider-neutral web contracts. |
|
Package jina integrates Jina Search and Jina Reader with the provider-neutral web contracts.
|
Package jina integrates Jina Search and Jina Reader with the provider-neutral web contracts. |
|
Package perplexity wires Perplexity's Search API into web.Searcher.
|
Package perplexity wires Perplexity's Search API into web.Searcher. |
|
Package serper wires Serper's Google Search API into web.Searcher.
|
Package serper wires Serper's Google Search API into web.Searcher. |
|
Package tavily integrates Tavily Search and Extract with the provider-neutral web contracts.
|
Package tavily integrates Tavily Search and Extract with the provider-neutral web contracts. |