web

package
v0.58.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 26, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Overview

Package web defines provider-neutral web search and page-fetching tools.

Searcher and Fetcher are independent capabilities because not every provider supports both. A provider client may implement either or both; each provider owns one package and one transport client. NewSearchTool and NewFetchTool adapt those capabilities to the core tool contract. Requests and responses reject invalid UTF-8. Published times use Core's lossless timestamp admission before a search result can be returned by a tool. Provider clients validate their own result projections: malformed records and missing required result payloads fail, while an explicit empty search result remains valid. Requested fetched content must be present even when it is empty.

Index

Constants

This section is empty.

Variables

View Source
var (
	ErrMissingFetchRequest = errors.New("web: fetch request must not be nil")

	ErrEmptyURL      = errors.New("web: fetch URL must not be empty")
	ErrInvalidURL    = errors.New("web: fetch URL must be an absolute http(s) URL")
	ErrInvalidFormat = errors.New("web: content format must be markdown, html, or text")

	ErrMissingFetcher       = errors.New("web: fetcher is required")
	ErrMissingFetchResponse = errors.New("web: fetch response must not be nil")
	ErrInvalidFetchResponse = errors.New("web: fetch response is invalid")
	ErrUnsupportedFormat    = errors.New("web: content format is not supported by provider")
)
View Source
var (
	ErrMissingSearchRequest = errors.New("web: search request must not be nil")

	ErrEmptyQuery   = errors.New("web: search query must not be empty")
	ErrInvalidQuery = errors.New("web: search query must be valid UTF-8")

	ErrDomainsBothSides  = errors.New("web: allowed_domains and blocked_domains are mutually exclusive")
	ErrInvalidMaxResults = errors.New("web: max_results must be between 1 and 20 when set")
	ErrTooManyDomains    = errors.New("web: at most 20 allowed_domains or blocked_domains may be set")
	ErrInvalidDomain     = errors.New("web: domain filter must be a bare hostname without scheme, port, path, query, or fragment")
	ErrInvalidRecency    = errors.New("web: recency must be hour, day, week, month, or year")

	ErrMissingSearcher       = errors.New("web: searcher is required")
	ErrMissingSearchResponse = errors.New("web: search response must not be nil")
	ErrInvalidSearchResponse = errors.New("web: search response is invalid")
	ErrUnsupportedFilter     = errors.New("web: search filter is not supported by provider")
)

Functions

This section is empty.

Types

type ContentFormat

type ContentFormat string
const (
	// FormatMarkdown is the default format.
	FormatMarkdown ContentFormat = "markdown"
	FormatHTML     ContentFormat = "html"
	FormatText     ContentFormat = "text"
)

func (ContentFormat) JSONSchemaEnum added in v0.54.0

func (ContentFormat) JSONSchemaEnum() []string

func (ContentFormat) Normalize added in v0.24.0

func (c ContentFormat) Normalize() (ContentFormat, error)

func (ContentFormat) Validate

func (c ContentFormat) Validate() error

type FetchRequest

type FetchRequest struct {
	URL string `json:"url" jsonschema:"minLength=1" jsonschema_description:"Absolute http(s) URL of the page to fetch."`

	Format ContentFormat `` /* 131-byte string literal not displayed */
}

func (*FetchRequest) Prepare

func (f *FetchRequest) Prepare() (*FetchRequest, error)

Prepare does not mutate f; it returns an independently owned request.

func (*FetchRequest) Validate

func (f *FetchRequest) Validate() error

type FetchResponse

type FetchResponse struct {
	Content string        `json:"content"`
	Format  ContentFormat `json:"format"`
}

func (*FetchResponse) ValidateFor added in v0.58.0

func (f *FetchResponse) ValidateFor(request *FetchRequest) error

type FetchTool

type FetchTool struct {
	// contains filtered or unexported fields
}

func NewFetchTool

func NewFetchTool(fetcher Fetcher) (*FetchTool, error)

func (FetchTool) Call

func (r FetchTool) Call(ctx context.Context, invocation toolcontract.Invocation) (toolcontract.Result, error)

func (FetchTool) ConcurrencyPolicy added in v0.19.0

func (FetchTool) ConcurrencyPolicy() func(toolcontract.Invocation) (string, bool)

func (FetchTool) Definition

func (r FetchTool) Definition() chat.ToolDefinition

func (FetchTool) Unwrap added in v0.19.0

func (r FetchTool) Unwrap() toolcontract.Tool

type Fetcher

type Fetcher interface {
	// Fetch retrieves exactly request.URL in the requested format without
	// mutating or retaining request, and transfers ownership of the response to
	// the caller. Implementations must honor ctx and preserve network error
	// causes.
	Fetch(ctx context.Context, request *FetchRequest) (*FetchResponse, error)
}

Fetcher is the provider boundary behind the model-facing page fetch tool. Network authority, authentication, redirects, and provider defaults are fixed by the implementation rather than supplied by model arguments. Implementations must be safe for concurrent use because fetch tools advertise parallel calls.

type Recency

type Recency string

Recency expresses a relative freshness window shared across backends.

const (
	RecencyHour  Recency = "hour"
	RecencyDay   Recency = "day"
	RecencyWeek  Recency = "week"
	RecencyMonth Recency = "month"
	RecencyYear  Recency = "year"
)

func (Recency) JSONSchemaEnum added in v0.54.0

func (Recency) JSONSchemaEnum() []string

func (Recency) Validate

func (r Recency) Validate() error

type SearchRequest

type SearchRequest struct {
	Query string `` /* 155-byte string literal not displayed */

	MaxResults int `` /* 173-byte string literal not displayed */

	AllowedDomains []string `` /* 178-byte string literal not displayed */

	BlockedDomains []string `` /* 167-byte string literal not displayed */

	Recency Recency `json:"recency,omitempty" jsonschema_description:"Optional time window: hour, day, week, month, or year."`
}

func (SearchRequest) JSONSchemaConstraints added in v0.54.0

func (SearchRequest) JSONSchemaConstraints() jsonschema.Constraints

func (*SearchRequest) Prepare

func (s *SearchRequest) Prepare() (*SearchRequest, error)

Prepare returns an independently owned, normalized request after validating it. The receiver is never mutated. It normalizes each domain list once, keeping the result Validate would otherwise compute and discard.

func (*SearchRequest) QueryWithSiteOperators

func (s *SearchRequest) QueryWithSiteOperators() string

QueryWithSiteOperators returns Query with Google-style site:/-site: operators for the request's domain filters. Providers without native domain fields use this projection. Allowed domains form an OR group; every blocked domain is excluded. Empty domain entries are ignored.

func (*SearchRequest) Validate

func (s *SearchRequest) Validate() error

type SearchResponse

type SearchResponse struct {
	// Query is the caller's prepared query, never a provider's echo of it:
	// providers may receive rewritten text such as appended site: operators
	// for domain filters.
	Query   string          `json:"query"`
	Results []*SearchResult `json:"results"`
}

func (*SearchResponse) ValidateFor added in v0.57.0

func (s *SearchResponse) ValidateFor(request *SearchRequest) error

type SearchResult

type SearchResult struct {
	Title         string    `json:"title"`
	URL           string    `json:"url"`
	Snippet       string    `json:"snippet"`
	FaviconURL    string    `json:"favicon_url,omitempty"`
	PublishedTime time.Time `json:"published_time,omitzero"`
	Source        string    `json:"source,omitempty"`
}

func (*SearchResult) Validate added in v0.44.0

func (s *SearchResult) Validate() error

type SearchTool

type SearchTool struct {
	// contains filtered or unexported fields
}

func NewSearchTool

func NewSearchTool(searcher Searcher) (*SearchTool, error)

func (SearchTool) Call

func (r SearchTool) Call(ctx context.Context, invocation toolcontract.Invocation) (toolcontract.Result, error)

func (SearchTool) ConcurrencyPolicy added in v0.19.0

func (SearchTool) ConcurrencyPolicy() func(toolcontract.Invocation) (string, bool)

func (SearchTool) Definition

func (r SearchTool) Definition() chat.ToolDefinition

func (SearchTool) Unwrap added in v0.19.0

func (r SearchTool) Unwrap() toolcontract.Tool

type Searcher

type Searcher interface {
	// Search performs one request without mutating or retaining it and
	// transfers ownership of the response to the caller. Implementations must
	// honor ctx, preserve provider error causes, and reject unsupported
	// explicit fields instead of ignoring them.
	Search(ctx context.Context, request *SearchRequest) (*SearchResponse, error)
}

Searcher is the provider boundary behind the model-facing search tool. Authentication, endpoint selection, and provider defaults are fixed by the implementation. Implementations must be safe for concurrent use because search tools advertise parallel calls.

Directories

Path Synopsis
Package brave wires Brave's Web Search API into web.Searcher.
Package brave wires Brave's Web Search API into web.Searcher.
Package exa integrates Exa's Search and Contents APIs with the provider-neutral web contracts.
Package exa integrates Exa's Search and Contents APIs with the provider-neutral web contracts.
Package firecrawl integrates Firecrawl's Search and Scrape APIs with the provider-neutral web contracts.
Package firecrawl integrates Firecrawl's Search and Scrape APIs with the provider-neutral web contracts.
internal
providerhttp
Package providerhttp owns the JSON exchange shared by the web providers, so transport, status, and decoding failures have one definition.
Package providerhttp owns the JSON exchange shared by the web providers, so transport, status, and decoding failures have one definition.
Package jina integrates Jina Search and Jina Reader with the provider-neutral web contracts.
Package jina integrates Jina Search and Jina Reader with the provider-neutral web contracts.
Package perplexity wires Perplexity's Search API into web.Searcher.
Package perplexity wires Perplexity's Search API into web.Searcher.
Package serper wires Serper's Google Search API into web.Searcher.
Package serper wires Serper's Google Search API into web.Searcher.
Package tavily integrates Tavily Search and Extract with the provider-neutral web contracts.
Package tavily integrates Tavily Search and Extract with the provider-neutral web contracts.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL