Help us shape the future of Go! Take the 2026 Go Developer Survey

webfetch

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 7, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func New

func New(client *Client) []gollem.Tool

New returns the webfetch-backed agent tools. It returns nil (the tool is not registered) when client is nil or has no LLM client: the LLM screen is the only injection defense in this codebase — there is no HITL fallback — so webfetch fails closed rather than serving unscreened content.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client fetches web content over HTTP and screens it through an LLM-based pipeline (Markdown extraction + indirect-prompt-injection detection).

func NewClient

func NewClient(cfg ClientConfig) *Client

NewClient builds a Client whose HTTP transport rejects connections to non-public IP ranges (SSRF guard) unless cfg.AllowPrivateIP is set.

type ClientConfig

type ClientConfig struct {
	// Timeout bounds the whole HTTP request (dial + TLS + headers + body).
	Timeout time.Duration
	// MaxBytes caps how much of the response body is read; the remainder is
	// dropped and the result is marked truncated.
	MaxBytes int64
	// MaxPDFBytes caps how much of a PDF response body is read. It is separate
	// from MaxBytes because the two need sizes an order of magnitude apart: a
	// cap large enough for a real PDF would let one HTML page swamp the model's
	// context. A PDF that hits this cap is rejected rather than truncated (a
	// truncated PDF is a broken file that still passes the %PDF- check), so the
	// caller never sees a partial document. It is an int rather than an int64
	// because the CLI flag is an int and gollem.WithMaxPDFSize takes one.
	MaxPDFBytes int
	// UserAgent is sent on every request.
	UserAgent string
	// LLM screens fetched bodies for indirect prompt injection and reformats
	// them to Markdown. Required: New returns no tools when it is nil.
	LLM gollem.LLMClient
	// AllowPrivateIP disables the SSRF guard. It exists only as a test seam so
	// the fetch path can be exercised against loopback httptest servers;
	// production callers always leave it false.
	AllowPrivateIP bool
}

ClientConfig carries every parameter needed to construct a Client. All values are supplied by the caller (CLI config) — the package embeds no defaults so timeout / size limits stay configurable from one place.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL