Documentation
¶
Overview ¶
Package webcache provides a Go-native HTTP fetch + SQLite cache for web documentation. It replaces the synapses-scout Python sidecar for the core use cases that have no built-in equivalent in AI agent runtimes:
- Fetching and caching Go package documentation at a pinned version (invalidated only when go.mod changes, never on a timer)
- Caching arbitrary URLs for cross-session reuse (24h TTL)
All network fetches are performed by the caller's goroutine or the background indexer — never on the MCP hot path.
Index ¶
- Constants
- func IndexProjectImports(ctx context.Context, projectPath string, g *graph.Graph, cache *Cache, ...)
- func IsStdlib(importPath string) bool
- func PackageCacheKey(importPath, version string) string
- func ParseGoMod(projectPath string) (map[string]string, error)
- func StripHTML(s string) string
- type Cache
- func (c *Cache) Fetch(ctx context.Context, rawURL string, ttlHours int) (string, bool, error)
- func (c *Cache) FetchFresh(ctx context.Context, url string) (string, error)
- func (c *Cache) FetchPackageDocs(ctx context.Context, importPath, version string) (string, bool, error)
- func (c *Cache) FetchPackageDocsFresh(ctx context.Context, importPath, version string) (string, error)
- func (c *Cache) InvalidatePackage(importPath, oldVersion string)
Constants ¶
const ( // PkgDocTTL is 0 — version-pinned package docs never expire on their own. // They are invalidated only when go.mod bumps a package version. PkgDocTTL = 0 // URLCacheTTL is 24 hours for arbitrary URL content. URLCacheTTL = 24 )
Variables ¶
This section is empty.
Functions ¶
func IndexProjectImports ¶
func IndexProjectImports(ctx context.Context, projectPath string, g *graph.Graph, cache *Cache, maxPackages int)
IndexProjectImports walks the project's IMPORTS edges to find external Go packages, then fetches and caches their pkg.go.dev documentation in order of usage (most-called packages first).
It must be called as a goroutine — it blocks until all packages are indexed or ctx is cancelled:
go webcache.IndexProjectImports(ctx, projectPath, g, cache, 20)
It skips packages already in the cache (TTL=0 entries are never stale). Fetches are paced at 1 per 2 seconds to be polite to pkg.go.dev.
func IsStdlib ¶
IsStdlib returns true if importPath is a Go standard library package. Stdlib packages never contain a dot in the first path segment.
func PackageCacheKey ¶
PackageCacheKey returns the web_cache URL key for a Go package at a version. Example: "pkg:github.com/mark3labs/mcp-go@v0.32.0"
func ParseGoMod ¶
ParseGoMod parses the go.mod file at projectPath and returns a map of module import path → version for all direct and indirect dependencies.
func StripHTML ¶
StripHTML removes HTML tags and decodes common entities from s, returning plain text. Uses stdlib only — no external dependencies.
This is intentionally simple: it handles pkg.go.dev and most documentation pages well enough for API signature verification. It does not implement a full HTML parser.
Types ¶
type Cache ¶
type Cache struct {
// contains filtered or unexported fields
}
Cache is the Go-native web documentation cache backed by the main store DB. It is safe for concurrent use.
func (*Cache) Fetch ¶
Fetch returns content for a URL, using the cache when available. ttlHours controls expiry: 0 = never expire, >0 = expire after N hours.
Returns (content, fromCache, error).
func (*Cache) FetchFresh ¶
FetchFresh fetches url from the network without reading or writing the cache. Use this when the user has disabled web doc caching (cache_web_searches=false). Returns (content, false, error) — fromCache is always false.
func (*Cache) FetchPackageDocs ¶
func (c *Cache) FetchPackageDocs(ctx context.Context, importPath, version string) (string, bool, error)
FetchPackageDocs returns the cached docs for a Go package at the given version. If not cached (or version changed), it fetches from pkg.go.dev and caches with TTL=0 (version-pinned — valid until go.mod changes).
Returns (content, fromCache, error).
func (*Cache) FetchPackageDocsFresh ¶
func (c *Cache) FetchPackageDocsFresh(ctx context.Context, importPath, version string) (string, error)
FetchPackageDocsFresh fetches package docs without reading or writing the cache.
func (*Cache) InvalidatePackage ¶
InvalidatePackage removes cached docs for importPath at oldVersion so they will be re-fetched at the new version. Called when go.mod bumps a dependency.
Source Files