feedworker

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 21, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Index

Constants

View Source 
const CatalogName = "feed"

CatalogName is the VGI catalog name advertised by this worker.

View Source 
const SampleRSS = "<rss version=\"2.0\"><channel>" +
	"<title>Query Farm Sample Feed</title>" +
	"<link>https://query.farm/</link>" +
	"<description>A small RSS feed used by vgi-feed examples</description>" +
	"<language>en-us</language>" +
	"<item><title>First Post</title><link>https://query.farm/first</link>" +
	"<category>news</category><category>release</category>" +
	"<description>The first sample entry</description></item>" +
	"<item><title>Second Post</title><link>https://query.farm/second</link>" +
	"<category>news</category>" +
	"<description>The second sample entry</description></item>" +
	"</channel></rss>"

SampleRSS is a tiny, self-contained RSS 2.0 document used by the catalog examples and executable examples. It is parsed directly with NO network access, so the examples execute deterministically against an attached worker even when no feed server is reachable. It deliberately contains NO single quotes so it can be embedded verbatim inside a single-quoted SQL literal.

Variables

View Source 
var AgentTestTasks = buildAgentTestTasks()

AgentTestTasks is the VGI152/VGI920 analyst-task suite (catalog tag vgi.agent_test_tasks). Each task is a natural-language prompt an agent should be able to satisfy using this worker, paired with the reference SQL that produces the ground-truth answer. Every task operates on the inline SampleRSS document, so `vgi-lint simulate` can execute them deterministically with NO network access.

View Source 
var SchemaExampleQueries = exampleQueriesJSON(
	[2]string{
		"List the items parsed from an inline RSS 2.0 feed, ordered by position.",
		"SELECT seq, title, link FROM feed.main.feed_items('" + SampleRSS + "') ORDER BY seq;",
	},
	[2]string{
		"Count the items in an inline feed.",
		"SELECT count(*) AS items FROM feed.main.feed_items('" + SampleRSS + "');",
	},
	[2]string{
		"Read feed-level metadata: title, detected format, language, and item count.",
		"SELECT title, feed_type, language, item_count FROM feed.main.feed_info('" + SampleRSS + "');",
	},
)

SchemaExampleQueries is the main schema's vgi.example_queries value: a JSON list of {description, sql} so each example carries a human-readable description (VGI506/VGI515). Every query parses an inline RSS document, so it runs with no network access.

Functions

func NewInfoFunction 

func NewInfoFunction() vgi.TableFunction

NewInfoFunction builds the registerable table function.

func NewItemsFunction 

func NewItemsFunction() vgi.TableFunction

NewItemsFunction builds the registerable table function.

func ParseFeed 

func ParseFeed(ctx context.Context, opts FetchOptions) (*gofeed.Feed, error)

ParseFeed resolves the input (URL → fetch, raw text → parse directly) and returns the parsed feed. All error paths return a clear, wrapped error and never panic.

func Register 

func Register(w *vgi.Worker)

Register registers all feed table functions and the browsable feed_registry view on the worker.

func RegisterRegistry 

func RegisterRegistry(w *vgi.Worker)

RegisterRegistry registers the curated feed_registry view on the worker. It is a browsable, credential-free discovery relation (see the WHY note above) that satisfies VGI146 and gives agents a starting set of real feeds.

Types

type Cursor 

type Cursor[T any] struct {
	Rows   []T
	Offset int
}

Cursor is the shared streaming cursor embedded by every table-function state: the eagerly fetched rows plus the offset of the next unemitted row. Both fields are exported so gob round-trips them through the HTTP continuation token. The TYPE is exported (Cursor, not cursor) because the SDK counts a state struct's exported FIELDS at registration to verify it is gob-encodable.

type FetchOptions 

type FetchOptions struct {
	// Input is either an http(s):// URL to fetch, or a raw feed document
	// (RSS/Atom XML or JSON Feed text) to parse directly.
	Input string
	// Timeout bounds the HTTP fetch. Zero uses defaultTimeout.
	Timeout time.Duration
	// MaxItems caps the number of items materialised. <= 0 means no cap (a hard
	// safety cap of hardMaxItems still applies).
	MaxItems int
}

FetchOptions carries the parameters shared by both table functions.

type Info 

type Info struct {
	Title       string
	Description string
	Link        string
	FeedType    string
	Language    string
	Updated     *time.Time
	ItemCount   int32
}

Info is feed-level metadata (one row).

func InfoFrom 

func InfoFrom(feed *gofeed.Feed) Info

InfoFrom builds feed-level metadata from a parsed feed.

type InfoFunction 

type InfoFunction struct{}

InfoFunction returns one row of feed-level metadata.

func (*InfoFunction) ArgumentSpecs 

func (f *InfoFunction) ArgumentSpecs() []vgi.ArgSpec

func (*InfoFunction) Metadata 

func (f *InfoFunction) Metadata() vgi.FunctionMetadata

func (*InfoFunction) Name 

func (f *InfoFunction) Name() string

func (*InfoFunction) NewState 

func (f *InfoFunction) NewState(params *vgi.ProcessParams) (*infoState, error)

func (*InfoFunction) OnBind 

func (f *InfoFunction) OnBind(_ *vgi.BindParams) (*vgi.BindResponse, error)

func (*InfoFunction) Process 

func (f *InfoFunction) Process(_ context.Context, _ *vgi.ProcessParams, state *infoState, out *vgirpc.OutputCollector) error

type Item 

type Item struct {
	Seq        int64
	GUID       string
	Title      string
	Link       string
	Published  *time.Time
	Updated    *time.Time
	Author     string
	Categories []string
	Summary    string
	Content    string
}

Item is a single feed entry flattened to the columns the worker exposes. All fields are exported and gob-encodable: these structs are stored directly in table-function state, which the SDK gob-encodes between NewState and Process.

Published / Updated are *time.Time so a missing or unparseable date stays nil and is emitted as a NULL TIMESTAMP rather than a zero value.

func ItemsFrom 

func ItemsFrom(feed *gofeed.Feed, maxItems int) []Item

ItemsFrom flattens a parsed feed into Items, capping the count at maxItems (<= 0 means no cap). Categories is normalised to a non-nil slice so the Arrow list column is an empty list (not NULL) when a feed item has no categories.

type ItemsFunction 

type ItemsFunction struct{}

ItemsFunction returns one row per feed item.

func (*ItemsFunction) ArgumentSpecs 

func (f *ItemsFunction) ArgumentSpecs() []vgi.ArgSpec

func (*ItemsFunction) Metadata 

func (f *ItemsFunction) Metadata() vgi.FunctionMetadata

func (*ItemsFunction) Name 

func (f *ItemsFunction) Name() string

func (*ItemsFunction) NewState 

func (f *ItemsFunction) NewState(params *vgi.ProcessParams) (*itemsState, error)

func (*ItemsFunction) OnBind 

func (f *ItemsFunction) OnBind(_ *vgi.BindParams) (*vgi.BindResponse, error)

func (*ItemsFunction) Process 

func (f *ItemsFunction) Process(_ context.Context, _ *vgi.ProcessParams, state *itemsState, out *vgirpc.OutputCollector) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.