feed

package
v0.15.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 3, 2026 License: MIT Imports: 11 Imported by: 0

Documentation

Overview

Package feed reads the estate feed, projects.json, which is canon for which sources the bot may index (spec 0034).

The feed is fetched once per discovery run. A copy that fetched and parsed is stored, and used whenever a later fetch fails, however old it is (D8, D13): a feed outage never shrinks the corpus.

Index

Constants

View Source
const DefaultURL = "https://phpboyscout.uk/projects.json"

DefaultURL is where the estate publishes the feed.

View Source
const MaxBytes = 1 << 20

MaxBytes bounds a fetched or stored feed. It is about 66 KB today.

Variables

View Source
var (
	// ErrSchema is a feed whose schema_version this package does not understand.
	ErrSchema = errors.NewSentinel("feed.schema", "feed: unsupported schema_version")

	// ErrTooLarge is a feed over MaxBytes.
	ErrTooLarge = errors.NewSentinel("feed.too_large", "feed: body over the size bound")

	// ErrNotHTTPS is a feed URL that is not HTTPS.
	ErrNotHTTPS = errors.NewSentinel("feed.not_https", "feed: the URL must be HTTPS")
)

Functions

func Fetch

func Fetch(ctx context.Context, client *http.Client, url string) ([]byte, error)

Fetch reads the feed body from url, refusing anything over MaxBytes.

Types

type Feed

type Feed struct {
	GeneratedAt time.Time
	// contains filtered or unexported fields
}

Feed is a parsed feed: the repositories it records, by path.

func Parse

func Parse(body []byte) (*Feed, error)

Parse validates and reads a feed body.

func (*Feed) Lookup

func (f *Feed) Lookup(path string) corpus.FeedFacts

Lookup reports what the feed records about path. Only path, listed, family and role are read (D7).

type Option

type Option func(*Resolver)

Option configures a Resolver.

func WithClient

func WithClient(c *http.Client) Option

WithClient replaces the hardened default client, for tests.

func WithClock

func WithClock(now func() time.Time) Option

WithClock replaces the wall clock, for tests.

func WithLogger

func WithLogger(l *slog.Logger) Option

WithLogger sets where a failed fetch and a stale copy are reported.

type Resolver

type Resolver struct {
	// contains filtered or unexported fields
}

Resolver supplies discovery with the feed for each run.

func NewResolver

func NewResolver(rawURL string, store Store, opts ...Option) (*Resolver, error)

NewResolver reads the feed from rawURL, which must be HTTPS, keeping the last good copy in store.

func (*Resolver) Feed

func (r *Resolver) Feed(ctx context.Context) corpus.Feed

Feed returns this run's feed: a fresh fetch, else the stored copy, else nil. It never fails the run (D8).

func (*Resolver) Last

func (r *Resolver) Last() Use

Last is which copy the most recent run used.

func (*Resolver) URL

func (r *Resolver) URL() string

URL is where the feed is fetched from.

type Store

type Store interface {
	SaveFeed(ctx context.Context, s Stored) error
	LoadFeed(ctx context.Context) (Stored, bool, error)
}

Store keeps the last good copy, in the database's one-row feed table (D8).

type Stored

type Stored struct {
	Body        []byte
	GeneratedAt time.Time
	FetchedAt   time.Time
}

Stored is the last feed that fetched and parsed.

type Use

type Use struct {
	Fetched     bool
	GeneratedAt time.Time
	FetchedAt   time.Time
}

Use is which feed copy the last run used, for `index sources` (D10).

func (Use) Line

func (u Use) Line() string

Line says which copy was used, in one line.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL