internal

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 31, 2022 License: GPL-3.0 Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source
const (
	GeneratorName    string = "feedloggr"
	GeneratorVersion string = "v0.3.0"
	GeneratorSource  string = "https://github.com/lmas/feedloggr"
)

Basic info about this generator

Variables

View Source
var TmplFuncs = html.FuncMap{
	"shortdate": func(t time.Time) string {
		return t.Format("2006-01-02")
	},
	"prevday": func(t time.Time) time.Time {
		return t.AddDate(0, 0, -1)
	},
	"nextday": func(t time.Time) time.Time {
		return t.AddDate(0, 0, 1)
	},
}

TmplFuncs contains some custom funcs for being used in the templates

Functions

func LoadTemplates

func LoadTemplates() (*html.Template, error)

LoadTemplates returns a html.Template struct, loaded with the parsed templates and ready for use

func Symlink(src, dst string) error

Symlink tries to make a new symlink dst pointing to file src

func WriteTemplate

func WriteTemplate(file, name string, tmpls *html.Template, vars interface{}) error

WriteTemplate executes a loaded template and writes it's output to a file

Types

type Conf

type Conf struct {
	Settings Settings // General settings
	Feeds    []Feed   // Per feed settings
}

Conf contains ALL settings for a Generator

func ExampleConf

func ExampleConf() Conf

ExampleConf returns a working, example Conf

func LoadConf

func LoadConf(path string) (Conf, error)

LoadConf tries to load a Conf from path

func (Conf) String

func (conf Conf) String() string

String returns a yaml formatted string of Conf

type Feed

type Feed struct {
	Title  string // Custom title
	Url    string // URL to feed
	Parser Parser `yaml:",omitempty"` // Custom parsing rule
}

Feed represents a single news feed and how to download and parse it

func (Feed) Source

func (f Feed) Source() string

Source returns the "correct" URL host used as the source for the feed

type FilterStats

type FilterStats struct {
	Capacity  uint    // Total capacity for the internal series of bloom filters
	Hashes    uint    // Number of hash functions for each internal filter
	FillRatio float64 // Average ratio of set bits across all internal filters
}

FilterStats contains basic info about the internal Bloom Filter

func (FilterStats) String

func (fs FilterStats) String() string

String returns a pretty string of FilterStats

type Generator

type Generator struct {
	// contains filtered or unexported fields
}

Generator contains the runtime state for downloading/parsing/filtering and finally writing news feeds

func New

func New(conf Conf) (*Generator, error)

New creates a new Generator instance, based on conf

func (*Generator) Download

func (g *Generator) Download(feed Feed) (io.ReadCloser, error)

Download simply tries to download the body of a feed, using a custom http.Transport

func (*Generator) FilterStats

func (g *Generator) FilterStats() FilterStats

FilterStats returns a FilterStats struct with the current state of the internal bloom filter.

func (*Generator) NewItems

func (g *Generator) NewItems(f Feed) ([]Item, error)

NewItems is a shortcut to download/parse/filter a news feed

func (*Generator) ParseFeed

func (g *Generator) ParseFeed(body io.ReadCloser) ([]Item, error)

ParseFeed tries to parse a normal atom/rss/json feed and return it's items

func (*Generator) ParsePage

func (g *Generator) ParsePage(body io.ReadCloser, feed Feed) ([]Item, error)

ParsePage sets up a bunch of regexp rules and urls and tries to parse a raw page body for custom items

func (*Generator) WriteFilter

func (g *Generator) WriteFilter(dir string) error

WriteFilter writes the internal bloom filter to dir.

type Item

type Item struct {
	Title   string
	Url     string
	Content string // optional
}

Item represents a single news item in a feed

type Parser

type Parser struct {
	Rule string // Regexp rule for gragging items' title/url fields in a feed body
	Host string // Optional prefix the item urls with this host
}

Parser contains a custom regexp rule for parsing non-atom/rss/json feeds

type Settings

type Settings struct {
	Output   string // Dir to output the feeds and internal bloom filter
	MaxItems int    // Max amount of items per feed and per day
	Throttle int    // Time in seconds to sleep after a feed has been downloaded
	Timeout  int    // Max time in seconds when trying to download a feed
	Verbose  bool   // Verbose, debug output
}

Settings contains the general Generator settings

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL