feeds

package module
v0.0.0-...-939e56e Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 24, 2026 License: MIT Imports: 16 Imported by: 0

README

Logo


Project license

Pull Requests welcome code with love by joshuar

tests

Table of Contents

About

go-syndication is Go package for dealing with various feed syndication formats. It supports:

  • RSS (1.x and 2.x)
  • Atom
  • JSONFeed
  • OPML
  • Various RSS/Atom extensions such as media, Dublin Core, iTunes, and GooglePlay, with more to come…

The package can read and write all formats. It includes built-in validation of elements.

Built With

Getting Started

Installation

go get github.com/immanent-tech/go-syndication

Usage

Encoding and Decoding

You can decode a io.Reader containing feed data into a format using func Decode[T any](namespace string, rd io.Reader) (T, error). T would be one of *atom.Feed, *rss.RSS or *jsonfeed.Feed:

// Decode RSS feed data.
// use bytes.NewReader(data) for a []byte
rss, err := Decode[*rss.RSS]("", data)

Likewise, func Encode[T any](feed T) ([]byte, error) can be used to encode feed data:

data, err := Encode[*rss.RSS](rss)

Validation

By default, encoding/decoding performs no validation. As long as the XML data is well-formed and can be read by the Go XML parser, your feed data should be encoded/decoded. This means the package will handle invalid feed data (feeds that don't adhere to the RSS/Atom specs).

If you want to check that the feed data is valid, you can use the validation sub-package:

// Decode atom feed data.
// use bytes.NewReader(data) for a []byte
rss, err := Decode[*rss.RSS]("", data)

// Validate the atom feed.
err := validation.ValidateStruct(rss)

Generic Feed/Item Types

In addition to providing the source-specific atom.Feed, rss.RSS and jsonfeed.Feed types and their item counterparts, this library provides generic feeds.Feed and feeds.Item types, that wrap the source types with common methods for accessing their fields.

Use func NewDecoder[T any](data io.Reader) (*Feed, error) to read data into the generic object:

// data is a []byte containing an atom feed.
feed, err = feeds.NewDecoder[*rss.RSS](bytes.NewReader(data))

Feed exposes the original data as the FeedSource, which can be converted back to the original source format with a type conversion:

atom, ok := feed.FeedSource.(*atom.Feed)

This gives you the best of both worlds; a generic container with common methods for canonical fields across all formats, with access to the original source to manipulate the format directly as needed.

Command Line Interface (CLI)

A basic CLI can be found in cmd/ that can be used for basic reading/writing of feeds using the library.

To fetch and display feed data from a URL:

go run github.com/immanent-tech/go-syndication/cmd@latest fetch http://my.site/feed

To read a file containing feed data:

go run github.com/immanent-tech/go-syndication/cmd@latest parse /path/to/my/feed.xml

The commands will auto-detect a supported feed format.

Design

OpenAPI for Models

go-syndication uses OpenAPI schemas (through oapi-codegen) to define the custom types for each syndication format and all extensions. This provides a way to have consistent, reusable types across the package.

Validation Built In

go-syndication attempts to build validation into all types using go-playground/validator. Wherever possible, types will be annotated with struct tags that then allow the validation to work.

The library aims to pass all the must test cases for Atom/RSS from feedvalidator, as well as select tests for supported extensions. You can view test results with the standard go test:

go test -v ./...

Dynamic Namespace Support

The formats in go-syndication provide dynamic namespace support. This means you can use extensions not defined in this package on top of it and get correct marshaling/unmarshaling behavior.

Custom Marshal/Unmarshal

go-syndication provides a custom Encode and Decode methods for marshaling/unmarshaling of formats (see Usage). While you can directly marshal/unmarshal, you'll lose some features (like dynamic namespaces). It's therefore recommended to always use the Encode/Decode methods in this package.

Development

Setup

  1. Clone the go-syndication repo.
  2. Run ./setup-feedvalidator-submodule.sh to correctly clone and filter the feedvalidator submobule to testcases directory only.

Roadmap

See the open issues for a list of proposed features (and known issues).

Support

Reach out to the maintainer at one of the following places:

Project Assistance

If you want to say thank you or/and support active development of go-syndication:

  • Add a GitHub Star to the project.
  • Tweet about the go-syndication.
  • Write interesting articles about the project on Dev.to, Medium or your personal blog.

Together, we can make go-syndication better!

Contributing

First off, thanks for taking the time to contribute! Contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make will benefit everybody else and are greatly appreciated.

Please read our contribution guidelines, and thank you for being involved!

Authors & Contributors

The original setup of this repository is by joshuar.

For a full list of all authors and contributors, see the contributors page.

Security

go-syndication follows good practices of security, but 100% security cannot be assured. go-syndication is provided "as is" without any warranty. Use at your own risk.

For more information and to report security issues, please refer to our security documentation.

License

This project is licensed under the MIT license.

See LICENSE for more information.

Documentation

Overview

Package feeds provides primitives to interact with the openapi HTTP API.

Code generated by github.com/oapi-codegen/oapi-codegen/v2 version v2.8.0 DO NOT EDIT.

Package types contains methods and objects that are shared across different Feed schemas/specifications.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// MimeTypesIndeterminate contains mimetypes that can be used for either RSS/Atom feeds and don't give any clues to
	// the actual type.
	MimeTypesIndeterminate = []string{"application/xml", "text/xml"}
	// MimeTypesFeed is the concatenation of all feed mime types.
	MimeTypesFeed = slices.Concat(atom.MimeTypes, rss.MimeTypes, jsonfeed.MimeTypes, MimeTypesIndeterminate)
	// MimeTypesHTML contains canonical/standard mimetypes for HTML.
	MimeTypesHTML = []string{"text/html", "application/xhtml+xml"}
	// MimeTypesImage contains canonical/standard/common mimetypes for images.
	MimeTypesImage = []string{"image/avif", "image/gif", "image/jpeg", "image/png", "image/svg+xml", "image/webp"}
)
View Source 
var (
	// ErrParseBytes indicates an error occurred trying to parse a byte array as a feed.
	ErrParseBytes = errors.New("unable to parse bytes as feed")
)
View Source 
var ErrUnmarshal = errors.New("unmarshaling object failed")

ErrUnmarshal indicates an error occurred trying to unmarshal data into a given feed object.

Functions

func Decode 

func Decode[T any](namespace string, rd io.Reader) (T, error)

Decode will decode the byte array into the given type T, and assign values without a namespace with the given namespace.

func Encode 

func Encode[T any](feed T) ([]byte, error)

Encode will encode the given type T into a byte array.

Types

type Feed 

type Feed struct {
	types.FeedSource `json:"source"`

	SourceType SourceType `json:"type"`
}

Feed represents any feed type containing a number of items.

func NewDecoder 

func NewDecoder[T any](data io.Reader) (*Feed, error)

NewDecoder will create a new Feed of the given type from the given io.Reader.

func NewFeedFromSource 

func NewFeedFromSource[T types.FeedSource](source T) *Feed

NewFeedFromSource will create a new Feed from the given source that satisfies the FeedSource interface. This can be used to create a Feed from an existing rss.RSS or atom.Feed object.

func (*Feed) GetItems 

func (f *Feed) GetItems() []Item

GetItems retrieves a slice of Item for the Feed.

func (*Feed) UnmarshalJSON 

func (f *Feed) UnmarshalJSON(v []byte) error

UnmarshalJSON handles unmarshaling of a Feed from JSON.

type Image 

type Image struct {
	// URL the URL to the image.
	URL string `json:"URL" validate:"required,url"`

	// Description is a description of the image (for example, the alt tag associated with the image).
	Description *string `json:"description,omitempty"`
}

Image represents a remote image.

type Item 

type Item struct {
	types.ItemSource `json:"source"`

	SourceType SourceType `json:"type"`
	FeedTitle  string     `json:"feed_title"`
}

Item represents a single item or entry (or article) in a feed.

func (*Item) UnmarshalJSON 

func (i *Item) UnmarshalJSON(v []byte) error

UnmarshalJSON handles unmarshaling of an Item from JSON. 

type Link struct {
	// Type is the type of address. While there are some canonical values for the type, they are not conclusive and the type can be any value, including one of the canonical types. However, a canonical type should be used whenever appropriate.
	Type LinkType `json:"type" validate:"required"`

	// Value is the link value. It can be a URI, email or any string that acts as a unique identifier (i.e., social media handle).
	Value string `json:"value" validate:"required|uri|email"`
}

Link is an 'address', usually of a person. For example, an email, website or social media handle. An address has a type and a value and/or URI associated with it.

func (Link) String 

func (a Link) String() string

type LinkType 

type LinkType string

LinkType is the type of address. While there are some canonical values for the type, they are not conclusive and the type can be any value, including one of the canonical types. However, a canonical type should be used whenever appropriate. 

const (
	LinkTypeEmail   LinkType = "email"
	LinkTypeId      LinkType = "id"
	LinkTypeWebsite LinkType = "website"
)

Defines values for LinkType.

func (LinkType) Valid 

func (e LinkType) Valid() bool

Valid indicates whether the value is a known member of the LinkType enum.

type Person 

type Person struct {
	// Bio is an optional short biography of the person.
	Bio *string `json:"bio,omitempty"`

	// Links are links associated with the person that provide more details about them.
	Links []Link `json:"links,omitempty" validate:"omitempty,dive,unique"`

	// Name is the person's name.
	Name string `json:"name" validate:"required"`
}

Person represents a person.

func (Person) String 

func (p Person) String() string

type SourceType 

type SourceType string

SourceType is the type of source the feed or object came from. This can be used with abstractions that generalize different feed types into a common format to preserve information on the original. 

const (
	SourceTypeAtom     SourceType = "Atom"
	SourceTypeHTML     SourceType = "HTML"
	SourceTypeJSONFeed SourceType = "JSONFeed"
	SourceTypeRDF      SourceType = "RDF"
	SourceTypeRSS      SourceType = "RSS"
	SourceTypeUnknown  SourceType = "Unknown"
)

Defines values for SourceType.

func DetectSourceType 

func DetectSourceType(r io.Reader) (SourceType, error)

DetectSourceType determines the feed source by extracting key signatures from the data. It can detect supported feed formats as well as HTML.

func (SourceType) Valid 

func (e SourceType) Valid() bool

Valid indicates whether the value is a known member of the SourceType enum.

Source Files

View all Source files

Directories

Path Synopsis
Package atom provides primitives to interact with the openapi HTTP API.
 Package atom provides primitives to interact with the openapi HTTP API.
Package extensions provides primitives to interact with the openapi HTTP API.
 Package extensions provides primitives to interact with the openapi HTTP API.
basicgeo
Package basicgeo provides primitives to interact with the openapi HTTP API.
 Package basicgeo provides primitives to interact with the openapi HTTP API.
dc
Package dc provides primitives to interact with the openapi HTTP API.
 Package dc provides primitives to interact with the openapi HTTP API.
googleplay
Package googleplay provides primitives to interact with the openapi HTTP API.
 Package googleplay provides primitives to interact with the openapi HTTP API.
itunes
Package itunes provides primitives to interact with the openapi HTTP API.
 Package itunes provides primitives to interact with the openapi HTTP API.
media
Package media provides primitives to interact with the openapi HTTP API.
 Package media provides primitives to interact with the openapi HTTP API.
rss
Package rss provides primitives to interact with the openapi HTTP API.
 Package rss provides primitives to interact with the openapi HTTP API.
source
Package source provides primitives to interact with the openapi HTTP API.
 Package source provides primitives to interact with the openapi HTTP API.
Package jsonfeed provides primitives to interact with the openapi HTTP API.
 Package jsonfeed provides primitives to interact with the openapi HTTP API.
Package opml provides primitives to interact with the openapi HTTP API.
 Package opml provides primitives to interact with the openapi HTTP API.
Package rdf provides primitives to interact with the openapi HTTP API.
 Package rdf provides primitives to interact with the openapi HTTP API.
Package rss provides primitives to interact with the openapi HTTP API.
 Package rss provides primitives to interact with the openapi HTTP API.
Package schema contains the OpenAPI schema definitions for go-syndication.
 Package schema contains the OpenAPI schema definitions for go-syndication.
Package types provides primitives to interact with the openapi HTTP API.
 Package types provides primitives to interact with the openapi HTTP API.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.