scraper

package

v0.2.2 Latest Latest Go to latest Published: Jan 6, 2025 License: Apache-2.0 Imports: 9 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/dntiontk/civic-code

Links

Open Source Insights

Documentation ¶

Index ¶

Variables
func After(t time.Time) func([]Document) []Document
func Before(t time.Time) func([]Document) []Document
type Document
- func GetDocuments(ctx context.Context) ([]Document, error)
type FilterFunc
type HtmlCard
type MeetingType
- func GetMeetingType(meeting string) MeetingType

Constants ¶

This section is empty.

Variables ¶

View Source

var (
	CC      = MeetingType{Code: "CC", Name: "City Council", /* contains filtered or unexported fields */}
	DHSC    = MeetingType{Code: "DHSC", Name: "Development & Heritage Standing Committee", /* contains filtered or unexported fields */}
	Special = MeetingType{Code: "Special", Name: "Special Meeting of Council", /* contains filtered or unexported fields */}
	ETP     = MeetingType{Code: "ETP", Name: "Environment, Transportation & Public Safety Standing Committee", /* contains filtered or unexported fields */}
	CSSC    = MeetingType{Code: "CSSC", Name: "Community Services Standing Committee", /* contains filtered or unexported fields */}
	Other   = MeetingType{Code: "Other", Name: "Other", /* contains filtered or unexported fields */}
)

Functions ¶

func After ¶

func After(t time.Time) func([]Document) []Document

After returns a FilterFunc for documents after a given time.Time

func Before ¶

func Before(t time.Time) func([]Document) []Document

Before returns a FilterFunc for documents before a given time.Time

Types ¶

type Document ¶

type Document struct {
	Link     string      `json:"link"`
	Name     string      `json:"name"`
	Meeting  MeetingType `json:"meeting"`
	Date     time.Time   `json:"date"`
	RawTitle string      `json:"rawTitle"`
}

Document represents the metadata associated with a given upstream document.

func GetDocuments ¶

func GetDocuments(ctx context.Context) ([]Document, error)

GetDocuments fetches and parses data from the upstream and returns a slice of Document. It currently only supports PDFs.

type FilterFunc ¶

type FilterFunc func([]Document) []Document

FilterFunc is a function type that returns a subset of the input documents.

func ByMeetingType ¶

func ByMeetingType(meeting MeetingType) FilterFunc

ByMeetingType returns a FilterFunc for documents that match a specific MeetingType

func ByStringInName ¶

func ByStringInName(str string) FilterFunc

ByStringInName returns a FilterFunc for a slice of Document that have a given string in its name

func ByYear ¶

func ByYear(year int) FilterFunc

ByYear returns a FilterFunc for documents that match a given year

type HtmlCard ¶

type HtmlCard struct {
	Title string
	Links []string
	*html.Node
}

HtmlCard is a html.Node with an extracted title and slice of links

type MeetingType ¶

type MeetingType struct {
	Code string `json:"code,omitempty"`
	Name string `json:"name,omitempty"`
	// contains filtered or unexported fields
}

MeetingType defines a meeting with associated search terms

func GetMeetingType ¶

func GetMeetingType(meeting string) MeetingType

GetMeetingType will return the matching MeetingType if the meeting argument matches the Code, Name or any of the search terms.

Source Files ¶

View all Source files

scraper.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL