Documentation
¶
Index ¶
Constants ¶
const DefaultCacheTTL = 24 * time.Hour // TODO: move to a cache pkg
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CrossrefHarvester ¶ added in v0.2.12
type CrossrefHarvester struct {
Client Doer
ApiEndpoint string
ApiFilter string
ApiEmail string
Rows int
UserAgent string
MaxRetries int
AcceptableMissRatio float64 // recommended: 0.1
}
CrossrefHarvester fetches data from crossref API and by default will write it to files on disk.
func (*CrossrefHarvester) WriteDaySlice ¶ added in v0.2.12
WriteDaySlice is a helper function to atomically write crossref data for a single day to file on disk under dir. Idempotent, once the data has been captured. TODO: add compression.
func (*CrossrefHarvester) WriteSlice ¶ added in v0.2.12
WriteSlice writes a slice of data from the API into a writer.
type Doer ¶ added in v0.2.12
Doer abstracts https://pkg.go.dev/net/http#Client.Do.
type PubMedFetcher ¶
PubMedFetcher handles fetching and parsing PubMed update files list
func NewPubMedFetcher ¶
func NewPubMedFetcher(baseURL string) (*PubMedFetcher, error)
NewPubMedFetcher creates a new fetcher with default settings
func (*PubMedFetcher) FetchFiles ¶
func (pf *PubMedFetcher) FetchFiles() ([]PubMedFile, error)
FetchFiles retrieves and parses the PubMed update files.
type PubMedFile ¶
PubMedFile represents metadata for a PubMed update file, cf. https://ftp.ncbi.nlm.nih.gov/pubmed/updatefiles/.
func FilterPubmedFiles ¶
func FilterPubmedFiles(files []PubMedFile, f func(PubMedFile) bool) (result []PubMedFile)
FilterPubmedFiles returns a list of file filtered by a given filter function.
type WorksResponse ¶ added in v0.2.12
type WorksResponse struct {
Message struct {
Facets struct {
} `json:"facets"`
Items []json.RawMessage `json:"items"`
ItemsPerPage int64 `json:"items-per-page"`
NextCursor string `json:"next-cursor"` // iterate
Query struct {
SearchTerms interface{} `json:"search-terms"`
StartIndex int64 `json:"start-index"`
} `json:"query"`
TotalResults int64 `json:"total-results"` // want to estimate total results (and verify download)
} `json:"message"`
MessageType string `json:"message-type"`
MessageVersion string `json:"message-version"`
Status string `json:"status"`
}
WorksResponse, stripped of the actual messages, as we only need the status and mayby total results.
func (*WorksResponse) IsLast ¶ added in v0.2.12
func (wr *WorksResponse) IsLast() bool
IsLast returns true, if there are no more records to fetch.