Documentation
¶
Overview ¶
Package storage of the Dataflow kit describes Store interface for read/ write operations with downloaded data and parsed results.
Storage types like Diskv, MongoDB implement methods which satisfy Store interface.
Index ¶
Constants ¶
View Source
const ( CACHE = "Cache" COOKIES = "Cookies" INTERMEDIATE = "Intermediate" )
Storage types
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Store ¶
type Store interface {
//Reads value from storage by specified key
Read(rec Record) (value []byte, err error)
//Writes specified pair key value to storage.
//expTime value sets TTL for Redis storage.
//expTime set Metadata Expires value for S3Storage
Write(rec Record) error
// IsExists checkes whether specified by key record exists
IsExists(rec Record) bool
//Is key expired ? It checks if parse results storage item is expired. Set up Expiration as "ITEM_EXPIRE_IN" environment variable.
//html pages cache stores this info in sResponse.Expires . It is not used for fetch endpoint.
Expired(rec Record) bool
//Delete deletes specified item from the store
Delete(rec Record) error
//DeleteAll erases all items from the store
DeleteAll() error
// Close storage connection
Close()
}
Store is the key interface of storage. All other structs implement methods wchich satisfy that interface.
Click to show internal directories.
Click to hide internal directories.