Documentation
¶
Overview ¶
Package pipeline provides the data collection and output pipeline.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func GetOutputLib ¶ added in v1.4.0
func GetOutputLib() []string
GetOutputLib returns a sorted list of all registered output backend names.
func RefreshOutput ¶
func RefreshOutput()
RefreshOutput refreshes the state of the configured output backend via the registry.
func RegisterOutput ¶ added in v1.4.0
func RegisterOutput(name string, fn func(*collector.Collector) result.VoidResult)
RegisterOutput registers an output backend at the pipeline level.
Types ¶
type Pipeline ¶
type Pipeline interface {
Start()
Stop()
CollectData(data.DataCell) result.VoidResult
CollectFile(data.FileCell) result.VoidResult
}
Pipeline collects spider results and writes them to the configured output.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package collector implements result collection and output.
|
Package collector implements result collection and output. |
|
data
Package data provides storage structure definitions for data and file cells.
|
Package data provides storage structure definitions for data and file cells. |
Click to show internal directories.
Click to hide internal directories.