Documentation
¶
Index ¶
Constants ¶
const ( // ORIGIN_QUEUE is used to add name of a source queue to message attributes of archive events. ORIGIN_QUEUE string = "origin_queue" )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Collector ¶ added in v1.0.1
type Collector interface {
// Run executes the collector's data processing logic.
Run(context.Context) error
}
Collector calls fetch method of a datasource and processes the returned event.
func NewContinuousCollector ¶ added in v1.1.1
NewContinuousCollector returns a new collector for continuous processing with given datasource.
func NewScheduledCollector ¶ added in v1.0.1
func NewScheduledCollector(queue string, datasource DataSource, conf config.Config, logger log.Logger) Collector
NewScheduledCollector returns a new scheduled collector for given config.
type ContinuousCollector ¶ added in v1.1.1
type ContinuousCollector struct {
// contains filtered or unexported fields
}
ContinuousCollector is used as a daemon to permanently collect data from a source. It mainly cares about observing OS signals to handle graceful shutdowns. The actual logic to process data is encapsulated in datasource member.
type DataSource ¶
type DataSource interface {
// Fetch will retrieve new data and return it as an event from central event lib.
// For more details about events see https://github.com/tommzn/hdb-events-go
Fetch() (proto.Message, error)
}
DataSource retrieves data from a specific source.
type EventHandlerS3 ¶ added in v1.0.4
type EventHandlerS3 struct {
// contains filtered or unexported fields
}
EventHandlerS3 is used to process an S3 event sent from CloudWatch to a Lambda function on AWS.
type Publisher ¶ added in v1.2.0
type Publisher interface {
// Send will publish passed message to given queues.
Send(message proto.Message) error
}
Publisher is used to send messages to one or multiple queues.
type S3EventHandler ¶ added in v1.0.4
type S3EventHandler interface {
// Handle processes passed S3 event.
Handle(ctx context.Context, event events.S3Event) error
}
S3EventHandler is used to process events published for S3 actions.
func NewS3EventHandler ¶ added in v1.0.4
func NewS3EventHandler(queue string, processor S3EventProcessor, conf goconfig.Config, logger log.Logger) S3EventHandler
NewS3EventHandler returns a new handler to process S3 events sent from CloudWatch.
type S3EventProcessor ¶ added in v1.0.4
type S3EventProcessor interface {
// Process is called to process given event for an S3 object.
// If download option is enabled via config it will pass S3 object content as well.
ProcessEvent(entity events.S3Entity, content []byte) (proto.Message, error)
}
S3EventProcessor processes an event for a specific S3 object.
type ScheduledCollector ¶ added in v1.0.1
type ScheduledCollector struct {
// contains filtered or unexported fields
}
A ScheduledCollector calls fetch method of a datasource one time and publishes returned event to a given AWS SQS queue. It contains a logger to provide insights to all processing steps and it requires a datasource and a publisher for AWS SQS.
type SqsEventProcessor ¶ added in v1.0.3
type SqsEventProcessor interface {
// Handle processes given SQS events.
Handle(ctx context.Context, sqsEvent events.SQSEvent) error
}
SqsEventProcessor is used to handle events forwarded from AWS SQS to a Lambda function.
type SqsPublisher ¶ added in v1.2.0
type SqsPublisher struct {
// contains filtered or unexported fields
}
SqsPublisher is used to publish messages on AWS SQS.