Documentation
¶
Index ¶
- type CsvWriter
- type DbWriter
- type ElasticWriter
- func (ew *ElasticWriter) CreateDoc(index string, data []byte, doc_id string) error
- func (ew *ElasticWriter) CreateDocBulk(index string, docs map[string][]byte) error
- func (ew *ElasticWriter) CreateIndex(index string, mapping string) error
- func (ew *ElasticWriter) Finalize() error
- func (ew *ElasticWriter) Flush() error
- func (ew *ElasticWriter) Marshal(v any) ([]byte, error)
- func (ew *ElasticWriter) MarshalAppend(marshalled []byte, new_data map[string]interface{}) ([]byte, error)
- func (ew *ElasticWriter) Write(result *models.File) error
- type FinalizableWriter
- type FlushableWriter
- type Interceptor
- type JsonWriter
- type MemoryWriter
- type NoneWriter
- type StdoutWriter
- type Writer
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CsvWriter ¶
type CsvWriter struct {
FilePath string
// contains filtered or unexported fields
}
CsvWriter writes CSV files
func NewCsvWriter ¶
NewCsvWriter gets a new CsvWriter
type DbWriter ¶
type DbWriter struct {
URI string
ControlOnly bool
ReadOnly bool
// contains filtered or unexported fields
}
DbWriter is a Database writer
func NewDbWriter ¶
NewDbWriter initialises a database writer
type ElasticWriter ¶
type ElasticWriter struct {
Client *elk.Client
Index string
// contains filtered or unexported fields
}
JsonWriter is a JSON lines writer
func NewElasticWriter ¶
func NewElasticWriter(uri string, debug bool) (*ElasticWriter, error)
NewElasticWriter returns a new Elasticsearch writer. When debug is true, operational logs are emitted at Info level; otherwise they are emitted at Debug level.
func (*ElasticWriter) CreateDoc ¶
func (ew *ElasticWriter) CreateDoc(index string, data []byte, doc_id string) error
func (*ElasticWriter) CreateDocBulk ¶
func (ew *ElasticWriter) CreateDocBulk(index string, docs map[string][]byte) error
func (*ElasticWriter) CreateIndex ¶
func (ew *ElasticWriter) CreateIndex(index string, mapping string) error
func (*ElasticWriter) Finalize ¶ added in v0.1.35
func (ew *ElasticWriter) Finalize() error
Finalize renders a human-friendly end-of-run table with the writer's aggregated ingestion statistics. Safe to call after Flush.
func (*ElasticWriter) Flush ¶ added in v0.1.35
func (ew *ElasticWriter) Flush() error
Flush closes the queue and waits for all in-flight writes to complete. Must be called once, after producers have stopped invoking Write.
func (*ElasticWriter) MarshalAppend ¶
func (ew *ElasticWriter) MarshalAppend(marshalled []byte, new_data map[string]interface{}) ([]byte, error)
type FinalizableWriter ¶ added in v0.1.35
type FinalizableWriter interface {
Finalize() error
}
FinalizableWriter is optionally implemented by writers that can report a final summary after all writes are flushed. The runner calls Finalize once per writer at the end of a run, after Flush. Used (for example) by the Elastic writer to print a table of server-side statistics.
type FlushableWriter ¶ added in v0.1.35
type FlushableWriter interface {
Flush() error
}
FlushableWriter is optionally implemented by writers that buffer work asynchronously. Callers should invoke Flush after all producers have stopped, to drain pending writes before shutdown.
type Interceptor ¶ added in v0.1.25
type Interceptor struct {
// contains filtered or unexported fields
}
type JsonWriter ¶
type JsonWriter struct {
FilePath string
}
JsonWriter is a JSON lines writer
func NewJsonWriter ¶
func NewJsonWriter(destination string) (*JsonWriter, error)
NewJsonWriter return a new Json lines writer
type MemoryWriter ¶
type MemoryWriter struct {
// contains filtered or unexported fields
}
MemoryWriter is a memory-based results queue with a maximum slot count
func NewMemoryWriter ¶
func NewMemoryWriter(slots int) (*MemoryWriter, error)
NewMemoryWriter initializes a MemoryWriter with the specified number of slots
func (*MemoryWriter) GetAllResults ¶
func (s *MemoryWriter) GetAllResults() []*models.File
GetAllResults returns a copy of all current results.
func (*MemoryWriter) GetFirst ¶
func (s *MemoryWriter) GetFirst() *models.File
GetFirst retrieves the oldest result in the MemoryWriter.
func (*MemoryWriter) GetLatest ¶
func (s *MemoryWriter) GetLatest() *models.File
GetLatest retrieves the most recently added result.
type NoneWriter ¶
type NoneWriter struct {
}
NoneWriter is a None writer
func NewNoneWriter ¶
func NewNoneWriter() (*NoneWriter, error)
NewNoneWriter initialises a none writer
type StdoutWriter ¶
type StdoutWriter struct {
}
StdoutWriter is a Stdout writer
func NewStdoutWriter ¶
func NewStdoutWriter() (*StdoutWriter, error)
NewStdoutWriter initialises a stdout writer
Source Files