Documentation
¶
Overview ¶
Package automultilinedetection contains auto multiline detection and aggregation logic.
Package automultilinedetection contains auto multiline detection and aggregation logic.
Package automultilinedetection contains auto multiline detection and aggregation logic.
Package automultilinedetection contains auto multiline detection and aggregation logic.
Package automultilinedetection contains auto multiline detection and aggregation logic.
Package automultilinedetection contains auto multiline detection and aggregation logic.
Package automultilinedetection contains auto multiline detection and aggregation logic.
Package automultilinedetection contains auto multiline detection and aggregation logic.
Package automultilinedetection contains auto multiline detection and aggregation logic.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Aggregator ¶
type Aggregator struct {
// contains filtered or unexported fields
}
Aggregator aggregates multiline logs with a given label.
func NewAggregator ¶
func NewAggregator(outputFn func(m *message.Message), maxContentSize int, tagTruncatedLogs bool, tagMultiLineLogs bool, tailerInfo *status.InfoRegistry) *Aggregator
NewAggregator creates a new aggregator.
func (*Aggregator) Aggregate ¶
func (a *Aggregator) Aggregate(msg *message.Message, label Label)
Aggregate aggregates a multiline log using a label.
func (*Aggregator) IsEmpty ¶
func (a *Aggregator) IsEmpty() bool
IsEmpty returns true if the bucket is empty.
type DiagnosticRow ¶
type DiagnosticRow struct {
TokenString string
LabelString string
Count int64
LastIndex int64
// contains filtered or unexported fields
}
DiagnosticRow is a struct that represents a diagnostic view of a row in the PatternTable.
type Heuristic ¶
type Heuristic interface {
// ProcessAndContinue processes a log message and annotates the context with a label. It returns false if the message should be done processing.
// Heuristic implementations may mutate the message context but must do so synchronously.
ProcessAndContinue(*messageContext) bool
}
Heuristic is an interface representing a strategy to label log messages.
type IncrementalJSONValidator ¶
type IncrementalJSONValidator struct {
// contains filtered or unexported fields
}
IncrementalJSONValidator is a JSON validator that processes JSON messages incrementally.
func NewIncrementalJSONValidator ¶
func NewIncrementalJSONValidator() *IncrementalJSONValidator
NewIncrementalJSONValidator creates a new IncrementalJSONValidator.
func (*IncrementalJSONValidator) Reset ¶
func (d *IncrementalJSONValidator) Reset()
Reset resets the IncrementalJSONValidator.
func (*IncrementalJSONValidator) Write ¶
func (d *IncrementalJSONValidator) Write(s []byte) JSONState
Write writes a byte slice to the IncrementalJSONValidator.
type JSONAggregator ¶
type JSONAggregator struct {
// contains filtered or unexported fields
}
JSONAggregator aggregates pretty printed JSON messages into single line JSON messages.
func NewJSONAggregator ¶
func NewJSONAggregator(tagCompleteJSON bool, maxContentSize int) *JSONAggregator
NewJSONAggregator creates a new JSONAggregator.
func (*JSONAggregator) Flush ¶
func (r *JSONAggregator) Flush() []*message.Message
Flush flushes the buffer and returns the messages.
func (*JSONAggregator) IsEmpty ¶
func (r *JSONAggregator) IsEmpty() bool
IsEmpty returns true if the buffer is empty.
func (*JSONAggregator) Process ¶
func (r *JSONAggregator) Process(msg *message.Message) []*message.Message
Process processes a message. If the message is a complete JSON message, it will be aggregated into a single line JSON message. If the message is an incomplete JSON message, it will be added to the buffer and processed later. If the message is not a JSON message, it will be returned as is, and any buffered messages will be flushed (unmodified).
type JSONDetector ¶
type JSONDetector struct{}
JSONDetector is a heuristic to detect JSON messages.
func NewJSONDetector ¶
func NewJSONDetector() *JSONDetector
NewJSONDetector returns a new JSON detection heuristic.
func (*JSONDetector) ProcessAndContinue ¶
func (j *JSONDetector) ProcessAndContinue(context *messageContext) bool
ProcessAndContinue checks if a message is a JSON message. This implements the Heuristic interface - so we should stop processing if we detect a JSON message by returning false.
type Labeler ¶
type Labeler struct {
// contains filtered or unexported fields
}
Labeler labels log messages based on a set of heuristics. Each Heuristic operates on the output of the previous heuristic - mutating the message context. A label is chosen when a herusitc signals the labeler to stop or when all Heuristics have been processed.
func NewLabeler ¶
NewLabeler creates a new labeler with the given heuristics. lablerHeuristics are used to mutate the label of a log message. analyticsHeuristics are used to analyze the log message and labeling process for the status page and telemetry.
type MatchContext ¶
type MatchContext struct {
// contains filtered or unexported fields
}
MatchContext is the context of a match.
type PatternTable ¶
type PatternTable struct {
// contains filtered or unexported fields
}
PatternTable is a table of patterns that occur over time from a log source. The pattern table is always sorted by the frequency of the patterns. When the table becomes full, the least recently updated pattern is evicted.
func NewPatternTable ¶
func NewPatternTable(maxTableSize int, matchThreshold float64, tailerInfo *status.InfoRegistry) *PatternTable
NewPatternTable returns a new PatternTable heuristic.
func (*PatternTable) DumpTable ¶
func (p *PatternTable) DumpTable() []DiagnosticRow
DumpTable returns a slice of DiagnosticRow structs that represent the current state of the table.
func (*PatternTable) Info ¶
func (p *PatternTable) Info() []string
Info returns a breakdown of the patterns in the table.
func (*PatternTable) InfoKey ¶
func (p *PatternTable) InfoKey() string
InfoKey returns a string representing the key for the pattern table.
func (*PatternTable) ProcessAndContinue ¶
func (p *PatternTable) ProcessAndContinue(context *messageContext) bool
ProcessAndContinue adds a pattern to the table and updates its label based on it's frequency. This implements the Heuristic interface - so we should stop processing if the label was changed due to pattern detection.
type TimestampDetector ¶
type TimestampDetector struct {
// contains filtered or unexported fields
}
TimestampDetector is a heuristic to detect timestamps.
func NewTimestampDetector ¶
func NewTimestampDetector(matchThreshold float64) *TimestampDetector
NewTimestampDetector returns a new Timestamp detection heuristic.
func (*TimestampDetector) ProcessAndContinue ¶
func (t *TimestampDetector) ProcessAndContinue(context *messageContext) bool
ProcessAndContinue checks if a message is likely to be a timestamp.
type TokenGraph ¶
type TokenGraph struct {
// contains filtered or unexported fields
}
TokenGraph is a directed cyclic graph of tokens that model the relationship between any two tokens. It is used to calculate the probability of an unknown sequence of tokens being represented by the graph.
func NewTokenGraph ¶
func NewTokenGraph(minimumTokenLength int, inputData [][]tokens.Token) *TokenGraph
NewTokenGraph returns a new TokenGraph.
func (*TokenGraph) MatchProbability ¶
func (m *TokenGraph) MatchProbability(ts []tokens.Token) MatchContext
MatchProbability returns the probability of a sequence of tokens being represented by the graph.
type Tokenizer ¶
type Tokenizer struct {
// contains filtered or unexported fields
}
Tokenizer is a heuristic to compute tokens from a log message. The tokenizer is used to convert a log message (string of bytes) into a list of tokens that represents the underlying structure of the log. The string of tokens is a compact slice of bytes that can be used to compare log messages structure. A tokenizer instance is not thread safe as bufferes are reused to avoid allocations.
func NewTokenizer ¶
NewTokenizer returns a new Tokenizer detection heuristic.
func (*Tokenizer) ProcessAndContinue ¶
ProcessAndContinue enriches the message context with tokens. This implements the Heuristic interface - this heuristic does not stop processing.
type UserSample ¶
type UserSample struct {
// contains filtered or unexported fields
}
UserSample represents a user-defined sample for auto multi-line detection.
type UserSamples ¶
type UserSamples struct {
// contains filtered or unexported fields
}
UserSamples is a heuristic that represents a collection of user-defined samples for auto multi-line aggreagtion.
func NewUserSamples ¶
func NewUserSamples(cfgRdr model.Reader, sourceSamples []*config.AutoMultilineSample) *UserSamples
NewUserSamples creates a new UserSamples instance.
func (*UserSamples) ProcessAndContinue ¶
func (j *UserSamples) ProcessAndContinue(context *messageContext) bool
ProcessAndContinue applies a user sample to a log message. If it matches, a label is assigned. This implements the Heuristic interface - so we should stop processing if we detect a user pattern by returning false.
Source Files
Directories