Documentation
¶
Overview ¶
Package unraw implements a facility to convert raw logs from external sources into leveled logs.
This is not the same as raw logging inside the logtree, which exists to ingest logs that are either fully arbitrary or do not map cleanly to the leveled logging concept. The unraw library is instead made to parse logs from systems that also use leveled logs internally, but emit them to a serialized byte stream that then needs to be turned back into something leveled inside metropolis.
Logs converted this way are unfortunately lossy and do not come with the same guarantees as logs directly emitted via logtree. For example, there's no built-in protection against systems emiting fudged timestamps or file locations. Thus, this functionality should be used to interact with trusted systems, not fully arbitrary logs.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Converter ¶
type Converter struct {
// Parser is the user-defined parsing function for converting log lines
// into leveled logging payloads. This must be set.
Parser Parser
// MaximumLineLength is the maximum length of a single log line when
// splitting incoming writes into lines. If a line is longer than this, it
// will be truncated (and will be sent to the Parser regardless).
//
// If not set, this defaults to 1024 bytes.
MaximumLineLength int
// LeveledLogger is the logtree leveled logger into which events from the
// Parser will be sent.
LeveledLogger logging.Leveled
// contains filtered or unexported fields
}
Converter is the main entrypoint of the unraw library. It wraps a LeveledLogger in combination with a Parser to create an io.Writer that can be sent raw log data.
func (*Converter) NamedPipeReader ¶
func (e *Converter) NamedPipeReader(path string) (supervisor.Runnable, error)
NamedPipeReader returns a supervisor runnable that continously reads logs from the given path and attempts to parse them into leveled logs using this Converter.
If the given path doesn't exist, a named pipe will be created there before the function exits. This guarantee means that as long as any writing process is not started before NamedPipeReader returns ther is no need to remove/recreate the named pipe.
TODO(q3k): defer the creation of the FIFO to localstorage so this doesn't need to be taken care of in the first place.
type LeveledWriter ¶
type LeveledWriter func(*logtree.ExternalLeveledPayload)
LeveledWriter is called by a Parser for every ExternelLeveledPayload it wishes to emit into a backing LeveledLogger. If the payload is missing some fields, these will default to some sensible values - see the ExternalLeveledPayload structure definition for more information.
type Parser ¶
type Parser func(*logbuffer.Line, LeveledWriter)
Parser is a user-defined function for converting a log line received from an external system into a leveled logging payload. The given LeveledWriter should be called for every leveled log entry that results from this line. This means that a parser might skip some lines, or emit multiple leveled payloads per line.
Source Files