logutil

package
v0.58.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 3, 2026 License: Apache-2.0 Imports: 10 Imported by: 0

Documentation

Overview

Package logutil provides log-related utilities used by multiple log processing tasks, such as parsers for different log string formats and log string converters.

Index

Constants

View Source 
const EnvoyAccessLogAuthorityFieldKey = "authority"

EnvoyAccessLogAuthorityFieldKey is the key stored in Fields for the request authority/host.

View Source 
const EnvoyAccessLogDownstreamLocalAddressFieldKey = "downstream_local_address"

EnvoyAccessLogDownstreamLocalAddressFieldKey is the key stored in Fields for the downstream local address.

View Source 
const EnvoyAccessLogDownstreamRemoteAddressFieldKey = "downstream_remote_address"

EnvoyAccessLogDownstreamRemoteAddressFieldKey is the key stored in Fields for the downstream remote address.

View Source 
const EnvoyAccessLogDurationFieldKey = "duration"

EnvoyAccessLogDurationFieldKey is the key stored in Fields for the total duration.

View Source 
const EnvoyAccessLogMethodFieldKey = "method"

EnvoyAccessLogMethodFieldKey is the key stored in Fields for the HTTP method.

View Source 
const EnvoyAccessLogPathFieldKey = "path"

EnvoyAccessLogPathFieldKey is the key stored in Fields for the request path.

View Source 
const EnvoyAccessLogProtocolFieldKey = "protocol"

EnvoyAccessLogProtocolFieldKey is the key stored in Fields for the protocol.

View Source 
const EnvoyAccessLogRequestIDFieldKey = "request_id"

EnvoyAccessLogRequestIDFieldKey is the key stored in Fields for the request ID.

View Source 
const EnvoyAccessLogRequestedServerNameFieldKey = "requested_server_name"

EnvoyAccessLogRequestedServerNameFieldKey is the key stored in Fields for the requested server name (SNI).

View Source 
const EnvoyAccessLogResponseFlagsFieldKey = "response_flags"

EnvoyAccessLogResponseFlagsFieldKey is the key stored in Fields for the response flags.

View Source 
const EnvoyAccessLogRouteNameFieldKey = "route_name"

EnvoyAccessLogRouteNameFieldKey is the key stored in Fields for the route name.

View Source 
const EnvoyAccessLogStatusCodeFieldKey = "status_code"

EnvoyAccessLogStatusCodeFieldKey is the key stored in Fields for the response status code.

View Source 
const EnvoyAccessLogTimestampFieldKey = "@timestamp"

EnvoyAccessLogTimestampFieldKey is the key stored in Fields for the timestamp in an Envoy access log.

View Source 
const EnvoyAccessLogUpstreamClusterFieldKey = "upstream_cluster"

EnvoyAccessLogUpstreamClusterFieldKey is the key stored in Fields for the upstream cluster.

View Source 
const EnvoyAccessLogUpstreamHostFieldKey = "upstream_host"

EnvoyAccessLogUpstreamHostFieldKey is the key stored in Fields for the upstream host.

View Source 
const EnvoyAccessLogUpstreamLocalAddressFieldKey = "upstream_local_address"

EnvoyAccessLogUpstreamLocalAddressFieldKey is the key stored in Fields for the upstream local address.

View Source 
const EnvoyAccessLogUserAgentFieldKey = "user_agent"

EnvoyAccessLogUserAgentFieldKey is the key stored in Fields for the user agent.

View Source 
const KLogHeaderDateFieldKey = "@date"

Special header field keys stored in fields.

View Source 
const KLogHeaderSourceLocationFieldKey = "@source"
View Source 
const KLogHeaderThreadIDFieldKey = "@threadid"
View Source 
const KLogHeaderTimeFieldKey = "@time"
View Source 
const MainMessageStructuredFieldKey = "@msg"

MainMessageStructuredFieldKey is the key used to store the main log message in a structured log result.

View Source 
const OriginalMessageFieldKey = "@original"
View Source 
const SeverityStructuredFieldKey = "@severity"

SeverityStructuredFieldKey is the key used to store the log severity in a structured log result.

View Source 
const ZapConsoleCallerFieldKey = "@caller"

ZapConsoleCallerFieldKey is the key stored in Fields for the caller or source location in a Zap console log.

View Source 
const ZapConsoleTimestampFieldKey = "@timestamp"

ZapConsoleTimestampFieldKey is the key stored in Fields for the timestamp in a Zap console log.

Variables

View Source 
var ANSIBeginSequences []string = []string{
	"\\x1b[",
	"\\033[",
	"\\u001B[",
}

ANSIBeginSequences is a list of prefixes of ANSI escape sequences.

View Source 
var EnvoyResponseFlagDescriptions = map[EnvoyResponseFlag]string{
	EnvoyResponseFlagNoError:                          "OK",
	EnvoyResponseFlagNoHealthyUpstream:                "No healthy upstream",
	EnvoyResponseFlagUpstreamConnectionFailure:        "Upstream connection failure",
	EnvoyResponseFlagUpstreamOverflow:                 "Upstream overflow",
	EnvoyResponseFlagNoRouteFound:                     "No route found",
	EnvoyResponseFlagUpstreamRetryLimitExceeded:       "Upstream retry limit exceeded",
	EnvoyResponseFlagNoClusterFound:                   "No cluster found",
	EnvoyResponseFlagDurationTimeout:                  "Duration timeout",
	EnvoyResponseFlagDownstreamConnectionTermination:  "Downstream connection termination",
	EnvoyResponseFlagFailedLocalHealthCheck:           "Failed local health check",
	EnvoyResponseFlagUpstreamRequestTimeout:           "Upstream request timeout",
	EnvoyResponseFlagLocalReset:                       "Local reset",
	EnvoyResponseFlagUpstreamRemoteReset:              "Upstream remote reset",
	EnvoyResponseFlagUpstreamConnectionTermination:    "Upstream connection termination",
	EnvoyResponseFlagDelayInjected:                    "Delay injected",
	EnvoyResponseFlagFaultInjected:                    "Fault injected",
	EnvoyResponseFlagRateLimited:                      "Rate limited",
	EnvoyResponseFlagUnauthorizedExternalService:      "Unauthorized external service",
	EnvoyResponseFlagRateLimitServiceError:            "Rate limit service error",
	EnvoyResponseFlagInvalidEnvoyRequestHeaders:       "Invalid Envoy request headers",
	EnvoyResponseFlagStreamIdleTimeout:                "Stream idle timeout",
	EnvoyResponseFlagDownstreamProtocolError:          "Downstream protocol error",
	EnvoyResponseFlagUpstreamProtocolError:            "Upstream protocol error",
	EnvoyResponseFlagUpstreamMaxStreamDurationReached: "Upstream max stream duration reached",
	EnvoyResponseFlagResponseFromCacheFilter:          "Response from cache filter",
	EnvoyResponseFlagNoFilterConfigFound:              "No filter config found",
	EnvoyResponseFlagOverloadManagerTerminated:        "Overload manager terminated",
	EnvoyResponseFlagDnsResolutionFailed:              "DNS resolution failed",
	EnvoyResponseFlagDropOverload:                     "Drop overload",
	EnvoyResponseFlagDownstreamRemoteReset:            "Downstream remote reset",
	EnvoyResponseFlagUnconditionalDropOverload:        "Unconditional drop overload",
}

EnvoyResponseFlagDescriptions maps Envoy response flags to their human-readable descriptions.

Functions

func ConvertSpecialSequences 

func ConvertSpecialSequences(original string, converter ...SpecialSequenceConverter) string

ConvertSpecialSequences returns the stripped string with applying provided strippers to the original string.

func FormatEnvoySummary added in v0.58.2 

func FormatEnvoySummary(statusCode int, method, requestURL string, responseFlags EnvoyResponseFlags) string

FormatEnvoySummary formats an Envoy access log summary line with status code, method, request URL, and optional response flags.

Types

type ANSIEscapeSequenceStripper 

type ANSIEscapeSequenceStripper struct {
}

ANSIEscapeSequenceStripper removes ANSI escape sequences.

func (*ANSIEscapeSequenceStripper) Convert 

func (a *ANSIEscapeSequenceStripper) Convert(s string) string

Convert implements SpecialSequenceStripper.

type ColumnBoundary added in v0.52.8 

type ColumnBoundary struct {
	Name  string
	Left  int
	Right int
}

ColumnBoundary defines the character indices bounding a single column.

type EnvoyAccessLogTextParser added in v0.58.2 

type EnvoyAccessLogTextParser struct{}

EnvoyAccessLogTextParser parses access log lines formatted in the default Istio text format.

func NewEnvoyAccessLogTextParser added in v0.58.2 

func NewEnvoyAccessLogTextParser() *EnvoyAccessLogTextParser

NewEnvoyAccessLogTextParser creates a new EnvoyAccessLogTextParser instance.

func (*EnvoyAccessLogTextParser) TryParse added in v0.58.2 

func (p *EnvoyAccessLogTextParser) TryParse(message string) *ParseStructuredLogResult

TryParse attempts to parse the given message as an Istio access log.

type EnvoyResponseFlag added in v0.58.2 

type EnvoyResponseFlag string

EnvoyResponseFlag represents standard Envoy response flags. See: https://github.com/envoyproxy/envoy/blob/main/docs/root/configuration/advanced/substitution_formatter.rst 

const (
	EnvoyResponseFlagNoError                    EnvoyResponseFlag = "-"
	EnvoyResponseFlagNoHealthyUpstream          EnvoyResponseFlag = "UH"
	EnvoyResponseFlagUpstreamConnectionFailure  EnvoyResponseFlag = "UF"
	EnvoyResponseFlagUpstreamOverflow           EnvoyResponseFlag = "UO"
	EnvoyResponseFlagNoRouteFound               EnvoyResponseFlag = "NR"
	EnvoyResponseFlagUpstreamRetryLimitExceeded EnvoyResponseFlag = "URX"
	EnvoyResponseFlagNoClusterFound             EnvoyResponseFlag = "NC"
	EnvoyResponseFlagDurationTimeout            EnvoyResponseFlag = "DT"

	// HTTP only
	EnvoyResponseFlagDownstreamConnectionTermination  EnvoyResponseFlag = "DC"
	EnvoyResponseFlagFailedLocalHealthCheck           EnvoyResponseFlag = "LH"
	EnvoyResponseFlagUpstreamRequestTimeout           EnvoyResponseFlag = "UT"
	EnvoyResponseFlagLocalReset                       EnvoyResponseFlag = "LR"
	EnvoyResponseFlagUpstreamRemoteReset              EnvoyResponseFlag = "UR"
	EnvoyResponseFlagUpstreamConnectionTermination    EnvoyResponseFlag = "UC"
	EnvoyResponseFlagDelayInjected                    EnvoyResponseFlag = "DI"
	EnvoyResponseFlagFaultInjected                    EnvoyResponseFlag = "FI"
	EnvoyResponseFlagRateLimited                      EnvoyResponseFlag = "RL"
	EnvoyResponseFlagUnauthorizedExternalService      EnvoyResponseFlag = "UAEX"
	EnvoyResponseFlagRateLimitServiceError            EnvoyResponseFlag = "RLSE"
	EnvoyResponseFlagInvalidEnvoyRequestHeaders       EnvoyResponseFlag = "IH"
	EnvoyResponseFlagStreamIdleTimeout                EnvoyResponseFlag = "SI"
	EnvoyResponseFlagDownstreamProtocolError          EnvoyResponseFlag = "DPE"
	EnvoyResponseFlagUpstreamProtocolError            EnvoyResponseFlag = "UPE"
	EnvoyResponseFlagUpstreamMaxStreamDurationReached EnvoyResponseFlag = "UMSDR"
	EnvoyResponseFlagResponseFromCacheFilter          EnvoyResponseFlag = "RFCF"
	EnvoyResponseFlagNoFilterConfigFound              EnvoyResponseFlag = "NFCF"
	EnvoyResponseFlagOverloadManagerTerminated        EnvoyResponseFlag = "OM"
	EnvoyResponseFlagDnsResolutionFailed              EnvoyResponseFlag = "DF"
	EnvoyResponseFlagDropOverload                     EnvoyResponseFlag = "DO"
	EnvoyResponseFlagDownstreamRemoteReset            EnvoyResponseFlag = "DR"
	EnvoyResponseFlagUnconditionalDropOverload        EnvoyResponseFlag = "UDO"
)

type EnvoyResponseFlags added in v0.58.2 

type EnvoyResponseFlags []EnvoyResponseFlag

EnvoyResponseFlags represents a collection of Envoy response flags.

func ParseEnvoyResponseFlags added in v0.58.2 

func ParseEnvoyResponseFlags(raw string) EnvoyResponseFlags

ParseEnvoyResponseFlags parses single or comma-separated Envoy response flag strings.

func (EnvoyResponseFlags) Contains added in v0.58.2 

func (f EnvoyResponseFlags) Contains(flag EnvoyResponseFlag) bool

Contains returns true if the specified flag is present in the response flags.

func (EnvoyResponseFlags) HasError added in v0.58.2 

func (f EnvoyResponseFlags) HasError() bool

HasError returns true if there is any error flag present, excluding non-error flags such as NoError, RFCF, and DI.

func (EnvoyResponseFlags) String added in v0.58.2 

func (f EnvoyResponseFlags) String() string

String returns the comma-separated flag representation (e.g. "UH,URX").

func (EnvoyResponseFlags) Summary added in v0.58.2 

func (f EnvoyResponseFlags) Summary() string

Summary returns human-readable descriptions of the flags joined by comma.

type FallbackRawTextLogParser 

type FallbackRawTextLogParser struct{}

FallbackRawTextLogParser uses the given message directly as the result of main message of parsing structured message.

func (*FallbackRawTextLogParser) TryParse 

func (f *FallbackRawTextLogParser) TryParse(message string) *ParseStructuredLogResult

TryParse implements StructuredLogParser.

type JsonlTextParser added in v0.52.8 

type JsonlTextParser struct{}

JsonlTextParser parses given JSONL formatted string.

func NewJsonlTextParser added in v0.52.8 

func NewJsonlTextParser() *JsonlTextParser

NewJsonlTextParser creates a new JsonlTextParser.

func (*JsonlTextParser) TryParse added in v0.52.8 

func (j *JsonlTextParser) TryParse(originalMessage string) *ParseStructuredLogResult

TryParse implements StructuredLogParser.

type KLogTextParser 

type KLogTextParser struct {
	// contains filtered or unexported fields
}

KLogTextParser parses given klog formatted string. Example klog: I0929 08:20:24.205299 1949 kubelet_getters.go:219] "Pod status updated" pod="kube-system/kube-proxy-gke-p0-gke-basic-1-default-6400229f-0hgr" status="Running"

func NewKLogTextParser 

func NewKLogTextParser(hasHeader bool) *KLogTextParser

func (*KLogTextParser) TryParse 

func (k *KLogTextParser) TryParse(message string) *ParseStructuredLogResult

TryParse implements StructuredLogParser.

type LogfmtTextParser 

type LogfmtTextParser struct {
	// contains filtered or unexported fields
}

LogfmtTextParser parses given logfmt formatted string. Reference: https://github.com/hynek/structlog/issues/511#issuecomment-1916426273

func NewLogfmtTextParser 

func NewLogfmtTextParser() *LogfmtTextParser

func (*LogfmtTextParser) TryParse 

func (l *LogfmtTextParser) TryParse(message string) *ParseStructuredLogResult

TryParse implements StructuredLogParser.

type MultiTextLogParser 

type MultiTextLogParser struct {
	// contains filtered or unexported fields
}

func NewMultiTextLogParser 

func NewMultiTextLogParser(parsers ...StructuredLogParser) *MultiTextLogParser

func (*MultiTextLogParser) TryParse 

func (m *MultiTextLogParser) TryParse(message string) *ParseStructuredLogResult

TryParse implements StructuredLogParser.

type ParseStructuredLogResult 

type ParseStructuredLogResult struct {
	Fields map[string]any
}

ParseStructuredLogResult represents the result of parsing a structured log message.

func (*ParseStructuredLogResult) MainMessage 

func (p *ParseStructuredLogResult) MainMessage() (string, error)

MainMessage returns the main message of the structured log, or an error if not found or not a string.

func (*ParseStructuredLogResult) Raw 

func (p *ParseStructuredLogResult) Raw() string

func (*ParseStructuredLogResult) Severity 

func (p *ParseStructuredLogResult) Severity() (*pb.Severity, error)

Severity returns the severity of the structured log, or an error if not found or not of type enum.Severity.

func (*ParseStructuredLogResult) StringField 

func (p *ParseStructuredLogResult) StringField(field string) (string, error)

StringField returns the string value of a specific field, or an error if not found or not a string.

type ParserRule added in v0.57.3 

type ParserRule[T any] struct {
	// Match returns true if the rule applies to the given context.
	Match func(ctx T) bool
	// Parser is the structured log parser to execute when Match returns true.
	Parser StructuredLogParser
}

ParserRule defines a matching rule for a specific context of type T.

type RegexSequenceConverter 

type RegexSequenceConverter struct {
	// Regex is the regular expression to match.
	Regex *regexp.Regexp
	// Repl is the replacement string.
	Repl string
}

func MustNewRegexSequenceConverter 

func MustNewRegexSequenceConverter(regex string, repl string) *RegexSequenceConverter

func NewRegexSequenceConverter 

func NewRegexSequenceConverter(regex string, repl string) (*RegexSequenceConverter, error)

NewRegexSequenceConverter instantiates RegexSequenceConverter from given string regex and replace target.

func (*RegexSequenceConverter) Convert 

func (r *RegexSequenceConverter) Convert(s string) string

Convert implements SpecialSequenceConverter.

type SelectorLogParser added in v0.57.3 

type SelectorLogParser[T any] struct {
	// contains filtered or unexported fields
}

SelectorLogParser selects and executes a StructuredLogParser based on a context object of type T.

func NewSelectorLogParser added in v0.57.3 

func NewSelectorLogParser[T any](defaultParser StructuredLogParser, rules ...ParserRule[T]) *SelectorLogParser[T]

NewSelectorLogParser creates a new SelectorLogParser instance.

func (*SelectorLogParser[T]) TryParse added in v0.57.3 

func (s *SelectorLogParser[T]) TryParse(ctx T, message string) *ParseStructuredLogResult

TryParse evaluates rules against the provided context T and executes the matching StructuredLogParser. If no rule matches or the matching parser returns nil, it falls back to defaultParser.

type SequenceConverter 

type SequenceConverter struct {
	// To is the replace result of any items in the From field.
	To string
	// From is the list of string to be replaced with the To field.
	From []string
}

SequenceConverter replaces specific sequences to a string.

func (*SequenceConverter) Convert 

func (c *SequenceConverter) Convert(s string) string

type SpecialSequenceConverter 

type SpecialSequenceConverter interface {
	// Convert receives the original string and returns the converted string.
	Convert(s string) string
}

SpecialSequenceConverter converts specific sequences from string to the other or removes them. (e.g ASCII escape characters, ANSI color characters)

type StructuredLogParser 

type StructuredLogParser interface {
	// TryParse attempts to parse the given message into *ParseStructuredLogResult.
	// It returns nil when the given message is not in the format.
	TryParse(message string) *ParseStructuredLogResult
}

StructuredLogParser is the interface to parse string represented structured logs like klog.

type TabulateLineType added in v0.52.8 

type TabulateLineType int

TabulateLineType indicates the classified type of a line in a tabulate format. 

const (
	// TabulateLineTypeUnknown represents an unrecognized or out-of-table line.
	TabulateLineTypeUnknown TabulateLineType = iota
	// TabulateLineTypeHeaderCandidate represents a line that looks like a table header.
	TabulateLineTypeHeaderCandidate
	// TabulateLineTypeSeparator represents a separator line (e.g., "--- ---").
	TabulateLineTypeSeparator
	// TabulateLineTypeBody represents a data row within a table.
	TabulateLineTypeBody
)

type TabulateParseResult added in v0.52.8 

type TabulateParseResult struct {
	// Type classification of the processed line.
	Type TabulateLineType
	// Columns is populated for HeaderCandidate and Body types, representing the column names.
	Columns []string
	// Values contains the parsed key-value pairs for a Body line.
	Values map[string]string
}

TabulateParseResult holds the parsing result of a single line.

type TabulateReader added in v0.52.8 

type TabulateReader struct {
	ColumnBoundaries []ColumnBoundary
	Headers          []string
}

TabulateReader is a stateful reader for space-padded tabulate tables. It tracks headers and vertical column boundaries dynamically.

func NewTabulateReader added in v0.52.8 

func NewTabulateReader() *TabulateReader

NewTabulateReader creates a new TabulateReader instance.

func (*TabulateReader) ParseLine added in v0.52.8 

func (r *TabulateReader) ParseLine(line string) (*TabulateParseResult, error)

ParseLine processes a single line, returning the parsed TabulateParseResult. It sequentially checks if the line is a separator, a body row (if inside a table), or a header candidate. Returns an error if a format violation is found (which resets internal state).

func (*TabulateReader) Reset added in v0.52.8 

func (r *TabulateReader) Reset()

Reset clears the internal state when a table ends or a format error occurs.

type UnicodeUnquoteConverter 

type UnicodeUnquoteConverter struct{}

UnicodeUnquoteConverter replaces escaped unicode characters like `\\xe2\\x80\\xa6` into the corresponded unicode string.

func (*UnicodeUnquoteConverter) Convert 

func (u *UnicodeUnquoteConverter) Convert(s string) string

Convert implements SpecialSequenceConverter.

type ZapConsoleTextParser added in v0.57.3 

type ZapConsoleTextParser struct{}

ZapConsoleTextParser parses log lines formatted using Zap's ConsoleEncoder. Zap console logs start with a timestamp (part 0), followed by severity level (part 1), caller location (part 2), and message. An example log entry is: 2026-02-18T06:58:06.999Z\tinfo\tkubernetes/kubernetes.go:282\tUsing pod service account via in-cluster config\t{"kind": "receiver", "name": "prometheus", "discovery": "kubernetes"}

func NewZapConsoleTextParser added in v0.57.3 

func NewZapConsoleTextParser() *ZapConsoleTextParser

NewZapConsoleTextParser creates a new ZapConsoleTextParser instance.

func (*ZapConsoleTextParser) TryParse added in v0.57.3 

func (z *ZapConsoleTextParser) TryParse(message string) *ParseStructuredLogResult

TryParse attempts to parse the given message as a Zap ConsoleEncoder formatted log line. It requires part 0 to be a timestamp, part 1 to be a severity level, and part 2 to be a caller location. It returns nil when the message does not match these specifications.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.