logging

package
v0.5.81 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 5, 2026 License: MIT Imports: 39 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// GraylogVersion - GELF spec version
	GraylogVersion = "1.1"
	// GraylogLevel - Log Level (informational)
	GraylogLevel = 6
	// GraylogMethod - Method to send
	GraylogMethod = "POST"
)
View Source 
const (
	// LogstashTCP for TCP inputs
	LogstashTCP = "tcp"
	// LogstashUDP for UDP inputs
	LogstashUDP = "udp"
	// LogstashHTTP for HTTP inputs
	LogstashHTTP = "http"
)
View Source 
const (
	// LogstashMethod Method to send requests
	LogstashMethod = "POST"
	// LogstashContentType Content Type for requests
	LogstashContentType = "application/json"
)
View Source 
const (
	// SplunkMethod Method to send requests
	SplunkMethod = "POST"
	// SplunkContentType Content Type for requests
	SplunkContentType = "application/json"
)
View Source 
const (
	// NotReturned - Value not returned from agent
	NotReturned = "not returned"
	// Mismatched - Value mismatched in log entries
	Mismatched = "mismatched"
)
View Source 
const (
	// Default time format for loggers
	LoggerTimeFormat string = "2006-01-02T15:04:05.999Z07:00"
)

Variables

This section is empty.

Functions

func CreateDebugHTTP added in v0.4.5 

func CreateDebugHTTP(cfg config.LocalLogger) (*zerolog.Logger, error)

CreateDebugHTTP to initialize the debug HTTP logger

func GetNodeLogs added in v0.5.2 

func GetNodeLogs(db *gorm.DB, logType, env, uuid string, since time.Time, limit int, search string, severity string) ([]map[string]any, error)

GetNodeLogs retrieves recent log entries for a single node (status or result). logType must be "status" or "result". Results are ordered by created_at DESC. If since is non-zero only entries created strictly after that time are returned. limit is clamped to [1, 1000].

search is an optional free-text filter (substring, case-insensitive). It runs as a `LIKE` against the human-readable text columns of the row:

  • status: line + message + filename
  • result: name + action + columns (the serialized JSON of matched fields)

Empty search disables the filter — same behavior as a missing param.

The `LIKE` is unindexed today. If the result_data / status_data tables grow large enough to make this slow, an operator-side workaround is to narrow `since` first, which keeps the matched row count small.

func GetNodeResultBucketed added in v0.5.2 

func GetNodeResultBucketed(db *gorm.DB, env, uuid string, since time.Time, bucketSeconds int) ([]dbutil.BucketedRow, error)

GetNodeResultBucketed mirrors GetNodeStatusBucketed for osquery_result_data.

func GetNodeResultTimestamps added in v0.5.2 

func GetNodeResultTimestamps(db *gorm.DB, env, uuid string, since time.Time) ([]time.Time, error)

func GetNodeStatusBucketed added in v0.5.2 

func GetNodeStatusBucketed(db *gorm.DB, env, uuid string, since time.Time, bucketSeconds int) ([]dbutil.BucketedRow, error)

GetNodeStatusBucketed returns per-bucket row counts for `uuid` in `env` since `since`, with buckets aligned to `bucketSeconds`. The SQL pushes the histogram into the database (one GROUP BY) instead of shipping every timestamp to the API process — orders of magnitude less wire traffic on chatty nodes.

func GetNodeStatusTimestamps added in v0.5.2 

func GetNodeStatusTimestamps(db *gorm.DB, env, uuid string, since time.Time) ([]time.Time, error)

GetNodeStatusTimestamps and GetNodeResultTimestamps return just the CreatedAt column for every status/result log row a given node has shipped since `since`. Used by the per-node activity heatmap so it can bucket on the API side without dragging the row bodies across the wire.

Returning a slice of timestamps (rather than int64 epochs) keeps the downstream bucketing arithmetic in Go's time domain, which is what the rest of cmd/api/handlers/stats.go uses.

func GetQueryResults added in v0.5.2 

func GetQueryResults(db *gorm.DB, name string, since time.Time, page, pageSize int) ([]map[string]any, int64, error)

GetQueryResults retrieves rows of query result data (one per node) for a single query name. Results are ordered by created_at ASC (oldest first — query results are append-only). If since is non-zero only rows created strictly after that time are returned. page is 1-indexed; pageSize is clamped to [1, 1000]; pageSize <= 0 defaults to 100. Returns the page items, total matching rows, and any error.

func LoadLogstash 

func LoadLogstash(file string) (config.LogstashLogger, error)

LoadLogstash - Function to load the Logstash configuration from JSON file

func StreamQueryResults added in v0.5.2 

func StreamQueryResults(db *gorm.DB, name string, fn func(OsqueryQueryData) error) error

StreamQueryResults invokes fn for each row of query result data for `name`, ordered by created_at ASC. Rows are read via a cursor so memory usage stays bounded — used by the CSV exporter. fn may return an error to stop iteration; that error is returned by StreamQueryResults.

Types

type AlertMatcher added in v0.5.8 

type AlertMatcher interface {
	// MatchResultLogs is called with the decoded result-log batch
	// after parse, before dispatch. It must not block.
	MatchResultLogs(envID uint, environment string, logs []types.LogResultData)
	// MatchStatusLogs is the status-log counterpart.
	MatchStatusLogs(envID uint, environment string, logs []types.LogStatusData)
	// MatchQueryResult is called per query in ProcessLogQueryResult.
	MatchQueryResult(envID uint, environment, queryName string, result json.RawMessage, status int, message string)
}

AlertMatcher is the ingest-path hook into the alerting subsystem. Implemented by the alerts worker in osctrl-tls; the nil case is the feature-off state (nil interface + nil receiver = zero cost).

type CountedExporter added in v0.5.7 

type CountedExporter struct {
	// contains filtered or unexported fields
}

CountedExporter wraps a DataExporter and increments SinkStats counters on every Export call. The wrapping is transparent — the underlying exporter receives the call as normal; the counters are incremented before the call so a failed export still counts the attempt.

func NewCountedExporter added in v0.5.7 

func NewCountedExporter(exp DataExporter, stats *SinkStats) *CountedExporter

NewCountedExporter wraps exporter exp and records bytes/count in stats.

func (*CountedExporter) Close added in v0.5.7 

func (c *CountedExporter) Close() error

func (*CountedExporter) Export added in v0.5.7 

func (c *CountedExporter) Export(logType string, data []byte, params ExportParams) error

func (*CountedExporter) IsEnabled added in v0.5.7 

func (c *CountedExporter) IsEnabled() bool

func (*CountedExporter) Name added in v0.5.7 

func (c *CountedExporter) Name() string

func (*CountedExporter) Stats added in v0.5.7 

func (c *CountedExporter) Stats() *SinkStats

Stats returns the SinkStats pointer, or nil if the exporter is not counted (e.g. a plain DataExporter that was not wrapped).

type DataExporter added in v0.5.6 

type DataExporter interface {
	Name() string
	IsEnabled() bool
	Export(logType string, data []byte, params ExportParams) error
	// Close releases any resources held by this exporter (network
	// clients, DB connections, file handles). It is called by
	// LoggerTLS.ReplaceExporters on every exporter in the old set
	// during a hot reload so sinks are not leaked across reloads.
	// Stateless exporters return nil.
	Close() error
}

DataExporter is a destination for osquery status, result, and query data.

func CreateExporter added in v0.5.6 

func CreateExporter(exporterType string, cfg config.ServiceParameters, mgr *settings.Settings) (DataExporter, bool, error)

CreateExporter builds one exporter implementation from its configured type. This is the only place that should switch on exporter type; request-time dispatch goes through the DataExporter interface.

type ExportParams added in v0.5.6 

type ExportParams struct {
	Environment string
	UUID        string
	QueryName   string
	Status      int
	Debug       bool
}

ExportParams contains the metadata shared by every osquery data exporter. QueryName and Status are set only for on-demand query result exports.

type ExporterEntry added in v0.5.7 

type ExporterEntry struct {
	SinkID     uint
	Exporter   DataExporter
	Categories []string
}

ExporterEntry pairs a sink DB row ID with its built DataExporter and the categories it should receive. Empty Categories means "all categories" (backwards-compatible default).

type GraylogMessage 

type GraylogMessage struct {
	Version      string `json:"version"`
	Host         string `json:"host"`
	ShortMessage string `json:"short_message"`
	Timestamp    int64  `json:"timestamp"`
	Level        uint   `json:"level"`
	Environment  string `json:"_environment"`
	Type         string `json:"_type"`
	UUID         string `json:"_uuid"`
}

GraylogMessage to handle log format to be sent to Graylog

type LogReader added in v0.5.6 

type LogReader interface {
	// NodeLogs returns recent log entries for a single node (status or
	// result), ordered by created_at DESC. since is exclusive; empty means
	// no lower bound. limit is clamped to [1,1000] by the caller. search
	// is an optional case-insensitive substring filter against the row's
	// human-readable columns (best-effort for S3). severity is an optional
	// filter for status logs only (osquery severity integers: 0=info,
	// 1=warning, 2=error); "" or -1 means "no severity filter". Ignored
	// for result logs.
	NodeLogs(logType, env, uuid string, since time.Time, limit int, search string, severity string) ([]map[string]any, error)

	// QueryResults returns rows of query result data (one per node) for a
	// single query name, ordered by created_at ASC. env is the environment
	// UUID/name used to scope the read (the DB reader ignores it; the S3
	// reader uses it as the key prefix). Pagination is 1-indexed; pageSize
	// clamped to [1,1000]. Returns the page items, the total matching
	// count, and any error.
	QueryResults(env, name string, since time.Time, page, pageSize int) ([]map[string]any, int64, error)

	// StreamQueryResults invokes fn for each row of query result data for
	// `name`, ordered by created_at ASC. env scopes the read (DB reader
	// ignores it; S3 reader uses it as the key prefix). Rows are streamed
	// so memory stays bounded — used by the CSV exporter and the
	// console/file-explorer result decoders. fn may return an error to
	// stop iteration.
	StreamQueryResults(env, name string, fn func(OsqueryQueryData) error) error
}

LogReader is the read-side abstraction over status / result / query logs.

The historical implementation reads the `osquery_status_data`, `osquery_result_data` and `osquery_query_data` tables from the GORM DB (see dbLogReader). When the TLS logger is configured to ship logs to S3, those tables are empty — the data lives in S3 objects instead — and s3LogReader (see s3_reader.go) is wired in so the API/console/file explorer can still surface logs to the frontend.

The interface intentionally mirrors the *read* functions the API and the console/file-explorer managers already use, so a handler does not care whether the backing store is the DB or S3.

func NewDBLogReader added in v0.5.6 

func NewDBLogReader(db *gorm.DB) LogReader

NewDBLogReader returns a LogReader backed by the given GORM DB.

func NewS3LogReader added in v0.5.6 

func NewS3LogReader(client *s3.Client, bucket string) LogReader

NewS3LogReader returns a LogReader backed by the given S3 client/bucket. The client is the same one LoggerS3 already constructs at TLS startup, so the reader shares the configured credentials/region.

type LoggerDB 

type LoggerDB struct {
	Database *backend.DBManager
	Enabled  bool
}

LoggerDB will be used to log data using a database

func CreateLoggerDB 

func CreateLoggerDB(backend *backend.DBManager) (*LoggerDB, error)

CreateLoggerDB to initialize the logger without reading a config file

func CreateLoggerDBConfig 

func CreateLoggerDBConfig(dbConfig *config.YAMLConfigurationDB) (*LoggerDB, error)

CreateLoggerDB to initialize the logger without reading a config file

func (*LoggerDB) CleanQueryLogs 

func (logDB *LoggerDB) CleanQueryLogs(entries int64) error

CleanQueryLogs will delete old query logs

func (*LoggerDB) CleanResultLogs 

func (logDB *LoggerDB) CleanResultLogs(environment string, seconds int64) error

CleanResultLogs will delete old status logs

func (*LoggerDB) CleanStatusLogs 

func (logDB *LoggerDB) CleanStatusLogs(environment string, seconds int64) error

CleanStatusLogs will delete old status logs

func (*LoggerDB) Close added in v0.5.7 

func (logDB *LoggerDB) Close() error

Close releases the underlying SQL connection pool when the DB logger was built from its own config (a separate logging DB). When the DB logger shares the primary osctrl backend, the caller owns that connection and Close is a no-op — the primary DB must outlive any one exporter set. We detect "owns the connection" by checking whether Database is non-nil; the primary-DB case is constructed by CreateLoggerDB which is only called from cmd/tls when reusing the primary backend, so we do not close there.

To keep this safe for both paths, Close only closes when the Database.Config has a non-empty Host or FilePath (i.e. it was built from a dedicated logging DB config), which is true for CreateLoggerDBConfig and not for the primary reuse path. This is conservative; a primary-DB logger that is hot-reloaded away simply leaves the primary pool intact.

func (*LoggerDB) Export added in v0.5.6 

func (logDB *LoggerDB) Export(logType string, data []byte, params ExportParams) error

func (*LoggerDB) IsEnabled added in v0.5.6 

func (logDB *LoggerDB) IsEnabled() bool

func (*LoggerDB) Log 

func (logDB *LoggerDB) Log(logType string, data []byte, environment, uuid string, debug bool)

Log - Function that sends JSON result/status/query logs to the configured DB

func (*LoggerDB) Name added in v0.5.6 

func (logDB *LoggerDB) Name() string

func (*LoggerDB) Query 

func (logDB *LoggerDB) Query(data []byte, environment, uuid, name string, status int, debug bool)

Query - Function that sends JSON query logs to the configured DB

func (*LoggerDB) QueryLogs 

func (logDB *LoggerDB) QueryLogs(name string) ([]OsqueryQueryData, error)

QueryLogs will retrieve all query logs

func (*LoggerDB) Result 

func (logDB *LoggerDB) Result(data []byte, environment, uuid string, debug bool)

Result - Function that sends JSON result logs to the configured DB

func (*LoggerDB) ResultLogs 

func (logDB *LoggerDB) ResultLogs(uuid, environment string, seconds int64) ([]OsqueryResultData, error)

ResultLogs will retrieve all result logs

func (*LoggerDB) ResultLogsLimit 

func (logDB *LoggerDB) ResultLogsLimit(uuid, environment string, limit int) ([]OsqueryResultData, error)

ResultLogsLimit will retrieve a limited number of result logs

func (*LoggerDB) Settings 

func (logDB *LoggerDB) Settings(mgr *settings.Settings)

Settings - Function to prepare settings for the logger

func (*LoggerDB) Status 

func (logDB *LoggerDB) Status(data []byte, environment, uuid string, debug bool)

Status - Function that sends JSON status logs to the configured DB

func (*LoggerDB) StatusLogs 

func (logDB *LoggerDB) StatusLogs(uuid, environment string, seconds int64) ([]OsqueryStatusData, error)

StatusLogs will retrieve all status logs

func (*LoggerDB) StatusLogsLimit 

func (logDB *LoggerDB) StatusLogsLimit(uuid, environment string, limit int) ([]OsqueryStatusData, error)

StatusLogsLimit will retrieve a limited number of status logs

type LoggerElastic 

type LoggerElastic struct {
	Configuration config.ElasticLogger
	Enabled       bool
	Client        *elasticsearch.Client
}

LoggerElastic will be used to log data using Elastic

func CreateLoggerElastic 

func CreateLoggerElastic(cfg *config.ElasticLogger) (*LoggerElastic, error)

CreateLoggerElastic to initialize the logger

func (*LoggerElastic) Close added in v0.5.7 

func (logE *LoggerElastic) Close() error

Close releases resources held by the Elasticsearch logger. The elasticsearch client maintains an HTTP transport pool; closing it releases those connections so a hot reload does not leak them.

func (*LoggerElastic) Export added in v0.5.6 

func (logE *LoggerElastic) Export(logType string, data []byte, params ExportParams) error

func (*LoggerElastic) IndexName 

func (logE *LoggerElastic) IndexName() string

IndexName - Function to return the index name

func (*LoggerElastic) IsEnabled added in v0.5.6 

func (logE *LoggerElastic) IsEnabled() bool

func (*LoggerElastic) Name added in v0.5.6 

func (logE *LoggerElastic) Name() string

func (*LoggerElastic) Send 

func (logE *LoggerElastic) Send(logType string, data []byte, environment, uuid string, debug bool)

Send - Function that sends JSON logs to Elastic

func (*LoggerElastic) Settings 

func (logE *LoggerElastic) Settings(mgr *settings.Settings)

Settings - Function to prepare settings for the logger

type LoggerFile 

type LoggerFile struct {
	Enabled  bool
	Filename string
	Logger   *zerolog.Logger
}

LoggerFile will be used to log data using external file

func CreateLoggerFile 

func CreateLoggerFile(cfg *config.LocalLogger) (*LoggerFile, error)

CreateLoggerFile to initialize the logger

func (*LoggerFile) Close added in v0.5.7 

func (logFile *LoggerFile) Close() error

Close releases resources held by the file logger. The underlying lumberjack rotating writer closes its file handle when garbage collected; there is no explicit Close on the zerolog wrapper to call here, so this is a no-op.

func (*LoggerFile) Export added in v0.5.6 

func (logFile *LoggerFile) Export(logType string, data []byte, params ExportParams) error

func (*LoggerFile) IsEnabled added in v0.5.6 

func (logFile *LoggerFile) IsEnabled() bool

func (*LoggerFile) Log 

func (logFile *LoggerFile) Log(logType string, data []byte, environment, uuid string, debug bool)

Log - Function that sends JSON result/status/query logs to stdout

func (*LoggerFile) Name added in v0.5.6 

func (logFile *LoggerFile) Name() string

func (*LoggerFile) Query 

func (logFile *LoggerFile) Query(data []byte, environment, uuid, name string, status int, debug bool)

Query - Function that sends JSON query logs to stdout

func (*LoggerFile) Result 

func (logFile *LoggerFile) Result(data []byte, environment, uuid string, debug bool)

Result - Function that sends JSON result logs to stdout

func (*LoggerFile) Settings 

func (logFile *LoggerFile) Settings(mgr *settings.Settings)

Settings - Function to prepare settings for the logger

func (*LoggerFile) Status 

func (logFile *LoggerFile) Status(data []byte, environment, uuid string, debug bool)

Status - Function that sends JSON status logs to stdout

type LoggerGraylog 

type LoggerGraylog struct {
	Configuration config.GraylogLogger
	Headers       map[string]string
	Enabled       bool
}

LoggerGraylog will be used to log data using Graylog

func CreateLoggerGraylog 

func CreateLoggerGraylog(cfg *config.GraylogLogger) (*LoggerGraylog, error)

CreateLoggerGraylog to initialize the logger

func (*LoggerGraylog) Close added in v0.5.7 

func (logGL *LoggerGraylog) Close() error

Close releases resources held by the Graylog logger. Graylog sends data via per-call HTTP POSTs, so there is no persistent client to close.

func (*LoggerGraylog) Export added in v0.5.6 

func (logGL *LoggerGraylog) Export(logType string, data []byte, params ExportParams) error

func (*LoggerGraylog) IsEnabled added in v0.5.6 

func (logGL *LoggerGraylog) IsEnabled() bool

func (*LoggerGraylog) Name added in v0.5.6 

func (logGL *LoggerGraylog) Name() string

func (*LoggerGraylog) Send 

func (logGL *LoggerGraylog) Send(logType string, data []byte, environment, uuid string, debug bool)

Send - Function that sends JSON logs to Graylog

func (*LoggerGraylog) Settings 

func (logGL *LoggerGraylog) Settings(mgr *settings.Settings)

Settings - Function to prepare settings for the logger

type LoggerKafka 

type LoggerKafka struct {
	Enabled bool
	// contains filtered or unexported fields
}

func CreateLoggerKafka 

func CreateLoggerKafka(config *config.KafkaLogger) (*LoggerKafka, error)

func (*LoggerKafka) Close added in v0.5.7 

func (l *LoggerKafka) Close() error

Close releases the Kafka producer client so a hot reload does not leak broker connections. In-flight records are flushed by kgo before the client returns.

func (*LoggerKafka) Export added in v0.5.6 

func (l *LoggerKafka) Export(logType string, data []byte, params ExportParams) error

func (*LoggerKafka) IsEnabled added in v0.5.6 

func (l *LoggerKafka) IsEnabled() bool

func (*LoggerKafka) Name added in v0.5.6 

func (l *LoggerKafka) Name() string

func (*LoggerKafka) Send 

func (l *LoggerKafka) Send(logType string, data []byte, environment, uuid string, debug bool)

func (*LoggerKafka) Settings 

func (l *LoggerKafka) Settings(mgr *settings.Settings)

type LoggerKinesis 

type LoggerKinesis struct {
	Configuration config.KinesisLogger
	KinesisClient *kinesis.Client
	Enabled       bool
}

LoggerKinesis will be used to log data using Kinesis

func CreateLoggerKinesis 

func CreateLoggerKinesis(cfg *config.KinesisLogger) (*LoggerKinesis, error)

CreateLoggerKinesis to initialize the logger

func (*LoggerKinesis) Close added in v0.5.7 

func (logSK *LoggerKinesis) Close() error

Close releases resources held by the Kinesis logger. The AWS SDK v2 kinesis.Client has no explicit Close; its HTTP transport pool is garbage-collected. This is a no-op kept for interface compliance.

func (*LoggerKinesis) Export added in v0.5.6 

func (logSK *LoggerKinesis) Export(logType string, data []byte, params ExportParams) error

func (*LoggerKinesis) IsEnabled added in v0.5.6 

func (logSK *LoggerKinesis) IsEnabled() bool

func (*LoggerKinesis) Name added in v0.5.6 

func (logSK *LoggerKinesis) Name() string

func (*LoggerKinesis) Send 

func (logSK *LoggerKinesis) Send(logType string, data []byte, environment, uuid string, debug bool)

Send - Function that sends JSON logs to Splunk HTTP Event Collector

func (*LoggerKinesis) Settings 

func (logSK *LoggerKinesis) Settings(mgr *settings.Settings)

Settings - Function to prepare settings for the logger

type LoggerLogstash 

type LoggerLogstash struct {
	Configuration config.LogstashLogger
	Headers       map[string]string
	Enabled       bool
}

LoggerLogstash will be used to log data using Logstash

func CreateLoggerLogstash 

func CreateLoggerLogstash(cfg *config.LogstashLogger) (*LoggerLogstash, error)

CreateLoggerLogstash to initialize the logger

func (*LoggerLogstash) Close added in v0.5.7 

func (logLS *LoggerLogstash) Close() error

Close releases resources held by the Logstash logger. Logstash opens a fresh TCP/UDP connection per Send and uses per-call HTTP POSTs, so there is no persistent client to close.

func (*LoggerLogstash) Export added in v0.5.6 

func (logLS *LoggerLogstash) Export(logType string, data []byte, params ExportParams) error

func (*LoggerLogstash) IsEnabled added in v0.5.6 

func (logLS *LoggerLogstash) IsEnabled() bool

func (*LoggerLogstash) Name added in v0.5.6 

func (logLS *LoggerLogstash) Name() string

func (*LoggerLogstash) Send added in v0.5.6 

func (logLS *LoggerLogstash) Send(logType string, data []byte, environment, uuid string, debug bool)

Send routes Logstash exports to the configured protocol.

func (*LoggerLogstash) SendHTTP 

func (logLS *LoggerLogstash) SendHTTP(logType string, data []byte, environment, uuid string, debug bool)

SendHTTP - Function that sends JSON logs to Logstash via HTTP

func (*LoggerLogstash) SendTCP 

func (logLS *LoggerLogstash) SendTCP(logType string, data []byte, environment, uuid string, debug bool)

SendTCP - Function that sends data to Logstash via TCP

func (*LoggerLogstash) SendUDP 

func (logLS *LoggerLogstash) SendUDP(logType string, data []byte, environment, uuid string, debug bool)

SendUDP - Function that sends data to Logstash via UDP

func (*LoggerLogstash) Settings 

func (logLS *LoggerLogstash) Settings(mgr *settings.Settings)

Settings - Function to prepare settings for the logger

type LoggerNone 

type LoggerNone struct {
	Enabled bool
}

LoggerNone will be used to not log any data

func CreateLoggerNone 

func CreateLoggerNone() (*LoggerNone, error)

CreateLoggerNone to initialize the logger

func (*LoggerNone) Close added in v0.5.7 

func (logNone *LoggerNone) Close() error

Close releases resources held by the none logger. None are held.

func (*LoggerNone) Export added in v0.5.6 

func (logNone *LoggerNone) Export(logType string, data []byte, params ExportParams) error

func (*LoggerNone) IsEnabled added in v0.5.6 

func (logNone *LoggerNone) IsEnabled() bool

func (*LoggerNone) Log 

func (logNone *LoggerNone) Log(logType string, data []byte, environment, uuid string, debug bool)

Log - Function that sends JSON result/status/query logs to stdout

func (*LoggerNone) Name added in v0.5.6 

func (logNone *LoggerNone) Name() string

func (*LoggerNone) Query 

func (logNone *LoggerNone) Query(data []byte, environment, uuid, name string, status int, debug bool)

Query - Function that sends JSON query logs to stdout

func (*LoggerNone) Result 

func (logNone *LoggerNone) Result(data []byte, environment, uuid string, debug bool)

Result - Function that sends JSON result logs to stdout

func (*LoggerNone) Settings 

func (logNone *LoggerNone) Settings(mgr *settings.Settings)

Settings - Function to prepare settings for the logger

func (*LoggerNone) Status 

func (logNone *LoggerNone) Status(data []byte, environment, uuid string, debug bool)

Status - Function that sends JSON status logs to stdout

type LoggerS3 

type LoggerS3 struct {
	S3Config  osctrl_config.S3Logger
	AWSConfig aws.Config
	Client    *s3.Client
	Enabled   bool
	Debug     bool
}

LoggerS3 will be used to log data using S3

func CreateLoggerS3 

func CreateLoggerS3(s3Config *osctrl_config.S3Logger) (*LoggerS3, error)

CreateLoggerS3 to initialize the logger

func (*LoggerS3) Close added in v0.5.7 

func (logS3 *LoggerS3) Close() error

Close releases resources held by the S3 logger. The AWS SDK v2 s3.Client has no explicit Close; its HTTP transport pool is garbage-collected. This is a no-op kept for interface compliance.

func (*LoggerS3) Export added in v0.5.6 

func (logS3 *LoggerS3) Export(logType string, data []byte, params ExportParams) error

func (*LoggerS3) IsEnabled added in v0.5.6 

func (logS3 *LoggerS3) IsEnabled() bool

func (*LoggerS3) Name added in v0.5.6 

func (logS3 *LoggerS3) Name() string

func (*LoggerS3) Query added in v0.5.6 

func (logS3 *LoggerS3) Query(data []byte, environment, uuid, name string, status int, debug bool)

Query - Function that sends JSON on-demand query result logs to S3.

The S3 key embeds the query `name` as a path segment so the reader can list by query name with a prefix filter — without it, the reader would have to list every query object in the environment and decode each body to find the ones matching `name`, which is catastrophically slow on a busy bucket.

Key layout: {env}/query/{name}/{uuid}/{ts}.json

The body is the same QueryWriteData JSON the DB logger would have stored, so the reader can decode it back into OsqueryQueryData rows.

func (*LoggerS3) Send 

func (logS3 *LoggerS3) Send(logType string, data []byte, environment, uuid string, debug bool)

Send - Function that sends JSON logs to S3

func (*LoggerS3) Settings 

func (logS3 *LoggerS3) Settings(mgr *settings.Settings)

Settings - Function to prepare settings for the logger

type LoggerSplunk 

type LoggerSplunk struct {
	Configuration config.SplunkLogger
	Headers       map[string]string
	Enabled       bool
}

LoggerSplunk will be used to log data using Splunk

func CreateLoggerSplunk 

func CreateLoggerSplunk(cfg *config.SplunkLogger) (*LoggerSplunk, error)

CreateLoggerSplunk to initialize the logger

func (*LoggerSplunk) Close added in v0.5.7 

func (logSP *LoggerSplunk) Close() error

Close releases resources held by the Splunk logger. Splunk sends data via per-call HTTP POSTs using the shared utils.SendRequest helper, so there is no persistent client to close.

func (*LoggerSplunk) Export added in v0.5.6 

func (logSP *LoggerSplunk) Export(logType string, data []byte, params ExportParams) error

func (*LoggerSplunk) IsEnabled added in v0.5.6 

func (logSP *LoggerSplunk) IsEnabled() bool

func (*LoggerSplunk) Name added in v0.5.6 

func (logSP *LoggerSplunk) Name() string

func (*LoggerSplunk) Send 

func (logSP *LoggerSplunk) Send(logType string, data []byte, environment, uuid string, debug bool)

Send - Function that sends JSON logs to Splunk HTTP Event Collector

func (*LoggerSplunk) Settings 

func (logSP *LoggerSplunk) Settings(mgr *settings.Settings)

Settings - Function to prepare settings for the logger

type LoggerStdout 

type LoggerStdout struct {
	Enabled bool
}

LoggerStdout will be used to log data using stdout

func CreateLoggerStdout 

func CreateLoggerStdout() (*LoggerStdout, error)

CreateLoggerStdout to initialize the logger

func (*LoggerStdout) Close added in v0.5.7 

func (logStdout *LoggerStdout) Close() error

Close releases resources held by the stdout logger. None are held.

func (*LoggerStdout) Export added in v0.5.6 

func (logStdout *LoggerStdout) Export(logType string, data []byte, params ExportParams) error

func (*LoggerStdout) IsEnabled added in v0.5.6 

func (logStdout *LoggerStdout) IsEnabled() bool

func (*LoggerStdout) Log 

func (logStdout *LoggerStdout) Log(logType string, data []byte, environment, uuid string, debug bool)

Log - Function that sends JSON result/status/query logs to stdout

func (*LoggerStdout) Name added in v0.5.6 

func (logStdout *LoggerStdout) Name() string

func (*LoggerStdout) Query 

func (logStdout *LoggerStdout) Query(data []byte, environment, uuid, name string, status int, debug bool)

Query - Function that sends JSON query logs to stdout

func (*LoggerStdout) Result 

func (logStdout *LoggerStdout) Result(data []byte, environment, uuid string, debug bool)

Result - Function that sends JSON result logs to stdout

func (*LoggerStdout) Settings 

func (logStdout *LoggerStdout) Settings(mgr *settings.Settings)

Settings - Function to prepare settings for the logger

func (*LoggerStdout) Status 

func (logStdout *LoggerStdout) Status(data []byte, environment, uuid string, debug bool)

Status - Function that sends JSON status logs to stdout

type LoggerTLS 

type LoggerTLS struct {

	// Logging is kept for backwards-compatible diagnostics (the
	// comma-joined name list of the global set). Callers that previously
	// read l.Logging should migrate to ExportersFor(0).Name().
	Logging string
	Nodes   *nodes.NodeManager
	Queries *queries.Queries
	// Alerts is the ingest-path alert matcher. nil (or a nil interface
	// value stored here) disables alert evaluation entirely — every
	// hook site is a nil-check that returns immediately.
	Alerts AlertMatcher
	// contains filtered or unexported fields
}

LoggerTLS will be used to handle logging for the TLS endpoint.

It is environment-aware: ExportersFor(envID) returns the MultiExporter for that environment, falling back to the global (key 0) set when the environment has no sinks of its own. The exporter map is swapped atomically by ReplaceExporters during a hot reload so an in-flight request finishes against its already-resolved exporter pointer while new requests pick up the new sinks.

func CreateLoggerTLS 

func CreateLoggerTLS(cfg config.ServiceParameters, mgr *settings.Settings, nodes *nodes.NodeManager, queries *queries.Queries) (*LoggerTLS, error)

CreateLoggerTLS to instantiate a new logger for the TLS endpoint from YAML configuration. Retained for backwards compatibility and tests. New code should prefer CreateLoggerTLSWith.

func CreateLoggerTLSWith added in v0.5.7 

func CreateLoggerTLSWith(exporters map[uint]*MultiExporter, nodes *nodes.NodeManager, queries *queries.Queries) *LoggerTLS

CreateLoggerTLSWith constructs an env-aware LoggerTLS from a pre-built map of per-environment exporters. Key 0 is the global fallback. The map is used as-is (not copied); callers must not mutate it after handing it over.

func (*LoggerTLS) AllExporters added in v0.5.7 

func (logTLS *LoggerTLS) AllExporters() map[uint]*MultiExporter

AllExporters returns a snapshot of the entire exporter map. Used by the SinkStatsWriter to iterate every live MultiExporter and snapshot its per-sink atomic counters. The returned map is a shallow copy so the caller can iterate without holding the read lock.

func (*LoggerTLS) DispatchLogs 

func (l *LoggerTLS) DispatchLogs(data []byte, uuid, logType string, envID uint, environment string, metadata nodes.NodeMetadata, debug bool)

DispatchLogs - Helper to dispatch logs. envID selects the exporter set for this environment (with global fallback handled inside LoggerTLS.LogWithEnv). The string environment is forwarded as metadata for exporters that embed it (Splunk source type, DB row, etc.).

func (*LoggerTLS) DispatchQueries 

func (l *LoggerTLS) DispatchQueries(queryData types.QueryWriteData, node nodes.OsqueryNode, debug bool)

DispatchQueries - Helper to dispatch queries. Uses the node's EnvironmentID to select the env-scoped exporter set.

func (*LoggerTLS) ExportersFor added in v0.5.7 

func (logTLS *LoggerTLS) ExportersFor(envID uint) *MultiExporter

ExportersFor returns the MultiExporter for the given environment, falling back to the global (key 0) set when the environment has no exporters of its own. Returns nil when neither exists — callers must handle nil (Log/QueryLog do so by warning and dropping).

func (*LoggerTLS) Log 

func (logTLS *LoggerTLS) Log(logType string, data []byte, environment, uuid string, debug bool)

Log will send status/result logs via the configured method of logging.

func (*LoggerTLS) LogWithEnv added in v0.5.7 

func (logTLS *LoggerTLS) LogWithEnv(logType string, data []byte, envID uint, environment, uuid string, debug bool)

LogWithEnv is the env-scoped dispatch used by the TLS log handler. It resolves the exporter set for the given environment and forwards the payload. envID is the numeric environment ID (env.ID); the string environment name is carried alongside for exporter metadata.

func (*LoggerTLS) ProcessLogQueryResult 

func (l *LoggerTLS) ProcessLogQueryResult(queriesWrite types.QueryWriteRequest, envid uint, debug bool)

ProcessLogQueryResult - Helper to process on-demand query result logs

func (*LoggerTLS) ProcessLogs 

func (l *LoggerTLS) ProcessLogs(data json.RawMessage, logType string, envID uint, environment, ipaddress string, dataLen int, debug bool) []types.LogResultData

ProcessLogs processes and dispatches logs. Result entries are returned so callers can reuse the decoded batch for secondary consumers such as posture. envID selects the environment-scoped exporter set.

func (*LoggerTLS) QueryLog 

func (logTLS *LoggerTLS) QueryLog(logType string, data []byte, environment, uuid, name string, status int, debug bool)

QueryLog will send query result logs via the configured method of logging.

func (*LoggerTLS) QueryLogWithEnv added in v0.5.7 

func (logTLS *LoggerTLS) QueryLogWithEnv(logType string, data []byte, envID uint, environment, uuid, name string, status int, debug bool)

QueryLogWithEnv is the env-scoped dispatch for on-demand query results. See LogWithEnv for the envID semantics.

func (*LoggerTLS) ReplaceExporters added in v0.5.7 

func (logTLS *LoggerTLS) ReplaceExporters(newExporters map[uint]*MultiExporter)

ReplaceExporters atomically swaps the entire exporter map and closes every exporter in the old map. The swap happens under the write lock; any Export call that already resolved its MultiExporter pointer finishes against the old set, then the old set is closed.

In-flight logs may be dropped during the switch because the old exporters' resources (Kafka producer, DB pool) are closed immediately after the swap. Callers that need to drain should do so before calling ReplaceExporters.

type LogstashMessage 

type LogstashMessage struct {
	Time        int64       `json:"time"`
	LogType     string      `json:"log_type"`
	UUID        string      `json:"uuid"`
	Environment string      `json:"environment"`
	Data        interface{} `json:"data"`
}

LogstashMessage to handle log format to be sent to Logstash

type MultiExporter added in v0.5.6 

type MultiExporter struct {
	// contains filtered or unexported fields
}

MultiExporter fans out one osquery payload to multiple destinations. A failing exporter is logged and returned, but does not prevent later exporters from receiving the same payload. It also holds per-exporter SinkStats counters so the background stats writer can snapshot them and persist bytes/count to the log_sinks table without touching the hot path. Each exporter carries an optional category list; an exporter whose categories don't match the incoming logType is skipped.

func CreateExporters added in v0.5.6 

func CreateExporters(cfg config.ServiceParameters, mgr *settings.Settings) (*MultiExporter, error)

CreateExporters builds the composite exporter used by osctrl-tls.

func NewMultiExporter added in v0.5.6 

func NewMultiExporter(exporters ...DataExporter) *MultiExporter

NewMultiExporter creates a composite exporter from the provided destinations. The stats slice is empty — callers that want per-exporter counters should use NewMultiExporterWithStats.

func NewMultiExporterWithStats added in v0.5.7 

func NewMultiExporterWithStats(entries []ExporterEntry) *MultiExporter

NewMultiExporterWithStats creates a composite exporter where each exporter is wrapped in a CountedExporter linked to its SinkStats. The returned stats slice can be snapshotted by a background writer. Each entry's Categories are carried through so Export can skip exporters that don't match the incoming logType.

func (*MultiExporter) Close added in v0.5.7 

func (m *MultiExporter) Close() error

Close closes every contained exporter. Errors are collected and joined; one exporter failing to close does not stop the rest from being closed.

func (*MultiExporter) Export added in v0.5.6 

func (m *MultiExporter) Export(logType string, data []byte, params ExportParams) error

Export sends data to every enabled destination whose categories match the incoming logType. An exporter with empty categories receives all logTypes (backwards-compatible default).

func (*MultiExporter) ExporterNames added in v0.5.6 

func (m *MultiExporter) ExporterNames() []string

ExporterNames returns the configured destination names in order.

func (*MultiExporter) IsEnabled added in v0.5.6 

func (m *MultiExporter) IsEnabled() bool

IsEnabled returns true when at least one contained exporter is enabled.

func (*MultiExporter) Name added in v0.5.6 

func (m *MultiExporter) Name() string

Name returns a comma-separated list of destination names for diagnostics.

func (*MultiExporter) Stats added in v0.5.7 

func (m *MultiExporter) Stats() []*SinkStats

Stats returns the per-exporter SinkStats slice, or nil if the MultiExporter was created without stats tracking.

type OsqueryQueryData 

type OsqueryQueryData struct {
	gorm.Model
	UUID        string `gorm:"index"`
	Environment string
	Name        string
	Data        string
	Status      int
}

OsqueryQueryData to log query data to database

type OsqueryResultData 

type OsqueryResultData struct {
	gorm.Model
	UUID        string `gorm:"index"`
	Environment string
	Name        string
	Action      string
	Epoch       int64
	Columns     string
	Counter     int
}

OsqueryResultData to log result data to database

type OsqueryStatusData 

type OsqueryStatusData struct {
	gorm.Model
	UUID        string `gorm:"index"`
	Environment string
	Line        string
	Message     string
	Version     string
	Filename    string
	Severity    string
}

OsqueryStatusData to log status data to database

type SinkStats added in v0.5.7 

type SinkStats struct {
	SinkID      uint
	Exporter    DataExporter
	BytesSent   atomic.Int64
	ExportCount atomic.Int64
}

SinkStats holds atomic per-sink counters — bytes sent and number of export calls. They are incremented on the hot path (every Export) via lock-free atomic adds and snapshotted by a background writer that flushes them to the database periodically. The sink ID links the counters back to the log_sinks row they belong to.

type SplunkMessage 

type SplunkMessage struct {
	Time       int64       `json:"time"`
	Host       string      `json:"host"`
	Source     string      `json:"source"`
	SourceType string      `json:"sourcetype"`
	Index      string      `json:"index"`
	Event      interface{} `json:"event"`
}

SplunkMessage to handle log format to be sent to Splunk

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.