inserter

package
v1.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 28, 2025 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrInsertTimeout error = errors.New("insert timed out")

ErrInsertTimeout is returned by Inserter.Insert.

Functions

func TruncateToStartOfDayUTC

func TruncateToStartOfDayUTC(t time.Time) time.Time

func TruncateToStartOfMinuteUTC

func TruncateToStartOfMinuteUTC(t time.Time) time.Time

Types

type ClickHouseOptions

type ClickHouseOptions struct {
	// Database name.
	Database string
	// Whether to enable this connection.
	Enabled bool
	// ClickHouse endpoint.
	Endpoint string
	// ClickHouse username.
	Username string
	// ClickHouse password.
	Password string

	// Timeout for dialing a new connection.
	DialTimeout time.Duration
	// Max idle connections pooled.
	MaxIdleConnections int
	// Batch size.
	BatchSize int
	// How soon the batch is sent if it's incomplete.
	BatchSendTimeout time.Duration
	// wait_for_async_insert setting
	WaitForAsyncInsert bool

	// Skip ClickHouse ping on startup.
	SkipPing bool
	// Disable TLS on ClickHouse connection.
	DisableTLS bool
}

ClickHouseOptions are for configuring the ClickHouse connection.

type Inserter

type Inserter struct {
	// contains filtered or unexported fields
}

Inserter implements InserterInterface.

func NewInserter

func NewInserter(clickhouseOptions ClickHouseOptions, runtimeInfo RuntimeInfo, numWorkers int) (*Inserter, error)

NewInserter creates a new Inserter.

func (*Inserter) Queue

func (inserter *Inserter) Queue(observation Observation) error

Queue sends the observation to the first available worker for batching. It does not block unless the insertion queue is filled up, which is unlikely and indicates we have a problem. It returns ErrInsertTimeout if the queue did not free up in ErrInsertTimeout.

type InserterInterface

type InserterInterface interface {
	Queue(observation Observation) error
}

InserterInterface inserts network observation into ClickHouse.

type Observation

type Observation struct {
	Flow      labeler.FlowData
	Timestamp time.Time
}

Observation is a conntrack observation from kubenetmon-agent labeled by the labeler.

type RuntimeInfo

type RuntimeInfo struct {
	Cloud   labeler.Cloud
	Env     labeler.Environment
	Region  string
	Cluster string
}

RuntimeInfo has some basic information about where this inserter is running. This observation gets inserted into ClickHouse as local endpoint annotations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL