ingest

package
v0.15.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 2, 2026 License: AGPL-3.0 Imports: 19 Imported by: 0

Documentation

Overview

Package ingest handles Loki push requests, validating and writing them to ClickHouse via the chstore.Writer.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewRateLimiter

func NewRateLimiter(limits *validation.Overrides) *limiter.RateLimiter

NewRateLimiter returns Loki's per-tenant ingestion rate limiter (ingestion_rate_mb, ingestion_burst_size_mb), recalculated every 10s like the distributor's.

Types

type Handler

type Handler struct {
	Writer      *chstore.Writer
	Limits      *validation.Overrides
	Logger      log.Logger
	RateLimiter *limiter.RateLimiter

	// MaxRecvMsgSize and MaxDecompressedSize bound a push body; zero selects
	// the package defaults.
	MaxRecvMsgSize      int
	MaxDecompressedSize int64
}

Handler handles Loki push requests and writes them to ClickHouse. RateLimiter is nil when the rate limit is disabled.

func (*Handler) ServeHTTP

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request)

Directories

Path Synopsis
Copied from grafana/loki@39c256e5 pkg/distributor/distributor.go (processStreamEntries, normalizeMetadataGroup, normalizeStructuredMetadata, parseStreamLabels, truncateLines, missingEnforcedLabels, metadataGroup, newMetadataGroup, duplicateTimestampTracker and the push request loop); modifications: methods turned into functions on a small preparer; rate limiting, writeFailuresManager, metrics, usage tracker, ingester/ring and Kafka paths removed; labelCache inlined; processStreamEntries takes a context; Prepare returns the kept bytes/lines for rate limiting (Loki's rlBuckets accounting, tenant bucket only); ReportDiscarded records the rate-limited discard metrics.
Copied from grafana/loki@39c256e5 pkg/distributor/distributor.go (processStreamEntries, normalizeMetadataGroup, normalizeStructuredMetadata, parseStreamLabels, truncateLines, missingEnforcedLabels, metadataGroup, newMetadataGroup, duplicateTimestampTracker and the push request loop); modifications: methods turned into functions on a small preparer; rate limiting, writeFailuresManager, metrics, usage tracker, ingester/ring and Kafka paths removed; labelCache inlined; processStreamEntries takes a context; Prepare returns the kept bytes/lines for rate limiting (Loki's rlBuckets accounting, tenant bucket only); ReportDiscarded records the rate-limited discard metrics.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL