Documentation
¶
Overview ¶
Package log holds the logs signal's ingest model: the []byte-based, OTLP-shaped, zero-alloc batch accepted at the storage boundary (in place of OTel-Go plog.Logs), and its projection into the columnar log-record model the logs engine ingests.
It mirrors signal/metric: a resettable, pool-friendly Resource→Scope→Record hierarchy holding all identity as []byte (never string), so an embedder decoding OTLP can build it by aliasing the decode buffer, and projecting copies nothing. The difference from metrics is the data shape: a metric is a stream of (ts, float) samples, a log is a stream of rich records (time, severity, body, attributes, trace context). Identity (the stream labels: Resource+Scope) is indexed like a metric series; the per-record fields are columns the query filters by condition.
Index ¶
Constants ¶
const ( ColObserved = "observed" ColSeverity = "severity" ColFlags = "flags" ColDropped = "dropped" ColSeverityText = "severity_text" ColBody = "body" ColTraceID = "trace_id" ColSpanID = "span_id" ColAttrs = "attrs" )
Column names of the logs schema (the per-record columns; the primary timestamp and the stream id are implicit in the record engine).
Variables ¶
var Schema = recordengine.NewSchema( recordengine.Column{Name: ColObserved, Kind: recordengine.KindInt64, Codec: chunk.CodecT64}, recordengine.Column{Name: ColSeverity, Kind: recordengine.KindInt64, Codec: chunk.CodecT64}, recordengine.Column{Name: ColFlags, Kind: recordengine.KindInt64, Codec: chunk.CodecT64}, recordengine.Column{Name: ColDropped, Kind: recordengine.KindInt64, Codec: chunk.CodecT64}, recordengine.Column{Name: ColSeverityText, Kind: recordengine.KindBytes, Codec: chunk.CodecDict}, recordengine.Column{Name: ColBody, Kind: recordengine.KindBytes, Codec: chunk.CodecDict, Bloom: recordengine.BloomFullText}, recordengine.Column{Name: ColTraceID, Kind: recordengine.KindBytes, Codec: chunk.CodecDict}, recordengine.Column{Name: ColSpanID, Kind: recordengine.KindBytes, Codec: chunk.CodecDict}, recordengine.Column{Name: ColAttrs, Kind: recordengine.KindBytes, Codec: chunk.CodecDict, Bloom: recordengine.BloomAttrs}, )
Schema is the logs vertical's record-engine column schema: four small int columns, the body (full-text bloom), the trace/span ids, severity text, and the serialized per-record attributes (attribute bloom). The primary timestamp (the record's event time) is the implicit sort key.
Functions ¶
func Project ¶ added in v0.2.0
func Project(ld Logs, emit func(*recordengine.Batch)) (accepted int)
Project iterates a Logs batch and calls emit once per stream (each Resource+Scope group) with a recordengine.Batch of that stream's records laid out in the logs Schema's column order. It returns how many records were emitted. The batch and its column buffers are reused across emit calls — do not retain them.
Types ¶
type Logs ¶
type Logs struct {
Resources []ResourceLogs
}
Logs is the internal logs ingest batch — the OTLP Resource→Scope→Record hierarchy with all identity as []byte. It is resettable and pool-friendly: Logs.Reset keeps every backing array, so a batch from GetLogs returned with PutLogs recycles across ingest calls with no allocation. Build it with the Add* helpers, which reuse the retained capacity.
func GetLogs ¶ added in v0.2.0
func GetLogs() *Logs
GetLogs returns a reset Logs from a shared pool. Pair it with PutLogs to recycle the batch (and its backing arrays) across ingest calls.
func (*Logs) AddResource ¶ added in v0.2.0
func (l *Logs) AddResource() *ResourceLogs
AddResource appends a fresh ResourceLogs and returns a pointer to it. The slot's Resource is zeroed and its Scopes emptied (capacity retained).
type Record ¶ added in v0.2.0
type Record struct {
// Attributes are the record's attributes. They must be sorted by key (use
// [signal.NewAttributes]); the engine serializes them in order into the attrs column.
Attributes signal.Attributes
Timestamp int64 // unix nanos; the record's event time (the part sort key)
ObservedTimestamp int64 // unix nanos; when the collector observed it (0 if unset)
SeverityNumber int32 // OTLP severity number (1..24; 0 unspecified)
SeverityText []byte
Body []byte
TraceID []byte // 16 bytes, or nil
SpanID []byte // 8 bytes, or nil
Flags uint32
Dropped uint32 // dropped_attributes_count
}
Record is a single OTLP log record. Body is the record body's text (a string AnyValue held as raw bytes; non-string bodies are rendered to text by the producer). TraceID/SpanID are the raw 16- and 8-byte ids (nil if unset). A record present in a Logs batch is well-formed by construction.
type ResourceLogs ¶ added in v0.2.0
ResourceLogs groups the log records emitted under one signal.Resource.
func (*ResourceLogs) AddScope ¶ added in v0.2.0
func (rl *ResourceLogs) AddScope() *ScopeLogs
AddScope appends a fresh ScopeLogs under the resource and returns a pointer to it. The slot's Scope is zeroed and its Records emptied (capacity retained).
type ScopeLogs ¶ added in v0.2.0
ScopeLogs groups the log records emitted under one signal.Scope (InstrumentationScope). A (Resource, Scope) pair is one log **stream**.
Source Files