storage

package module

v0.2.0 Latest Latest Go to latest Published: Jun 26, 2026 License: Apache-2.0 Imports: 35 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/oteldb/storage

Links

Open Source Insights

README ¶

storage

Low-level, distributed, OpenTelemetry-centric storage library (Go) for signals:

Metrics
Logs
Traces
Profiles

Built as the storage tier for oteldb, an OpenTelemetry observability backend — embeddable as a native Go storage engine for all signals.

Currently in WIP and PoC. See ARCHITECTURE.md for current design.

Documentation ¶

Overview ¶

Package storage is the embeddable entry point for the oteldb storage library.

It is a low-level, distributed, OpenTelemetry-centric storage engine for signals (metrics, logs, traces, profiles), exposed as a small Storage facade. Everything else is an implementation detail behind this package. See the package layout and milestone plan in DESIGN.md at the repository root.

The in-memory engine (InMemory) is the default in tests: a full ingest+query path with WAL and durable flush disabled.

Index ¶

Variables
type Accepted
type Durability
type Option
type Options
type Storage
- func InMemory(opts ...Option) (*Storage, error)
- func Open(ctx context.Context, o Options, opts ...Option) (*Storage, error)

Constants ¶

This section is empty.

Variables ¶

View Source

var ErrClosed = errors.New("storage: closed")

ErrClosed is returned by Storage methods after Storage.Close.

View Source

var ErrNotEphemeral = errors.New("storage: reset requires an ephemeral backend")

ErrNotEphemeral is returned by Storage.Reset when the backend is durable: Reset wipes all ingested data and is only permitted on an ephemeral (in-memory) store.

Functions ¶

This section is empty.

Types ¶

type Accepted ¶

type Accepted struct {
	// Accepted is the number of accepted data points (metric points, log records,
	// span, profile samples depending on signal).
	Accepted int64
	// Rejected is the number of rejected data points (e.g. over a tenant limit or
	// out of the OOO window). Rejections are reported back to the producer via OTLP
	// partial_success so it can retry.
	Rejected int64
	// RejectedReason is a machine-readable reason for rejections (empty if none).
	RejectedReason string
}

Accepted carries per-OTLP partial-success counts (DESIGN.md §5). It implements the OTLP partial-success semantics: accepted/rejected data points + a reason string.

type Durability ¶

type Durability uint8

Durability is the WAL + flush policy (DESIGN.md §5, §8).

const (
	// DurabilityDefault is the default durability for a durable backend (WAL on,
	// flush to backend). For the in-memory engine use [DurabilityEphemeral].
	DurabilityDefault Durability = iota
	// DurabilityEphemeral disables the WAL and durable flush: the head answers all
	// queries and "flushed" parts live in RAM, dropped on [Storage.Close]. This is
	// the in-memory engine and the default in tests (DESIGN.md §5).
	DurabilityEphemeral
)

type Option ¶

type Option func(*Options)

Option is a functional option applied to Options by Open/InMemory.

func WithBackend ¶

func WithBackend(b backend.Backend) Option

WithBackend sets the storage backend.

func WithCluster ¶

func WithCluster(c *cluster.Config) Option

WithCluster sets the cluster configuration. nil ⇒ single-node.

func WithDurability ¶

func WithDurability(d Durability) Option

WithDurability sets the WAL + flush policy.

func WithEncoding ¶

func WithEncoding(p encoding.Profile) Option

WithEncoding sets the default encoding profile.

func WithFlushInterval ¶

func WithFlushInterval(ns int64) Option

WithFlushInterval sets the head flush time interval in nanoseconds.

func WithFlushThresholdBytes ¶

func WithFlushThresholdBytes(n int64) Option

WithFlushThresholdBytes sets the head flush size threshold.

func WithOOOWindow ¶

func WithOOOWindow(ns int64) Option

WithOOOWindow sets the out-of-order ingestion window in nanoseconds.

func WithQueryCache ¶ added in v0.2.0

func WithQueryCache(maxEntries int) Option

WithQueryCache enables a shared bounded-LRU results cache on Storage.Fetcher holding at most maxEntries results. maxEntries ≤ 0 disables the cache. Only fully-pushable equality requests over an explicit tenant set are cached, and entries do not auto-invalidate.

func WithQueryCacheFreshness ¶ added in v0.2.0

func WithQueryCacheFreshness(ns int64) Option

WithQueryCacheFreshness sets the results-cache recent-window guard in nanoseconds: a query whose window ends within ns of now is not cached, so only settled windows are memoized. ns ≤ 0 disables the guard. Has effect only with the cache enabled (see WithQueryCache).

func WithQuerySplitInterval ¶ added in v0.2.0

func WithQuerySplitInterval(ns int64) Option

WithQuerySplitInterval enables split-by-interval on Storage.Fetcher: a query window is divided into aligned sub-windows of ns nanoseconds, fetched concurrently and merged. ns ≤ 0 disables splitting.

func WithTenancy ¶

func WithTenancy(r tenant.Resolver) Option

WithTenancy sets the tenant policy resolver.

func WithTenant ¶

func WithTenant(fn func(signal.Resource, signal.Scope) signal.TenantID) Option

WithTenant sets the record→tenant routing callback (resource/scope → tenant id).

func WithWALDir ¶

func WithWALDir(dir string) Option

WithWALDir sets the WAL directory (file backend + durability).

type Options ¶

type Options struct {
	// Backend is the storage backend (memory/file/s3). If nil, [Open] defaults to
	// [backend.Memory] — the in-memory, ephemeral engine (DESIGN.md §5).
	Backend backend.Backend

	// Cluster is the cluster configuration. nil ⇒ single-node (the cluster layer
	// is absent, DESIGN.md §3, §11). Single-node users ignore L0 entirely.
	Cluster *cluster.Config

	// Tenant derives a record's tenant id from its Resource and Scope (so one OTLP
	// batch may fan out to many tenants). If nil, every record routes to "default".
	Tenant func(signal.Resource, signal.Scope) signal.TenantID

	// Tenancy resolves a tenant id to limits, retention, downsampling, and routing.
	// If nil, a permissive default resolver is used.
	Tenancy tenant.Resolver

	// Encoding is the default codec-chain profile per column kind (DESIGN.md §6).
	// If nil, a sensible default is used.
	Encoding encoding.Profile

	// Durability is the WAL + flush policy. [DurabilityEphemeral] disables both —
	// the in-memory engine: the head answers all queries and "flushed" parts are
	// kept in RAM and dropped on [Storage.Close] (DESIGN.md §5).
	Durability Durability

	// WALDir is the WAL directory for the file backend with durability enabled.
	// Ignored when [Durability] is [DurabilityEphemeral].
	WALDir string

	// FlushThresholdBytes is the head size at which a flush to an immutable part is
	// triggered. Zero ⇒ default.
	FlushThresholdBytes int64

	// FlushInterval is the max age of unflushed head data. Zero ⇒ default.
	// (Resolved into a duration internally to keep the API import-light.)
	FlushInterval int64 // nanoseconds

	// OOOWindow is the out-of-order ingestion window in nanoseconds (DESIGN.md §8).
	// Samples older than (newest - OOOWindow) are rejected (counted). Zero ⇒ default.
	OOOWindow int64 // nanoseconds

	// QuerySplitInterval, when > 0, makes [Storage.Fetcher] split a query's time window
	// into aligned sub-windows of this width (nanoseconds), fetched concurrently and merged
	// (the query-frontend "split by interval" technique). Zero ⇒ no splitting.
	QuerySplitInterval int64 // nanoseconds

	// QueryCacheEntries, when > 0, gives [Storage.Fetcher] a shared bounded-LRU results
	// cache of this many entries. Only fully-pushable (serializable-equality) requests over
	// an explicit tenant set are cached. Zero ⇒ no cache.
	QueryCacheEntries int

	// QueryCacheFreshness is the recent-window guard for the results cache, in nanoseconds: a
	// query whose window ends within this of now is not cached (new samples may still land in
	// it), so only settled/historical windows are memoized. 0 ⇒ no guard (cache every eligible
	// request). Ignored when QueryCacheEntries is 0.
	QueryCacheFreshness int64 // nanoseconds
}

Options configures a Storage. Use [WithX] functional options to override defaults; Open applies them. The zero value is not valid — call Open (or InMemory) to construct a Storage.

type Storage ¶

type Storage struct {
	// contains filtered or unexported fields
}

Storage is the embeddable entry point (DESIGN.md §5). It is safe for concurrent use. Construct with Open or InMemory; never with a literal.

All ingestion is push-based and OTLP-shaped: methods accept the library's internal, []byte-based, zero-alloc ingest batches (e.g. metric.Metrics) and return an Accepted carrying per-OTLP partial-success counts. OTel-Go users convert pdata to these via the optional otlp/pdataconv bridge, which keeps pdata off this hot path. Reads go through the language-agnostic Storage.Fetcher seam; query languages live in the embedder.

All four signals are wired end-to-end: metrics (Storage.WriteMetrics/Storage.Fetcher) on the float-sample engine, and logs, traces, and profiles (Storage.WriteLogs/Storage.WriteTraces/ Storage.WriteProfiles and their fetchers) on the shared record engine.

func InMemory ¶

func InMemory(opts ...Option) (*Storage, error)

InMemory returns a fully in-memory, ephemeral Storage (DESIGN.md §5): equivalent to Open with backend.Memory and DurabilityEphemeral. It is the default in tests — a full ingest+query path with no disk or object store.

func Open ¶

func Open(ctx context.Context, o Options, opts ...Option) (*Storage, error)

Open constructs a Storage from Options (DESIGN.md §5). If Options.Backend is nil it defaults to backend.Memory; if the backend is ephemeral and no durability is chosen, it defaults to DurabilityEphemeral (the in-memory engine).

func (*Storage) Close ¶

func (s *Storage) Close(ctx context.Context) error

Close releases all resources. It is idempotent. After [Close], every method on s returns ErrClosed.

func (*Storage) Fetcher ¶

func (s *Storage) Fetcher(tenants ...signal.TenantID) fetch.Fetcher

Fetcher returns the read seam — a fetch.Fetcher over the named tenants' data (head ∪ flushed parts). It is the storage library's query surface: this is a language-agnostic columnar store, so the embedder drives its own query engines (PromQL/LogQL/TraceQL) over the fetch contract. The optional query/promql package bridges this to the Prometheus storage.Queryable for embedders using the Prometheus engine.

Tenant scoping:

Fetcher(t) — one tenant.
Fetcher(a, b, …) — a fan-out over several tenants (multi-tenant query): results are merged by series id, so a series with equal labels in more than one tenant is federated into one (see fetch.Merge). Add a tenant label upstream if per-tenant separation is wanted.
Fetcher() — all tenants that have ingested data (a cross-tenant query).

The returned Fetcher is always usable: with no matching tenant (or after [Close]) it is an empty fetcher that yields no series, so callers need not special-case "no data yet".

Scale-out: when Options.QuerySplitInterval and/or Options.QueryCacheEntries are set, the returned fetcher is wrapped with split-by-interval and/or a results cache (the query/scale decorators). The cache keys on the explicit tenant set, so it applies only to named-tenant queries — a no-arg cross-tenant query is never cached (its tenant membership is dynamic).

func (*Storage) LogFetcher ¶ added in v0.2.0

func (s *Storage) LogFetcher(tenants ...signal.TenantID) fetch.Fetcher

LogFetcher returns the read seam for logs — a fetch.Fetcher over the named tenants' log data (head ∪ flushed parts). Like Storage.Fetcher it scopes by tenant: one, several (concatenated), or none ⇒ all tenants with log data. Always usable: an empty fetcher when no tenant matches or after [Close]. Label matchers resolve streams; column Conditions filter records.

func (*Storage) ProfileFetcher ¶ added in v0.2.0

func (s *Storage) ProfileFetcher(tenants ...signal.TenantID) fetch.Fetcher

ProfileFetcher returns the read seam for profiles — a fetch.Fetcher over the named tenants' sample data. Label matchers resolve streams (service plus the profile type, carried in reserved `otel.profile.*` labels); column Conditions filter samples (value, profile id, attributes). Returned rows carry the global content-addressed `stack_id`; resolve it with Storage.ProfileResolver. Same tenant scoping as Storage.TraceFetcher.

func (*Storage) ProfileResolver ¶ added in v0.2.0

func (s *Storage) ProfileResolver(ctx context.Context, tenant signal.TenantID) (*profile.Resolver, error)

ProfileResolver returns a symbol resolver over a tenant's profile symbol store (the unioned head + part sidecars), so an embedder resolves the content-addressed `stack_id` column of a sample fetch to function frames and builds a flamegraph. An unknown tenant yields an empty resolver (every stack resolves to no frames), so callers need not special-case "no data". Resolution is correct across the cluster — `stack_id`s are global content ids — but reads the local node's symbol store; replicating the store itself is deferred (see `signal/profile`).

func (*Storage) ProfileSeries ¶ added in v0.2.0

func (s *Storage) ProfileSeries(
	ctx context.Context, tenant signal.TenantID, matchers []fetch.Matcher, start, end int64,
) ([]signal.Series, error)

ProfileSeries returns the identities of a tenant's profile streams matching the label matchers within [start, end] (zero start AND end disables the time filter). It is the enumeration primitive an embedder uses to build the Pyroscope-style ProfileTypes / LabelNames / LabelValues responses: the profile type is carried in each series' reserved `otel.profile.*` labels (see `signal/profile`), and the user labels are the resource/scope attributes. Local to this node — cluster fan-out for enumeration is not yet wired (the sample read path already fans out).

func (*Storage) Reset ¶

func (s *Storage) Reset(ctx context.Context) error

Reset discards all ingested data, returning every tenant engine to empty (head and flushed parts) without reconstructing the Storage. It is intended for tests and benchmarks that reuse one store across runs instead of rebuilding it. Reset is only valid on an ephemeral (in-memory) backend; on a durable backend it returns ErrNotEphemeral rather than wipe persisted data. The tenant engines themselves are retained (emptied, not dropped), so subsequent writes reuse them.

func (*Storage) Trace ¶ added in v0.2.0

func (s *Storage) Trace(ctx context.Context, tenant signal.TenantID, traceID []byte) ([]*fetch.Batch, error)

Trace fetches every span of one trace from a tenant by trace id: an equality condition on the trace_id column, pruned by its per-part equality bloom (trace-by-id). It returns one batch per stream (service) carrying the trace's spans — including the nested-set columns the embedder's TraceQL uses for structural operators.

func (*Storage) TraceFetcher ¶ added in v0.2.0

func (s *Storage) TraceFetcher(tenants ...signal.TenantID) fetch.Fetcher

TraceFetcher returns the read seam for traces — a fetch.Fetcher over the named tenants' span data. Label matchers resolve streams (services); column Conditions filter spans (name, kind, status, duration, attributes). Same tenant scoping as Storage.LogFetcher.

func (*Storage) WriteLogs ¶

func (s *Storage) WriteLogs(ctx context.Context, ld log.Logs) (Accepted, error)

WriteLogs ingests a logs batch. It projects the internal model, derives each record's tenant from its Resource+Scope, and appends to that tenant's logs engine (indexing stream labels + buffering records). Returns per-OTLP partial-success counts: rejected counts out-of-order drops.

func (*Storage) WriteMetrics ¶

func (s *Storage) WriteMetrics(ctx context.Context, md metric.Metrics) (Accepted, error)

WriteMetrics ingests a metrics batch. It projects the internal model, derives each point's tenant from its Resource+Scope, and appends to that tenant's engine (indexing labels + buffering samples). Returns per-OTLP partial-success counts: rejected counts out-of-order drops. (Unsupported point kinds and value-less points never reach here: they are filtered and counted by the producer — e.g. the otlp/pdataconv bridge.)

func (*Storage) WriteProfiles ¶

func (s *Storage) WriteProfiles(ctx context.Context, pd profile.Profiles) (Accepted, error)

WriteProfiles ingests a profiles batch. It projects each sample into a record row (flattening timestamped samples and denormalizing profile fields) and a content-addressed symbol delta, derives each sample's tenant from its Resource+Scope, and appends to that tenant's profiles engine — which persists the symbol store as part sidecars. Returns OTLP partial-success counts.

func (*Storage) WriteTraces ¶

func (s *Storage) WriteTraces(ctx context.Context, td trace.Traces) (Accepted, error)

WriteTraces ingests a traces batch. It projects the span model (computing nested-set ids and serializing events/links), derives each span's tenant from its Resource+Scope, and appends to that tenant's traces engine. Returns per-OTLP partial-success counts (out-of-order drops).

Source Files ¶

View all Source files

Directories ¶

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

Path	Synopsis
backend Package backend defines the L1 storage seam (DESIGN.md §3, §5): a common Backend interface (Read/Write/List/Delete over whole-object keys) with interchangeable implementations.	Package backend defines the L1 storage seam (DESIGN.md §3, §5): a common Backend interface (Read/Write/List/Delete over whole-object keys) with interchangeable implementations.
backendtest Package backendtest provides a shared conformance suite that every backend.Backend implementation must pass, proving the implementations are interchangeable (DESIGN.md §2: "backends are interchangeable behind backend.Backend").	Package backendtest provides a shared conformance suite that every backend.Backend implementation must pass, proving the implementations are interchangeable (DESIGN.md §2: "backends are interchangeable behind backend.Backend").
bucketindex Package bucketindex maintains a compact, incremental index of the immutable parts under a key prefix, so a stateless reader enumerates a tenant's parts (and prunes them by time) from a single object instead of a full, expensive bucket LIST (DESIGN.md §11, the object-store-native read path).	Package bucketindex maintains a compact, incremental index of the immutable parts under a key prefix, so a stateless reader enumerates a tenant's parts (and prunes them by time) from a single object instead of a full, expensive bucket LIST (DESIGN.md §11, the object-store-native read path).
file Package file implements a backend.Backend over a local directory tree.	Package file implements a backend.Backend over a local directory tree.
s3 Package s3 implements a backend.Backend over an S3-compatible object store.	Package s3 implements a backend.Backend over an S3-compatible object store.
block Package block implements the L2 immutable columnar part format (DESIGN.md §3, §14 M1): per-column streams with min/max stats and constant-column collapse, a sparse granule mark index, and an atomic manifest written last.	Package block implements the L2 immutable columnar part format (DESIGN.md §3, §14 M1): per-column streams with min/max stats and constant-column collapse, a sparse granule mark index, and an atomic manifest written last.
cluster Package cluster implements the L0 distribution layer (DESIGN.md §3, §11, §14 M6–M7): rendezvous (HRW) hashing with spread-minimizing tokens, etcd-backed ring state and leases, RF=3 quorum replication, and rebalancing.	Package cluster implements the L0 distribution layer (DESIGN.md §3, §11, §14 M6–M7): rendezvous (HRW) hashing with spread-minimizing tokens, etcd-backed ring state and leases, RF=3 quorum replication, and rebalancing.
etcd Package etcd backs the L0 cluster ring with etcd: a node registers itself under a lease and watches the member set, so membership is live and self-healing — a crashed node's lease expires and it drops out of every other node's ring within the TTL, with no manual deregistration.	Package etcd backs the L0 cluster ring with etcd: a node registers itself under a lease and watches the member set, so membership is live and self-healing — a crashed node's lease expires and it drops out of every other node's ring within the TTL, with no manual deregistration.
rebalance Package rebalance computes the minimal ownership changes to apply a cluster membership change (DESIGN.md §11).	Package rebalance computes the minimal ownership changes to apply a cluster membership change (DESIGN.md §11).
replica Package replica is the L0 write-replication layer: it fans an opaque write payload out to the ring-owners of a key and returns once a write quorum has durably applied it, so the unflushed head survives the loss of a minority of replicas (DESIGN.md §11; RF=3, quorum (RF/2)+1=2).	Package replica is the L0 write-replication layer: it fans an opaque write payload out to the ring-owners of a key and returns once a write quorum has durably applied it, so the unflushed head survives the loss of a minority of replicas (DESIGN.md §11; RF=3, quorum (RF/2)+1=2).
ring Package ring implements rendezvous (highest-random-weight, HRW) hashing — the L0 sharding primitive (DESIGN.md §11).	Package ring implements rendezvous (highest-random-weight, HRW) hashing — the L0 sharding primitive (DESIGN.md §11).
encoding Package encoding holds the L2 codec layers (DESIGN.md §3, §14 M0):	Package encoding holds the L2 codec layers (DESIGN.md §3, §14 M0):
bitstream Package bitstream implements a zero-alloc, MSB-first bit stream reader and writer.	Package bitstream implements a zero-alloc, MSB-first bit stream reader and writer.
chunk Package chunk implements the zero-alloc value-column codecs (DESIGN.md §14 M0): delta-of-delta timestamps, Gorilla XOR float gauges, T64, dictionary, and scaled-decimal+nearest-delta.	Package chunk implements the zero-alloc value-column codecs (DESIGN.md §14 M0): delta-of-delta timestamps, Gorilla XOR float gauges, T64, dictionary, and scaled-decimal+nearest-delta.
compress Package compress wraps zstd/lz4 block compression and defines the composable codec-chain type (preprocessor → general compressor) used to reach 0.4–0.8 bytes/point (DESIGN.md §3.3, §14 M0).	Package compress wraps zstd/lz4 block compression and defines the composable codec-chain type (preprocessor → general compressor) used to reach 0.4–0.8 bytes/point (DESIGN.md §3.3, §14 M0).
engine Package engine is the L3 single-node storage engine: an in-memory head that absorbs writes (indexing labels and buffering samples) with a bounded out-of-order window, optionally backed by a write-ahead log for crash recovery.	Package engine is the L3 single-node storage engine: an in-memory head that absorbs writes (indexing labels and buffering samples) with a bounded out-of-order window, optionally backed by a write-ahead log for crash recovery.
index Package index implements the L3 indexing layer (DESIGN.md §3, §14 M2):	Package index implements the L3 indexing layer (DESIGN.md §3, §14 M2):
bloom Package bloom is a token bloom filter for full-text pruning: a compact, approximate set of the terms present in a column block (e.g.	Package bloom is a token bloom filter for full-text pruning: a compact, approximate set of the terms present in a column block (e.g.
postings Package postings is the inverted index: for each (name, value) attribute — identified by interned symbol ids, not strings — it keeps the sorted list of [signal.SeriesID]s that carry it, and composes those lists with lazy set-op iterators (Intersect/Merge/Without) to resolve label matchers to series.	Package postings is the inverted index: for each (name, value) attribute — identified by interned symbol ids, not strings — it keeps the sorted list of [signal.SeriesID]s that carry it, and composes those lists with lazy set-op iterators (Intersect/Merge/Without) to resolve label matchers to series.
series Package series is the series-identity index: it maps a content-addressed signal.SeriesID to its full identity (signal.Series — Resource + Scope + data-point attributes) and back.	Package series is the series-identity index: it maps a content-addressed signal.SeriesID to its full identity (signal.Series — Resource + Scope + data-point attributes) and back.
symbols Package symbols is the string-interning symbol table.	Package symbols is the string-interning symbol table.
internal
cmd/fuzz command Command fuzz starts a fuzzing campaign for a randomly chosen fuzz target using gosentry (https://github.com/trailofbits/gosentry), Trail of Bits' Go-toolchain fork that runs `go test -fuzz` on top of LibAFL.	Command fuzz starts a fuzzing campaign for a randomly chosen fuzz target using gosentry (https://github.com/trailofbits/gosentry), Trail of Bits' Go-toolchain fork that runs `go test -fuzz` on top of LibAFL.
otlp
pdataconv Package pdataconv is the optional OTel-Go bridge: it converts the collector pdata metrics type (pmetric.Metrics) into the storage library's internal, []byte-based metric.Metrics ingest batch.	Package pdataconv is the optional OTel-Go bridge: it converts the collector pdata metrics type (pmetric.Metrics) into the storage library's internal, []byte-based metric.Metrics ingest batch.
pool Package pool provides cross-cutting sync.Pool helpers, capacity-bucketed byte pools, and arena allocation for same-lifetime batches (DESIGN.md §10).	Package pool provides cross-cutting sync.Pool helpers, capacity-bucketed byte pools, and arena allocation for same-lifetime batches (DESIGN.md §10).
query Package query groups the storage library's read seam and its language adapters.	Package query groups the storage library's read seam and its language adapters.
fetch Package fetch is the storage seam: the contract every query language compiles to and every data source (head, parts, cluster fan-out) implements.	Package fetch is the storage seam: the contract every query language compiles to and every data source (head, parts, cluster fan-out) implements.
promql Package promql is an optional adapter that bridges the storage fetch contract (query/fetch) to the Prometheus storage.Queryable interface.	Package promql is an optional adapter that bridges the storage fetch contract (query/fetch) to the Prometheus storage.Queryable interface.
scale Package scale provides fetch-seam scale-out primitives: [Fetcher → Fetcher] decorators that any embedder's query engine composes over fetch.Fetcher (a per-tenant engine, a cluster fan-out, a cross-tenant fetch.Merge) without the library owning a query language.	Package scale provides fetch-seam scale-out primitives: [Fetcher → Fetcher] decorators that any embedder's query engine composes over fetch.Fetcher (a per-tenant engine, a cluster fan-out, a cross-tenant fetch.Merge) without the library owning a query language.
recordengine Package recordengine is the shared storage engine for record-shaped signals — logs and traces (later, profiles' sample table).	Package recordengine is the shared storage engine for record-shaped signals — logs and traces (later, profiles' sample table).
signal Package signal holds the OTel data model, signal-neutral types shared across metrics, logs, traces, and profiles: the Signal enum, TenantID, and the typed attribute identity primitives (Value, KeyValue, Attributes, SeriesID).	Package signal holds the OTel data model, signal-neutral types shared across metrics, logs, traces, and profiles: the Signal enum, TenantID, and the typed attribute identity primitives (Value, KeyValue, Attributes, SeriesID).
log Package log holds the logs signal's ingest model: the []byte-based, OTLP-shaped, zero-alloc batch accepted at the storage boundary (in place of OTel-Go plog.Logs), and its projection into the columnar log-record model the logs engine ingests.	Package log holds the logs signal's ingest model: the []byte-based, OTLP-shaped, zero-alloc batch accepted at the storage boundary (in place of OTel-Go plog.Logs), and its projection into the columnar log-record model the logs engine ingests.
metric Package metric implements the metrics signal: OTLP metric point types, temporality, series identity, and the OTLP → internal projection.	Package metric implements the metrics signal: OTLP metric point types, temporality, series identity, and the OTLP → internal projection.
profile Package profile holds the profiles signal's ingest model: the []byte-based, OTLP-shaped batch accepted at the storage boundary (in place of OTel-Go pprofile.Profiles), and its projection into the columnar sample model the record engine ingests plus the content-addressed symbol store.	Package profile holds the profiles signal's ingest model: the []byte-based, OTLP-shaped batch accepted at the storage boundary (in place of OTel-Go pprofile.Profiles), and its projection into the columnar sample model the record engine ingests plus the content-addressed symbol store.
trace Package trace holds the traces signal's ingest model: the []byte-based, OTLP-shaped, zero-alloc batch accepted at the storage boundary (in place of OTel-Go ptrace.Traces), and its projection into the columnar span model the record engine ingests.	Package trace holds the traces signal's ingest model: the []byte-based, OTLP-shaped, zero-alloc batch accepted at the storage boundary (in place of OTel-Go ptrace.Traces), and its projection into the columnar span model the record engine ingests.
tenant Package tenant provides the cross-cutting policy-callback layer (DESIGN.md §2, §8): a tenant.Resolver maps a tenant id to limits, retention, downsampling, and routing, hot-reloadable by the consumer.	Package tenant provides the cross-cutting policy-callback layer (DESIGN.md §2, §8): a tenant.Resolver maps a tenant id to limits, retention, downsampling, and routing, hot-reloadable by the consumer.
wal Package wal is the write-ahead log: CRC-framed records appended to numbered segment files, replayed in order to reconstruct the in-memory index after a crash.	Package wal is the write-ahead log: CRC-framed records appended to numbered segment files, replayed in order to reconstruct the in-memory index after a crash.