identity

Overview ¶

Package identity is the on-disk form of a part's identity set: the signal.Series of every series the part holds, encoded against a symbol table private to the object.

It is what makes identity **part-scoped**. A whole-set identity object at the engine prefix has to be rewritten whenever the set changes and never shrinks with the data, so identities outlive the samples that named them and a node's resident index spans every series it has ever seen. Scoping identity to the part that holds it makes retention self-cleaning (deleting a part deletes its identities), makes a flush cost what it added rather than what the tenant has, and gives every node — owner or replica — a live identity set derived from its own parts. Prometheus' block index and VictoriaMetrics' per-partition indexDB have the same shape; the duplication of an identity across the parts that share it is the accepted price, folded back by merges.

Layout ¶

[magic "OTID" <4b>][version <1b>]
  section bytes …
[TOC: uvarint count, then per section: kind <1b>, uvarint offset, uvarint length, CRC32C <4b>]
[TOC offset <4b BE>][CRC32C of the TOC <4b>]

Sections are addressed by the trailing table of contents, and the TOC's own offset is the last fixed-width field, so a reader can find it from the object's tail alone — a section can be read by byte range without fetching the whole object, and a section kind added later without breaking a reader (unknown kinds are skipped). Today there are two: the symbol table and the series records.

The object is deliberately **not compressed**: interning already removes the repetition a compressor would find (measured ~3.8× on a churn-shaped set), and a compressed body would defeat the range-addressability the layout exists to preserve.