cockroachkvs

package
v2.1.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 21, 2026 License: BSD-3-Clause Imports: 21 Imported by: 0

Documentation

Overview

Package cockroachkvs provides CockroachDB's key-value schema and comparer implementation.

Index

Constants

View Source 
const MaxSuffixLen = 1 + max(
	engineKeyVersionLockTableLen,
	engineKeyVersionWallLogicalAndSyntheticTimeLen,
	engineKeyVersionWallAndLogicalTimeLen,
	engineKeyVersionWallTimeLen,
)

MaxSuffixLen is the maximum length of the CockroachDB key suffix.

Variables

View Source 
var BlockPropertyCollectors = []func() pebble.BlockPropertyCollector{
	func() pebble.BlockPropertyCollector {
		return sstable.NewBlockIntervalCollector(
			mvccWallTimeIntervalCollector,
			pebbleIntervalMapper{},
			MVCCBlockIntervalSuffixReplacer{},
		)
	},
}

BlockPropertyCollectors is a list of constructors for block-property collectors used by CockroachDB.

View Source 
var Comparer = base.Comparer{
	Split:                Split,
	ComparePointSuffixes: ComparePointSuffixes,
	CompareRangeSuffixes: CompareRangeSuffixes,
	Compare:              Compare,
	Equal:                Equal,

	AbbreviatedKey: func(k []byte) uint64 {
		key, ok := getKeyPartFromEngineKey(k)
		if !ok {
			return 0
		}
		return base.DefaultComparer.AbbreviatedKey(key)
	},

	FormatKey: FormatKey,

	Separator: func(dst, a, b []byte) []byte {
		if len(a) == 0 || len(b) == 0 {
			panic(errors.AssertionFailedf("cannot separate empty keys"))
		}
		aKey, ok := getKeyPartFromEngineKey(a)
		if !ok {
			return append(dst, a...)
		}
		bKey, ok := getKeyPartFromEngineKey(b)
		if !ok {
			return append(dst, a...)
		}

		if bytes.Equal(aKey, bKey) || len(aKey) == 0 || len(bKey) == 0 {
			return append(dst, a...)
		}
		n := len(dst)
		dst = base.DefaultComparer.Separator(dst, aKey, bKey)

		buf := dst[n:]
		if bytes.Equal(aKey, buf) {
			return append(dst[:n], a...)
		}

		return append(dst, 0)
	},

	Successor: func(dst, a []byte) []byte {

		if len(a) == 0 {
			return append(dst, 0x00)
		}
		aKey, ok := getKeyPartFromEngineKey(a)
		if !ok {
			return append(dst, a...)
		}
		n := len(dst)

		dst = base.DefaultComparer.Successor(dst, aKey)

		buf := dst[n:]
		if bytes.Equal(aKey, buf) {
			return append(dst[:n], a...)
		}

		return append(dst, 0)
	},

	ImmediateSuccessor: func(dst, a []byte) []byte {

		return append(append(dst, a...), 0)
	},

	ValidateKey: validateEngineKey,

	Name: "cockroach_comparator",
}

Comparer is a base.Comparer for CockroachDB keys.

View Source 
var KeySchema = colblk.KeySchema{
	Name:       "crdb1",
	HeaderSize: 1,
	ColumnTypes: []colblk.DataType{
		// contains filtered or unexported fields
	},
	NewKeyWriter: func() colblk.KeyWriter {
		return makeCockroachKeyWriter()
	},
	InitKeySeekerMetadata: func(meta *colblk.KeySeekerMetadata, d *colblk.DataBlockDecoder) {
		ks := (*cockroachKeySeeker)(unsafe.Pointer(meta))
		ks.init(d)
	},
	KeySeeker: func(meta *colblk.KeySeekerMetadata) colblk.KeySeeker {
		return (*cockroachKeySeeker)(unsafe.Pointer(meta))
	},
}

Functions

func AppendTimestamp 

func AppendTimestamp(key []byte, walltime uint64, logical uint32) []byte

AppendTimestamp appends an encoded MVCC timestamp onto key, returning the new key. The provided key should already have the 0x00 sentinel byte (i.e., key should be a proper prefix from the perspective of Pebble).

func Compare 

func Compare(a, b []byte) int

Compare compares cockroach keys, including the version (which could be MVCC timestamps).

func ComparePointSuffixes 

func ComparePointSuffixes(a, b []byte) int

ComparePointSuffixes compares suffixes of Cockroach point keys (which are composed of the version and a trailing version-length byte); the version can be an MVCC timestamp or a lock key. ComparePointSuffixes differs from CompareRangeSuffixes in that the ComparePointSuffixes normalizes the suffixes. Ideally we'd have one function that implemented the semantics of EnginePointSuffixCompare, but due to historical reasons, range key suffix comparisons must not perform normalization.

See https://github.com/cockroachdb/cockroach/issues/130533

func CompareRangeSuffixes 

func CompareRangeSuffixes(a, b []byte) int

CompareRangeSuffixes implements Comparer.CompareRangeSuffixes. It compares cockroach suffixes (which are composed of the version and a trailing sentinel byte); the version can be an MVCC timestamp or a lock key. It is stricter than ComparePointSuffixes due to historical reasons; see https://github.com/cockroachdb/cockroach/issues/130533

func DecodeEngineKey 

func DecodeEngineKey(b []byte) (roachKey, version []byte, ok bool)

DecodeEngineKey decodes the given bytes as an EngineKey.

func DecodeMVCCTimestampSuffix 

func DecodeMVCCTimestampSuffix(encodedTS []byte) (wallTime uint64, logical uint32, err error)

DecodeMVCCTimestampSuffix decodes an MVCC timestamp from its Pebble representation, including the length suffix.

func EncodeKey 

func EncodeKey(dst, roachKey, version []byte) []byte

EncodeKey serializes an engine key from a raochpb.Key and a version, appending the result to dst and returning the result.

func EncodeMVCCKey 

func EncodeMVCCKey(dst []byte, key []byte, walltime uint64, logical uint32) []byte

EncodeMVCCKey encodes a MVCC key into dst, growing dst as necessary.

func EncodeTimestamp 

func EncodeTimestamp(key []byte, walltime uint64, logical uint32) []byte

EncodeTimestamp encodes a MVCC timestamp into a key, returning the new key. The key's capacity must be sufficiently large to hold the encoded timestamp.

func Equal 

func Equal(a, b []byte) bool

Equal implements base.Equal for Cockroach keys.

func FormatKey 

func FormatKey(key []byte) fmt.Formatter

FormatKey returns a formatter for the user key.

func FormatKeySuffix 

func FormatKeySuffix(suffix []byte) fmt.Formatter

FormatKeySuffix returns a formatter for the user key suffix.

func NewMVCCTimeIntervalFilter 

func NewMVCCTimeIntervalFilter(minWallTime, maxWallTime uint64) sstable.BlockPropertyFilter

NewMVCCTimeIntervalFilter constructs a new block-property filter that skips keys that encode timestamps with wall times that do not fall within the interval [minWallTime,maxWallTime].

func NewTimestampSuffix 

func NewTimestampSuffix(wallTime uint64, logical uint32) []byte

NewTimestampSuffix allocates and encodes a new suffix byte slice encoding the provided timestamp.

func ParseFormattedKey 

func ParseFormattedKey(formattedKey string) []byte

ParseFormattedKey parses a human-readable representation of a Cockroach key.

ParseFormattedKey expects the roach key and version to be delimited by a '@' character. The '@' may be omitted for version-less keys. Roach keys may be quoted using Go syntax. The version may be:

  1. A MVCC timestamp in one of the formats (these are cribbed from the Cockroach repository's hlc timestamp formatting): a. <WALLTIME-SECONDS> b. <WALLTIME-SECONDS>.<WALLTIME-NANOS> c. <WALLTIME-SECONDS>,<LOGICAL> d. <WALLTIME-SECONDS>.<WALLTIME-NANOS>,<LOGICAL>
  2. A lock table key in the format: <STRENGTH-BYTE>,<TXNUUID>
  3. A raw hex-encoded string prefixed with "hex:".

func ParseFormattedKeySuffix 

func ParseFormattedKeySuffix(formattedSuffix string) []byte

ParseFormattedKeySuffix parses a human-readable representation of a Cockroach key's version. The formatted suffix may be:

  1. A MVCC timestamp in one of the formats (these are cribbed from the Cockroach repository's hlc timestamp formatting): a. <WALLTIME-SECONDS> b. <WALLTIME-SECONDS>.<WALLTIME-NANOS> c. <WALLTIME-SECONDS>,<LOGICAL> d. <WALLTIME-SECONDS>.<WALLTIME-NANOS>,<LOGICAL>
  2. A lock table key in the format: <STRENGTH-BYTE>,<TXN-UUID>

func RandomKVs 

func RandomKVs(rng *rand.Rand, count int, cfg KeyGenConfig, valueLen int) (keys, vals [][]byte)

RandomKVs constructs count random KVs with the provided parameters.

func Split 

func Split(key []byte) int

Split implements base.Split for CockroachDB keys.

Types

type KeyGenConfig 

type KeyGenConfig struct {
	PrefixAlphabetLen  int    // Number of bytes in the alphabet used for the prefix.
	PrefixLenShared    int    // Number of bytes shared by all key prefixes.
	RoachKeyLen        int    // Number of bytes in the prefix (without the 0 sentinel byte).
	AvgKeysPerPrefix   int    // Average number of keys (with varying suffixes) per prefix.
	BaseWallTime       uint64 // Smallest MVCC WallTime.
	PercentLogical     int    // Percent of MVCC keys with non-zero MVCC logical time.
	PercentEmptySuffix int    // Percent of keys with empty suffix.
	PercentLockSuffix  int    // Percent of keys with lock suffix.
}

KeyGenConfig configures the shape of the random keys generated.

func (KeyGenConfig) String 

func (cfg KeyGenConfig) String() string

type MVCCBlockIntervalSuffixReplacer 

type MVCCBlockIntervalSuffixReplacer struct{}

MVCCBlockIntervalSuffixReplacer implements the sstable.BlockIntervalSuffixReplacer interface for MVCC timestamp intervals.

func (MVCCBlockIntervalSuffixReplacer) ApplySuffixReplacement 

func (MVCCBlockIntervalSuffixReplacer) ApplySuffixReplacement(
	interval sstable.BlockInterval, newSuffix []byte,
) (sstable.BlockInterval, error)

type MVCCWallTimeIntervalRangeKeyMask 

type MVCCWallTimeIntervalRangeKeyMask struct {
	sstable.BlockIntervalFilter
}

MVCCWallTimeIntervalRangeKeyMask implements pebble.BlockPropertyFilterMask for filtering blocks using the MVCCTimeInterval block property during range key masking.

func (*MVCCWallTimeIntervalRangeKeyMask) Init 

func (m *MVCCWallTimeIntervalRangeKeyMask) Init()

Init initializees the mask and its block interval filter.

func (*MVCCWallTimeIntervalRangeKeyMask) SetSuffix 

func (m *MVCCWallTimeIntervalRangeKeyMask) SetSuffix(suffix []byte) error

SetSuffix implements the pebble.BlockPropertyFilterMask interface.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.