Documentation
¶
Overview ¶
Package cachekey defines several helper functions for building up and decoding keys for use by clients of galaxycache.
The encoding scheme has three key design criteria: reasonable performance, simplicity and robustness
- All fields are separated by null bytes
- Strings are prefixed with a varint length to eliminate any ambiguity (as handled by encoding/binary)
- Integers are encoded as a varint.
This design is inspired by [OrderedCodes](https://github.com/google/orderedcode), but, since keys do not need to preserve ordering, we can use a slightly simpler design.
Index ¶
- func AppendEpoch(buf []byte, cfg EpochConfig, now time.Time, jitter Jitter) []byte
- func AppendStrID[I ~string](buf []byte, id I) []byte
- func AppendUint[I ~uint64](buf []byte, id I) []byte
- func ConsumeInt(buf []byte) (uint64, []byte, error)
- func ConsumeString(buf []byte) (string, []byte, error)
- type EpochConfig
- type Jitter
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AppendEpoch ¶
AppendEpoch appends the jittered epoch suffix. Jitter is taken mod the epoch config's EpochLength (in nanoseconds) The epoch number is encoded with AppendUint See the comment on EpochConfig for information about how to use this tool.
func AppendStrID ¶
AppendStrID encodes a string ID into a form that can be concatenated into a V2 galaxycache key
func AppendUint ¶
AppendUint encodes an unsigned integer into a form that can be concatenated into a V2 galaxycache key
func ConsumeInt ¶
ConsumeInt returns a consumed int and the remaining buffer or an error if the input is malformed
Types ¶
type EpochConfig ¶
type EpochConfig struct {
// Base timestamp from which epochs are measured. By making a
// relatively recent timestamp that still guarantees that the epoch is
// positive, we can save a couple bytes in our keys.
EpochZero time.Time
// A prime number of nanoseconds indicating the length of an epoch.
// For example, one can use 396_999_999_953 for a ~6.5-ish minute epoch
// that's about 23s less than 7 minutes.
// note that 23, 7 and 397 are all primes, so the epochs will wrap around nicely over time.
EpochLength time.Duration
// A large prime to multiply integer IDs by to ensure that adjacent IDs get spread out.
// For example, one might use 330_999_999_991 to work with the 7 minute-ish value above
// it would be chosen to be about 5 minutes 31 seconds in nanoseconds.
// 331 is a prime number of seconds, as are 5 minutes and 31 seconds.
// Due to constraints on the bits.Div64 function, we can't have too
// large a value in hi, so this multiplier is required to be strictly
// less than epochLength if we want to support the full uint64 space for integer IDs.
// This should give a good spread of jitter values from epochMode.
IntIDJitterPrimeMultiplier time.Duration
}
EpochConfig defines the tunables behind how epoch-numbers are calculated. In particular, it encapsulates the reference time, epoch length, and an integer ID jitter coefficient (so small integer IDs get spread out sufficiently).
This implements the scheme described in the [Vimeo Engineering Blog post entitled "Vimeo metadata caching at Vimeo"](https://medium.com/vimeo-engineering-blog/video-metadata-caching-at-vimeo-a54b25f0b304). In particular, it provides a way to implement TTLs without explicit support from Galaxycache by rotating the cache-key every [EpochConfig.EpochLength], and offsetting that by a bit based on the input ID, we prevent individual keys from expiring/rotating at exactly the same time.
Note: for this to be useful during rollouts, these parameters should be stable over time, so we recommend defining constants in your code for the two time.Duration fields.
type Jitter ¶
type Jitter struct {
// contains filtered or unexported fields
}
Jitter is an int128 opaque value used with AppendEpoch, and EpochConfig to track the epoch boundary offset for a specific ID.
func JitterIntID ¶
func JitterIntID[I ~uint64](e *EpochConfig, id I) Jitter
JitterIntID returns a jittered offset within the ID
func JitterStrID ¶
JitterStrID hashes the ID and computes a jittered value using a truncated sha256.