Documentation
¶
Index ¶
Constants ¶
const ChunkLen = 1024
ChunkLen is the length of a chunk in bytes.
Variables ¶
var ( ErrInvalidChecksum = errs.New("invalid chunk checksum") ErrWrongMetadata = errs.New("wrong chunk metadata") ErrMetadataLength = errs.New("chunk metadata wrong length") ErrDataLength = errs.New("chunk data wrong length") ErrSliceOutOfRange = errs.New("chunk can't be sliced out of its data range") ErrChunkDecode = errs.New("error decoding freshly created chunk") )
var ( ErrRewriteNoDataLeft = errors.New("chunk has no data left after rewriting") ErrSliceChunkOverflow = errors.New("slicing should not overflow a chunk") )
Functions ¶
func MustRegisterEncoding ¶
MustRegisterEncoding add a new chunk encoding. There is no locking, so this must be called in init().
Types ¶
type Chunk ¶
type Chunk struct {
logproto.ChunkRef
Metric labels.Labels `json:"metric"`
// We never use Delta encoding (the zero value), so if this entry is
// missing, we default to DoubleDelta.
Encoding Encoding `json:"encoding"`
Data Data `json:"-"`
// contains filtered or unexported fields
}
Chunk contains encoded timeseries data
func NewChunk ¶
func NewChunk(userID string, fp model.Fingerprint, metric labels.Labels, c Data, from, through model.Time) Chunk
NewChunk creates a new chunk
func ParseExternalKey ¶
ParseExternalKey is used to construct a partially-populated chunk from the key in DynamoDB. This chunk can then be used to calculate the key needed to fetch the Chunk data from Memcache/S3, and then fully populate the chunk with decode().
Pre-checksums, the keys written to DynamoDB looked like `<fingerprint>:<start time>:<end time>` (aka the ID), and the key for memcache and S3 was `<user id>/<fingerprint>:<start time>:<end time>. Finger prints and times were written in base-10.
Post-checksums, externals keys become the same across DynamoDB, Memcache and S3. Numbers become hex encoded. Keys look like: `<user id>/<fingerprint>:<start time>:<end time>:<checksum>`.
v12+, fingerprint is now a prefix to support better read and write request parallelization: `<user>/<fprint>/<start>:<end>:<checksum>`
func (*Chunk) Decode ¶
func (c *Chunk) Decode(decodeContext *DecodeContext, input []byte) error
Decode the chunk from the given buffer, and confirm the chunk is the one we expected.
type Data ¶
type Data interface {
// Add adds a SamplePair to the chunks, performs any necessary
// re-encoding, and creates any necessary overflow chunk.
// The returned Chunk is the overflow chunk if it was created.
// The returned Chunk is nil if the sample got appended to the same chunk.
Add(sample model.SamplePair) (Data, error)
Marshal(io.Writer) error
UnmarshalFromBuf([]byte) error
Encoding() Encoding
// Rewrite rewrites the chunk after filtering out lines based on response from filter.Func.
// Filter.Func would be called for each log entry, and the ones for which it returns true would be removed.
Rewrite(filter filter.Func) (Data, error)
// Size returns the approximate length of the chunk in bytes.
Size() int
// UncompressedSize returns the length of uncompressed bytes.
UncompressedSize() int
// Entries returns the number of entries in a chunk
Entries() int
Utilization() float64
}
Data is the interface for all chunks. Chunks are generally not goroutine-safe.
func NewForEncoding ¶
NewForEncoding allows configuring what chunk type you want
type DecodeContext ¶
type DecodeContext struct {
// contains filtered or unexported fields
}
DecodeContext holds data that can be re-used between decodes of different chunks
func NewDecodeContext ¶
func NewDecodeContext() *DecodeContext
NewDecodeContext creates a new, blank, DecodeContext
type Encoding ¶
type Encoding byte
Encoding defines which encoding we are using, delta, doubledelta, or varbit
type RequestChunkFilterer ¶
RequestChunkFilterer creates ChunkFilterer for a given request context.