Documentation
¶
Index ¶
Constants ¶
const DefaultSegmentWalkInterval = 1 * time.Hour
DefaultSegmentWalkInterval is how long a walked segment's derivation is trusted before it is walked again (hint growth bypasses it). Walk volume needs no count cap: reads are paced by the shared compaction I/O limiter.
Variables ¶
This section is empty.
Functions ¶
func NewCompactionRateLimiter ¶ added in v1.10.2
NewCompactionRateLimiter creates the shared payload budget used by all background compaction work. A positive limit allows one startup burst no larger than 64 KiB or the configured per-second rate, whichever is smaller. A non-positive limit leaves a directly constructed Compactor unthrottled.
func PrepareEntryForCompaction ¶
PrepareEntryForCompaction prepares the key and value to store in compaction index.
Types ¶
type Compactor ¶
type Compactor struct {
// contains filtered or unexported fields
}
func NewCompactorWithConfig ¶
func NewCompactorWithConfig(cfg *CompactorConfig) *Compactor
NewCompactorWithConfig creates a new compactor with configuration
func (*Compactor) Close ¶
func (c *Compactor) Close()
Close stops the background compaction loop and waits for it to exit.
func (*Compactor) CompactFiles ¶
CompactFiles scans the RocksDB file-index and migrates files into segments. The context parameter allows for cancellation of the compaction process. The workerID is used to partition work across multiple workers. Compaction continues until the current segment is full or there are no more entries. Returns the number of files processed.
func (*Compactor) RateLimitBytesPerSecond ¶ added in v1.10.2
RateLimitBytesPerSecond reports the configured shared payload-read budget, or 0 when compaction is unthrottled. Exposed so callers (and tests) can verify the configured budget actually reached the compactor.
type CompactorConfig ¶
type CompactorConfig struct {
MetaDB *metadata.MetaDB
FileManager *files.FileManager
SegmentManager *segment.Manager
DeletionQueue *deletion.Queue
CompactionThreads int // Number of compaction threads
// Recompaction settings (optional)
EnableRecompaction bool
FragThreshold float64
MinSegmentAge time.Duration
MinSegments int
RecompactionInterval time.Duration
// CompactionBytesPerSecond is shared by file compaction and recompaction.
// A non-positive value disables limiting for directly constructed compactors.
CompactionBytesPerSecond int64
}
CompactorConfig contains configuration for the compactor
type ErrFileSizeMismatch ¶
ErrFileSizeMismatch is returned when a file's actual size doesn't match its metadata
func (*ErrFileSizeMismatch) Error ¶
func (e *ErrFileSizeMismatch) Error() string
type SegmentRecompactor ¶
type SegmentRecompactor struct {
// contains filtered or unexported fields
}
SegmentRecompactor handles recompaction of fragmented segments
func NewSegmentRecompactor ¶
func NewSegmentRecompactor(meta *metadata.MetaDB, sm *segment.Manager, deletionQueue *deletion.Queue, fragThreshold float64, minSegmentAge time.Duration, minSegments int) *SegmentRecompactor
NewSegmentRecompactor creates a new segment recompactor without a rate limit. NewCompactorWithConfig uses the internal constructor to share its limiter with file compaction workers.
func (*SegmentRecompactor) RecompactFragmentedSegments ¶
func (sr *SegmentRecompactor) RecompactFragmentedSegments(ctx context.Context) error
RecompactFragmentedSegments identifies and recompacts fragmented segments
Source Files