Documentation
¶
Index ¶
- func NewNMergedIterators(iterators []node.Node) node.Node
- type Config
- type EmptyIterator
- type Iterator
- type IteratorAsc
- type IteratorDesc
- type MappingProvider
- type SkipMask
- type SkipMaskBinIn
- type SkipMaskBinOut
- type SkipMaskManager
- func (smm *SkipMaskManager) GetIDsIteratorByFrac(fracName string, minLID, maxLID uint32, reverse bool) (node.Node, bool, error)
- func (smm *SkipMaskManager) IsDone() bool
- func (smm *SkipMaskManager) RefreshFrac(fraction frac.Fraction)
- func (smm *SkipMaskManager) RemoveFrac(fracName string)
- func (smm *SkipMaskManager) Start(fracs fracmanager.List)
- func (smm *SkipMaskManager) Stop()
- type SkipMaskParams
- type SkipMaskStatus
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Config ¶
type Config struct {
DataDir string // Directory to store skip mask files
Workers int // Number of concurrent workers for processing
CacheSizeLimit uint64 // Maximum size of the headers cache in bytes
}
Config holds configuration parameters for SkipMaskManager.
type EmptyIterator ¶
type EmptyIterator struct{}
func (*EmptyIterator) Next ¶
func (it *EmptyIterator) Next() node.LID
func (*EmptyIterator) String ¶
func (it *EmptyIterator) String() string
type Iterator ¶
type Iterator struct {
// contains filtered or unexported fields
}
func NewIterator ¶
type IteratorAsc ¶
type IteratorAsc Iterator
func (*IteratorAsc) Next ¶
func (it *IteratorAsc) Next() node.LID
func (*IteratorAsc) String ¶
func (it *IteratorAsc) String() string
type IteratorDesc ¶
type IteratorDesc Iterator
func (*IteratorDesc) Next ¶
func (it *IteratorDesc) Next() node.LID
func (*IteratorDesc) String ¶
func (it *IteratorDesc) String() string
type MappingProvider ¶
type SkipMask ¶
type SkipMask struct {
// contains filtered or unexported fields
}
func NewSkipMask ¶
func NewSkipMask(params SkipMaskParams) *SkipMask
type SkipMaskBinIn ¶
SkipMaskBinIn is the input structure for serializing a skip mask. It contains a slice of Local IDs (LIDs) that correspond to documents matching the skip mask query criteria.
type SkipMaskBinOut ¶
type SkipMaskBinOut struct {
LIDs []uint32
}
SkipMaskBinOut is the output structure for deserialized skip mask data. After unmarshaling, LIDs are converted to uint32 array.
type SkipMaskManager ¶
type SkipMaskManager struct {
// contains filtered or unexported fields
}
SkipMaskManager manages the lifecycle of skip masks across all fractions. It processes skip mask queries, stores results to disk, and provides iteration over matching document IDs.
Skip masks are organized by query parameters (query string, from/to MID range). Each skip mask maintains a directory containing files for each fraction:
- .queue file: fraction is currently being processed
- .skipmask file: processing complete, contains matching document LIDs
The manager runs background maintenance tasks for disk usage monitoring and cache cleanup.
func New ¶
func New( ctx context.Context, cfg Config, params []SkipMaskParams, mp MappingProvider, ) *SkipMaskManager
New creates a new SkipMaskManager with the given configuration. If Workers is not set (0), it defaults to GOMAXPROCS.
func (*SkipMaskManager) GetIDsIteratorByFrac ¶
func (smm *SkipMaskManager) GetIDsIteratorByFrac( fracName string, minLID, maxLID uint32, reverse bool, ) (node.Node, bool, error)
GetIDsIteratorByFrac returns an iterator over document IDs that match the skip mask queries for a specific fraction, within the given LID range.
Parameters:
- fracName: the name of the fraction to query
- minLID: minimum local ID (inclusive)
- maxLID: maximum local ID (inclusive)
- reverse: if true, iterates IDs in descending order
Returns:
- node.Node: iterator over matching document IDs
- bool: true if the fraction has any skip mask files, false otherwise
- error: any error encountered while opening the skip mask files
func (*SkipMaskManager) IsDone ¶
func (smm *SkipMaskManager) IsDone() bool
IsDone returns true if all skip masks have been processed and are in StatusDone state. This is useful for determining when initial skip mask computation is complete.
func (*SkipMaskManager) RefreshFrac ¶
func (smm *SkipMaskManager) RefreshFrac(fraction frac.Fraction)
RefreshFrac recomputes skip mask files for a fraction after it has been sealed. This is called when an active fraction becomes sealed. The method:
- Removes existing skip mask files for the fraction
- Marks relevant skip masks as in-progress
- Queues the fraction for reprocessing
- Asynchronously processes the fraction through all matching skip masks
func (*SkipMaskManager) RemoveFrac ¶
func (smm *SkipMaskManager) RemoveFrac(fracName string)
RemoveFrac removes all skip mask files associated with a fraction. This should be called when a fraction is deleted from the system. The removal is performed asynchronously in the background.
func (*SkipMaskManager) Start ¶
func (smm *SkipMaskManager) Start(fracs fracmanager.List)
Start initializes and starts the skip mask manager. It performs the following steps:
- Creates the data directory if it doesn't exist
- Loads any existing skip masks from previous sessions
- Builds the processing queue for all fractions
- Starts background maintenance and cache cleanup loops
- Begins asynchronous processing of all skip mask queries
This method must be called before using the manager.
func (*SkipMaskManager) Stop ¶
func (smm *SkipMaskManager) Stop()
Stop gracefully stops the skip mask manager. It cancels the context, waits for all background goroutines to complete, and logs a message when fully stopped.
type SkipMaskStatus ¶
type SkipMaskStatus byte
const ( StatusCreated SkipMaskStatus = iota StatusInProgress StatusDone StatusError )
Source Files