Documentation
¶
Overview ¶
Package batch groups file fingerprints into upload-sized batches. Batcher accumulates fingerprints up to a maximum byte size, and CombineFingerprints joins a set of fingerprints into a single WFP byte stream for the scan request.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CombineFingerprints ¶
func CombineFingerprints(fps []*fingerprint.FileFingerprint) string
CombineFingerprints combines multiple fingerprints into a single WFP string Ensures there is a blank line between each file.
Uses a strings.Builder with a pre-sized buffer: naive `result += ...` in a loop is O(n²) (each += copies the whole accumulated string), which dominated wall-clock time for large scans — e.g. ~38s to assemble a 16 MB WFP from ~8900 files. Builder grows amortized O(1), making this linear.
Types ¶
type BatchWithMetadata ¶
type BatchWithMetadata struct {
Data []*fingerprint.FileFingerprint
IsFinalChunk bool
}
BatchWithMetadata wraps a batch with metadata about its position
type Batcher ¶
type Batcher struct {
// contains filtered or unexported fields
}
Batcher groups fingerprints into batches according to a maximum size
func NewBatcher ¶
NewBatcher creates a new fingerprint batcher
func (*Batcher) Add ¶
func (b *Batcher) Add(fp *fingerprint.FileFingerprint)
Add adds a fingerprint to the batcher If it exceeds the maximum size, creates a new batch
func (*Batcher) GetBatches ¶
func (b *Batcher) GetBatches() [][]*fingerprint.FileFingerprint
GetBatches returns all batches, including the current batch if not empty
type StreamingBatcher ¶
type StreamingBatcher struct {
// contains filtered or unexported fields
}
StreamingBatcher groups fingerprints and sends them to a channel when they reach the maximum size
func NewStreamingBatcher ¶
func NewStreamingBatcher(maxSize int) *StreamingBatcher
NewStreamingBatcher creates a new batcher in streaming mode
func (*StreamingBatcher) Add ¶
func (sb *StreamingBatcher) Add(fp *fingerprint.FileFingerprint)
Add adds a fingerprint and sends the batch if it reaches the limit
func (*StreamingBatcher) Batches ¶
func (sb *StreamingBatcher) Batches() <-chan BatchWithMetadata
Batches returns the batches channel
func (*StreamingBatcher) Close ¶
func (sb *StreamingBatcher) Close()
Close closes the batcher sending the last pending batch marked as final
Source Files