dedup

package
v1.8.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 5, 2026 License: MIT Imports: 8 Imported by: 0

Documentation

Overview

Package dedup provides content-hash deduplication middleware for Trove. It hashes data on write and checks for existing content before storing.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Dedup

type Dedup struct {
	// contains filtered or unexported fields
}

Dedup provides content-hash deduplication middleware.

func New

func New(opts ...Option) *Dedup

New creates a new deduplication middleware.

func (*Dedup) Direction

func (d *Dedup) Direction() middleware.Direction

Direction returns Write since dedup only checks on the write path.

func (*Dedup) Name

func (d *Dedup) Name() string

Name returns the middleware identifier.

func (*Dedup) WrapWriter

func (d *Dedup) WrapWriter(ctx context.Context, w io.WriteCloser, key string) (io.WriteCloser, error)

WrapWriter wraps a writer to hash content and check for duplicates. If a duplicate is found, the write is skipped (empty data written).

type MemoryStore

type MemoryStore struct {
	// contains filtered or unexported fields
}

MemoryStore is an in-memory Store for testing.

func NewMemoryStore

func NewMemoryStore() *MemoryStore

NewMemoryStore creates a new in-memory dedup store.

func (*MemoryStore) Exists

func (s *MemoryStore) Exists(_ context.Context, hash string) (bool, error)

Exists checks if the hash exists.

func (*MemoryStore) Record

func (s *MemoryStore) Record(_ context.Context, hash, key string) error

Record stores a hash → key mapping.

type OnDuplicate

type OnDuplicate func(ctx context.Context, key, hash string)

OnDuplicate is called when a duplicate is detected. The handler receives the key being written and the hash that matched.

type Option

type Option func(*Dedup)

Option configures the Dedup middleware.

func WithHashAlgorithm

func WithHashAlgorithm(alg internal.ChecksumAlgorithm) Option

WithHashAlgorithm sets the hash algorithm (default: blake3).

func WithOnDuplicate

func WithOnDuplicate(fn OnDuplicate) Option

WithOnDuplicate sets a callback for duplicate detection.

func WithStore

func WithStore(store Store) Option

WithStore sets the dedup store for tracking hashes.

type Store

type Store interface {
	// Exists checks if content with the given hash already exists.
	Exists(ctx context.Context, hash string) (bool, error)

	// Record stores a hash → key mapping.
	Record(ctx context.Context, hash, key string) error
}

Store tracks content hashes to detect duplicates.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL