bbloom

func JSONUnmarshal ¶

func JSONUnmarshal(dbData []byte) (*Bloom, error)

JSONUnmarshal restores a bloom filter from a JSON byte slice produced by Bloom.JSONMarshal or Bloom.JSONMarshalTS.

// Create and populate a filter.
bf, err := New(float64(10000), 0.001)
if err != nil {
	panic(err)
}
bf.Add([]byte("persisted"))

// Serialize to JSON.
data := bf.JSONMarshal()

// Restore from JSON.
bf2, err := JSONUnmarshal(data)
if err != nil {
	panic(err)
}

fmt.Println(bf2.Has([]byte("persisted")))
fmt.Println(bf2.Has([]byte("missing")))

Output:
true
false

func New ¶

func New(params ...float64) (bloomfilter *Bloom, err error)

New creates a bloom filter with default SipHash keys. It accepts exactly two float64 arguments:

• If the second parameter is < 1 it is treated as a false-positive rate, and the filter is sized automatically. Example: New(float64(1<<16), 0.01) -- 65536 items, 1% FP rate.

• If the second parameter is >= 1 it is treated as the number of hash locations, and the first parameter is the bitset size. Example: New(650000.0, 7.0) -- 650000-bit filter, 7 hash locations.

The default SipHash keys are publicly known constants. If the filter will hold data controlled by untrusted parties, use NewWithKeys instead to prevent hash-flooding attacks.

// Create a filter sized for 10000 entries with a 0.1% false positive rate.
bf, err := New(float64(10000), 0.001)
if err != nil {
	panic(err)
}

bf.Add([]byte("hello"))
bf.Add([]byte("world"))

fmt.Println(bf.Has([]byte("hello")))
fmt.Println(bf.Has([]byte("world")))
fmt.Println(bf.Has([]byte("missing")))

Output:
true
true
false

func NewWithBoolset ¶

func NewWithBoolset(bs []byte, locs uint64) (bloomfilter *Bloom)

NewWithBoolset creates a bloom filter from a pre-existing bitset, typically obtained from a previous Bloom.JSONMarshal export or an external source. bs is the serialized bitset (big-endian uint64 words) and locs is the number of hash locations per entry. The filter uses default SipHash keys; use NewWithBoolsetAndKeys to restore a filter that was created with custom keys.

func NewWithBoolsetAndKeys ¶

func NewWithBoolsetAndKeys(bs []byte, locs, k0, k1 uint64) (bloomfilter *Bloom)

NewWithBoolsetAndKeys creates a bloom filter from a pre-existing bitset with caller-provided SipHash keys. This is the constructor to use when restoring a filter that was originally created with NewWithKeys. See NewWithKeys for why custom keys matter and NewWithBoolset for how the bitset parameter is interpreted.

func NewWithKeys ¶

func NewWithKeys(k0, k1 uint64, params ...float64) (*Bloom, error)

NewWithKeys creates a bloom filter with caller-provided SipHash keys.

The default keys used by New are publicly known constants baked into the source code. An attacker who knows the keys can craft inputs that all hash to the same bit positions, filling the filter faster than normal and raising the false-positive rate. This is a concern when the filter holds data chosen by untrusted parties (e.g. content-addressed blocks fetched from the network).

Providing random, secret keys (e.g. generated once per node from crypto/rand) restores SipHash's anti-collision guarantees and makes such attacks infeasible.

The params are interpreted the same way as in New. Custom keys are preserved across Bloom.JSONMarshal / JSONUnmarshal round-trips. Note: custom keys are included in plaintext in the Bloom.JSONMarshal output, so treat serialized filters accordingly.

// Use random secret keys to prevent hash-flooding attacks when the
// filter holds data from untrusted sources.
var k0, k1 uint64 = 0x0123456789abcdef, 0xfedcba9876543210

bf, err := NewWithKeys(k0, k1, float64(10000), 0.001)
if err != nil {
	panic(err)
}

bf.Add([]byte("hello"))
fmt.Println(bf.Has([]byte("hello")))
fmt.Println(bf.Has([]byte("missing")))

Output:
true
false