abloom

abloom is a high-performance Bloom filter implementation for Python, written in C.

Why abloom?

• Fastest: 3x faster than the fastest alternative rbloom on add/update, 1.2x faster on lookup
• Fully-Featured: Complete API with set operations and serialization
• Thoroughly Tested: 500+ tests for Python 3.8+ on Linux, macOS, and Windows
• Zero Dependencies: Pure C extension without external dependencies

Quick Start

Install with pip install abloom.

from abloom import BloomFilter

bf = BloomFilter(1_000_000, 0.01)  # capacity, false positive rate
bf.add(1)
bf.update(["a", "b", "c"])

1 in bf                 # True
2 in bf                 # False
bf2 = bf.copy()         # duplicate filter
combined = bf | bf2     # union of filters
bf.clear()              # reset to empty

Benchmarks

1M integers, 1% FPR, Apple M2. Full results here.

Use Cases

Database Optimization

user_cache = BloomFilter(10_000_000, 0.01)
if user_id not in user_cache:
    return None           # Definitely not in DB
return db.query(user_id)  # Probably in DB

Web Crawling

seen = BloomFilter(10_000_000, 0.001)
if seen.add(url):  # True if definitely new
    crawl(url)

Spam Detection

spam_filter = BloomFilter(1_000_000, 0.001)
spam_filter.update(spam_words)
if word in spam_filter:
    flag_as_potential_spam()

Serialization

Save and restore filters across sessions or processes:

from abloom import BloomFilter

# Create filter with serializable=True
bf = BloomFilter(100_000, 0.01, serializable=True)
bf.update(["user123", "user456", "user789"])

with open("filter.bloom", "wb") as f:
    f.write(bf.to_bytes())

with open("filter.bloom", "rb") as f:
    restored = BloomFilter.from_bytes(f.read())

"user123" in restored  # True

Note: You must set serializable=True during initialization to transfer filters between processes. This mode uses a deterministic hash function (xxHash) and supports bytes, str, int, and float types only. Otherwise, abloom will use Python's built-in hashing, which relies on a process-specific seed to hash bytes and str. int and float types in serializable mode will still behave "normally," for example, 15 and 15.0 will hash to the same value, as will 0.0 and -0.0. This is because abloom still uses Python's built-in hashing for int and float types.

API Summary

Properties: capacity, fp_rate, k, byte_count, bit_count, serializable, free_threading

See also: API Reference, Implementation Details

Thread Safety

By default, abloom is thread-safe on standard Python with the global interpreter lock (GIL). For free-threaded Python, set free_threading=True for thread safety. More details here.

Development

Testing

pip install -e . --group test
pytest tests/ --ignore=tests/test_benchmark.py -v

See Testing for more details.

Benchmarking

pip install -e . --group benchmark
pytest tests/test_benchmark.py --benchmark-only

See Benchmarking for more details.