Finished my deep dive into Bloom Filters (Classic, Counting, Cuckoo), and why they’re IMO a solid "pre-cache" tool you're probably not using
Posted by axel-user@reddit | programming | View on Reddit | 4 comments
I’ve just wrapped up a three-part deep-dive series on Bloom Filters and their modern cousins. If you're curious about data structures for fast membership checks, you might find it useful.
Approximate membership query (AMQ) filters don’t tell you exactly what's in a set, but they tell you what’s definitely not there and do it using very little memory. As for me, that’s a killer feature for systems that want to avoid unnecessarily hitting the bigger persistent cache, disk, or network.
Think of them as cheap pre-caches: a small test before the real lookup that helps skip unnecessary work.
Here's what the series covers:
Classic Bloom Filter
I walk through how they work, their false positive guarantees, and why deleting elements is dangerous. It includes an interactive playground to try out inserts and lookups in real time, also calculating parameters for your custom configuration.
Counting Bloom Filter and d-left variant
This is an upgrade that lets you delete elements (with counters instead of bits), but it comes at the cost of increased memory and a few gotchas if you’re not careful.
Cuckoo Filter
This is a modern alternative that supports deletion, lower false positives, and often better space efficiency. The most interesting part is the witty use of XOR to get two bucket choices with minimal metadata. And they are practically a solid replacement for classic Bloom Filters.
I aim to clarify the internals without deepening into formal proofs, more intuition, diagrams, and some practical notes, at least from my experience.
If you’re building distributed systems, databases, cache layers, or just enjoy clever data structures, I think you'll like this one.
arkvesper@reddit
!remindme 12 days
RemindMeBot@reddit
I will be messaging you in 12 days on 2025-07-16 06:02:28 UTC to remind you of this link
CLICK THIS LINK to send a PM to also be reminded and to reduce spam.
^(Parent commenter can ) ^(delete this message to hide from others.)
jaskij@reddit
Definitely something I'll read, if only to understand better. We deal with time series data at work and I've seen bloom filters mentioned quite a few times in release notes of the database we use.
Jolly-Warthog-1427@reddit
Bloom filters are amazing. At least the algorithms I've touched can tell you if an item definetly has not been seen before. So its a very quick way to exit early and it takes almost zero storage/memory compared to the actual data. I used it in my csp reporting service I made. A quick way to know if an item has been seen before and should be incremented or if its new and should be created