mercredi 22 avril 2020

Still worth using bloom filter if you database already does?

I have read in a couple of places that some databases use bloom filters for finding a match when querying a database. In my example I'm using Postgresql which is one of those databases.

My question comes up when talking about implementing a bloom filter with Redis which has a module that allows you to use a bloom filter when entering a member in a set. (Keep in mind the complexity of the lookup process though, not retrieving that value from disk)

Now the benefits of using Redis is to store values in memory, which when then trying to retrieve that value is more performant vs looking it up in a rdbms because that value is stored on a disk.

In my example, say I'm checking if a username already exists, is it still worth using Redis in memory solution with a bloom filter vs just checking with a Postgresql query?

My flow is something like:
CheckIfUserExsits() // using Redis bloom filter
If TRUE then confirm with rdbms // do to x% probability of false positive nature of bloom filter
If rdbms == MATCH then reply with "User does exist"
Else don't check rdbms at all // do to 0% probability of false negative nature of bloom filter

This flow is supposed to be more preformant because your not querying a rdbms and doing this quickly via memory lookup returning false more efficiently.

However since all I care about is whether a member exists or not, to increase performance on replying with false, does the Redis step really help? Because if Postgresql is already querying a table using a bloom filter then the preformance should already be relativly fast.

Aucun commentaire:

Enregistrer un commentaire