Re: Faster hashing for DHT

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



>> I note that Hsieh's SuperFastHash is already implemented in
>> GlusterFS and is used for other purposes.  It's about 3x as fast as the
>> DM hash, and has better collision resistance as well.  MurmurHash
>> (http://murmurhash.googlepages.com/) is even faster and more collision
>> resistant.  For future releases, I suggest dropping the DM hash and
>> switching to one of these others.
>>

We will investigate murmurhash and see if it suits the role better.
What we look for in the hashing algorithm is better hash distribution
specifically for filenames (ascii strings 1-255 characters long).
Better distribution of user data in general is not of much help for
DHT's needs. DM hash does a very good job in distributing hashes for
ascii strings 1-256 bytes long (file basenames). It is also the same
algorithm used in reiserfs which is very sensitive to filename hash
collisions. Speed is really a micro optimization considering the small
portion of time which is used for actual hash computation.

Will keep you posted when we have comparison results with murmurhash.

Avati




[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux