Faster hashing for DHT

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



While looking at the DHT code, I noticed that it's using a 10-round
Davies-Meyer construction to generate the hashes used for file
placement.  A little surprised, by this, I ran it by a couple of friends
who are experts in both cryptography and distributed data storage.  The
consensus seems to be that the hash used for this purpose needs to be
collision resistant but not cryptographically strong.  One theorized
that the choice made in DHT is probably based on prior examples (e.g.
Freenet and Mojo Nation) where cryptographically strong hashes were
chosen, but that the requirements driving those decisions probably don't
apply to GlusterFS.  This is a non-trivial issue because these hashes
are used quite frequently and the current one is quite computationally
expensive.  I note that Hsieh's SuperFastHash is already implemented in
GlusterFS and is used for other purposes.  It's about 3x as fast as the
DM hash, and has better collision resistance as well.  MurmurHash
(http://murmurhash.googlepages.com/) is even faster and more collision
resistant.  For future releases, I suggest dropping the DM hash and
switching to one of these others.




[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux