On Wed, Mar 30, 2016 at 3:57 PM, Sage Weil <sage@xxxxxxxxxxxx> wrote: > On Wed, 30 Mar 2016, Allen Samuels wrote: >> One thing to also factor in is that if you increase the span of a >> checksum, you degrade the quality of the checksum. So if you go with >> 128K chunks of data you'll likely want to increase the checksum itself >> from something beyond a CRC-32. Maybe somebody out there has a good way >> of describing this quanitatively. > > Good point. FWIW, I think we should default to xxhash over crc32c: > > https://github.com/Cyan4973/xxHash > > Note that there is a 64-bit version that's faster on 64-bit procs. Random googling (...and StackOverflow) lead me to https://users.ece.cmu.edu/~koopman/roses/dsn04/koopman04_crc_poly_embedded.pdf, which only extends up to 2KB (with a crc16, which makes me think crc32 can go a long way) but for anybody who actually reads it can probably be extended to larger block sizes without much difficulty. -Greg -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
