On 29/06/15 12:51, Gregory Farnum wrote: > > Yes, we have our own CRC32 checksum because loooong ago (before I > started!) Sage saw a lot of network corruption that wasn't being > caught by the TCP checksums so he added some to the Ceph message > stream. I can't tell you with any authority whatsoever how common that > problem is, but I don't think we're turning them off by default in > upstream. :) If the CRC32 implementation in Ceph is that dated (particularly the software implementations that will be used on AMD hardware), would it be worth checking out some of the updated implementations, such as the slice-by-16 or chunked methods? I found this link http://create.stephan-brumme.com/crc32/ and tried running the benchmark on an AMD Opteron 6386 SE system, with the following results: bitwise : CRC=221F390F, 47.525s, 21.546 MB/s half-byte : CRC=221F390F, 11.828s, 86.576 MB/s 1 byte at once: CRC=221F390F, 6.347s, 161.332 MB/s 4 bytes at once: CRC=221F390F, 2.875s, 356.178 MB/s 8 bytes at once: CRC=221F390F, 2.004s, 510.932 MB/s 4x8 bytes at once: CRC=221F390F, 1.929s, 530.811 MB/s 16 bytes at once: CRC=221F390F, 1.892s, 541.179 MB/s 16 bytes at once: CRC=221F390F, 1.926s, 531.797 MB/s (including prefetching) chunked : CRC=221F390F, 1.919s, 533.656 MB/s AFAIK, Ceph uses the slice-by-8 method if no hardware crc32 is found. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
