Re: Ceph vs zfs data integrity

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, Jan 23, 2014 at 2:21 PM, Schlacta, Christ <aarcane@xxxxxxxxxxx> wrote:
> What guarantees does ceph place on data integrity? Zfs uses a Merkel tree to
> guarantee the integrity of all data and metadata on disk and will ultimately
> refuse to return "duff" data to an end user consumer.
>
> I know ceph provides some integrity mechanisms and has a scrub feature. Does
> it provide full data integrity guarantees in a manner similar to zfs? What
> will happen if duff data is encountered at runtime? What happens if all
> copies of some data are present, but damaged?
>
> Can ceph provide these guarantees, whatever they are, even when underlying
> storage provides no such guarantees ( xfs, extn, reiser)?

Sadly, Ceph cannot do as well as ZFS can here. The relevant interfaces
to pass data integrity information through the whole stack simply
don't exist. :( What Ceph does do:
1) All data sent over the wire is checksummed (crc32) and validated so
we know that what we got is what was sent.
2) All data in the osd journals is checksummed (crc32) so if we crash
and restart we can validate that independently of what the disk/fs is
telling us.
3) All data is regularly deep-scrubbed (based on your configuration)
to compare checksums across replicas.
And if you're on btrfs or zfs, there's also 4) the local fs won't
return duff data.

We'd love to do more and have experimented with it some (there is a
"sloppy crc" mechanism in the OSDs you can enable that tries to track
and validate what we've written to disk on our own, but it's simply
not suitable for production due to the extra disk traffic it
necessarily incurs), but until there's a way for us to get integrity
information straight from the client user application into the OSD
data store there's not much point.
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux