On Mon, Nov 2, 2020 at 3:50 AM Patrick O'Callaghan <pocallaghan@xxxxxxxxx> wrote: > > On Sun, 2020-11-01 at 19:00 -0700, Chris Murphy wrote: > > On Sun, Nov 1, 2020 at 6:51 AM John Mellor <john.mellor@xxxxxxxxx> wrote: > > > On 2020-10-31 10:46 p.m., Tim via users wrote: > > > > On Sat, 2020-10-31 at 16:11 +0000, lancelassetter@xxxxxxxxx wrote: > > > > > Will NFS tell you data has been corrupted during the transfer and > > > > > write process? > > > > Does any filing system? In general, writes to storage are assumed to > > > > have worked unless something throws up an error message. Your hard > > > > drive could be silently corrupting data as it writes to the drive due > > > > to various reasons (defects in its media, bugs in its firmware, > > > > glitches from bad power supplies). You'd never know unless your > > > > filing system did a sanity check after writing. Some specialised ones > > > > might do that, but the average ones don't > > > > > > > You are correct for some very popular filesystems. EXT2/3/4, XFS, NTFS > > > etc. will not detect this situation. However, newer filesystems (<10 > > > years old) do handle silent data glitches, bad RAM and cosmic ray hits > > > correctly. > > > > > > BTRFS has been the default filesystem on SUSE Linux for years, and is > > > now the default filesystem on Fedora-33. ZFS is an optional filesystem > > > on Ubuntu-20 and all the Berkeley-derived Unixen like FreeBSD, and > > > standard on Oracle Linux and Solaris. BTRFS and ZFS are both COW > > > filesystems using checksumming of both data and metadata. When you push > > > something to the disk(s) with some kind of RAM error or power glitch, > > > the first write will be stored with the error, and then the checksummed > > > metadata is simply redirected to reference the new stuff. This will > > > detect the checksum errors on the data on ZFS with the reread to verify > > > the checksum, but I believe that BTRFS will return a successful write > > > without one of the RAID configurations set on the pool. If you are > > > running one of the RAID configurations, the checksum error will be > > > detected before the write completes. To guard against on-disk > > > corruption (bit rot), both ZFS and BTRFS will also correct it on the > > > next read of that data if you are running the filesystem in one of the > > > RAID-z configurations (multiple copies stored), or upon running a > > > filesystem integrity check. > > > > Short story: > [...] > > Thanks. That more or less matches what I thought. So BTRFS does not do > read-after-write verification and ZFS does, correct? Just trying to > clarify. Btrfs doesn't do that, and I'm not aware of such an option in ZFS. https://github.com/openzfs/zfs/issues/2526 https://www.illumos.org/issues/2008 Btrfs and ZFS both do checksum verification of every read, and also have a scrub option. -- Chris Murphy _______________________________________________ users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
