Re: fsync() errors is unsafe and risks data loss

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

On 2018-04-12 12:19:26 +0200, Lukas Czerner wrote:
> On Wed, Apr 11, 2018 at 07:32:21PM -0700, Andres Freund wrote:
> 
> > And there's cases where that just doesn't help at all. Being able to
> > untar a database from backup / archive / timetravel / whatnot, and then
> > fsyncing the directory tree to make sure it's actually safe, is really
> > not an insane idea.  Or even just cp -r ing it, and then starting up a
> > copy of the database.  What you're saying is that none of that is doable
> > in a safe way, unless you use special-case DIO using tooling for the
> > whole operation (or at least tools that fsync carefully without ever
> > closing a fd, which certainly isn't the case for cp et al).
> 
> Does not seem like a problem to me, just checksum the thing if you
> really need to be extra safe. You should probably be doing it anyway if
> you backup / archive / timetravel / whatnot.

That doesn't really help, unless you want to sync() and then re-read all
the data to make sure it's the same. Rereading multi-TB backups just to
know whether there was an error that the OS knew about isn't
particularly fun. Without verifying after sync it's not going to improve
the situation measurably, you're still only going to discover that $data
isn't available when it's needed.

What you're saying here is that there's no way to use standard linux
tools to manipulate files and know whether it failed, without filtering
kernel logs for IO errors. Or am I missing something?

Greetings,

Andres Freund



[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [Samba]     [Device Mapper]     [CEPH Development]

  Powered by Linux