fsync notes

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello,

the fsync(2) man page currently [1] doesn't contain many details on the
semantics of a succeeding/failing fsync().

Perhaps it makes sense to explicitly state some implications/usage
notes/pitfalls in the fsync man page. Points that could be included:

- An fsync() call returning success implies that all modifications since
  the previous fsync() call (or since the file-descriptor was opened)
  reached the disk.
- A failed fsync() call implies that some or all modifications since the
  previous fsync() call (or since the file-descriptor was openend) are
  possibly lost.
- A successful fsync() after a failed fsync() call doesn't imply that
  any previous failed modifications now have succeeded. In other words,
  in order to retry failed modifications it isn't sufficient to invoke
  fsync() again. Instead one has to re-open the file-descriptor,
  assess what modifications actually succeeded, apply the remaining
  ones (and/or rolling back some) before calling fsync() again.

See also this Stackoverflow question where a Postgresql developer seeks
for clarification regarding the last point:

https://stackoverflow.com/questions/42434872/writing-programs-to-cope-with-i-o-errors-causing-lost-writes-on-linux

Related LWN article that gives some context:

https://lwn.net/Articles/752063/

Also, perhaps it would make sense to add an example (or two) to the man
page that demonstrates correct fsync usage. For example:

Achieve durability before closing after a (possibly newly created) file:

    fd = open(filename, O_WRONLY | O_CREAT | O_TRUNC, 0666)
    // error checking
    // multiple writes
    // ...
    write(fd, buffer, buffer_size)
    // error checking
    // ...
    fsync(fd)
    // error checking
    close(fd)
    // error checking
    dfd = open(parent_directory_of_filename, O_RDONLY)
    // error checking
    fsync(dfd)
    // error checking
    close(dfd)
    // error checking

Atomically create/replace a - say - config file:
    
    fd = open(tmp_filename, O_WRONLY | O_CREAT | O_TRUNC, 0666)
    // error checking
    // ...
    // ... write the new config file via fd
    // ...
    fsync(fd)
    // error checking
    close(fd)
    // error checking
    dfd = open(parent_directory_of_filename, O_RDONLY)
    // error checking
    rename(tmp_filename, filename)
    // error checking
    fsync(dfd)
    // error checking
    close(dfd)
    // error checking

If tmp_filename is located in another directory and if it's important
that tmp_filename disappears atomically, as well, then one has to call
fsync() on the source-directory, too.

Best regards
Georg

[1]: http://man7.org/linux/man-pages/man2/fsync.2.html
-- 
'It is not only other programmer who make mistakes. It is us.
Programming is really hard.' (A. Stepanov)
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [Kernel Documentation]     [Netdev]     [Linux Ethernet Bridging]     [Linux Wireless]     [Kernel Newbies]     [Security]     [Linux for Hams]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux RAID]     [Linux Admin]     [Samba]

  Powered by Linux