[CCing a few people who may correct my errors; perhaps there are some improvements that are needed for the mmap() and msync() man pages ] Hello Heinrich, On 04/20/2014 12:28 PM, Heinrich Schuchardt wrote: > Hello Michael, > > when analyzing how the fanotify API interacts with mmap(2) I stumbled > over the following issues in the manpages: > > > The manpage of msync(2) says: > "msync() flushes changes made to the in-core copy of a file that was > mapped into memory using mmap(2) back to disk." > > "back to disk" implies that the file system is forced to actually write > to the hard disk, somewhat equivalent to invoking sync(1). Is that > guaranteed for all file systems? > > Not all file systems are necessarily disk based (e.g. davfs, tmpfs). > > So shouldn't we write: > "... back to the file system." Yes, that seems better to me. Done. > http://pubs.opengroup.org/onlinepubs/007904875/functions/msync.html > says > "... to permanent storage locations, if any," > > > The manpage of munmap(2) leaves it unclear, if copying back to the > filesystem is synchronous or asynchronous. In fact, the page says nearly nothing about whether it synchs at all. That is (I think) more or less deliberate. See below. > This bit of information is important, because, if munmap is > asynchronous, applications might want to call msync(,,MS_SYNC), before > calling munmap. If munmap is synchronous it might block until the file > system responds (think of waiting for a tape to be loaded, or a webdav > server to respond). > > > What happens to an unfinished prior asynchronous update by > mmap(,,MS_ASYNC) when munmap is called? I believe the answer is: On Linux, nothing special; the asynchronous update will still be done. (I'm not sure that anything needs to be said in the man page... But, if you have a good argument about why something should be said, I'm open to hearing it.) > Will munmap "invalidate other mappings of the same file (so that they > can be updated with the fresh values just written)" like > msync(,,MS_INVALIDATE) does? I don't believe there's any requirement that it does. (Again, I'm not sure that anything needs to be said in the man page... But, if you have a good argument...) So, here's how things are as I understand them. 1. In the bad old days (even on Linux, AFAIK, but that was in days before I looked closely at what goes on), the page cache and the buffer cache were not unified. That meant that a page from a file might both be in the buffer cache (because of file I/O syscalls) and in the page cache (because of mmap()). 2. In a non-unified cache system, pages can naturally get out of synch in the two locations. Before it had a unified cache, Linux used to jump some hoops to ensure that contents in the two locations remained consistent. 3. Nowadays Linux--like most (all?) UNIX systems--has a unified cache: file I/O, mmap(), and the paging system all use the same cache. If a file is mmap()-ed and also subject to file I?/, there will be only one copy of each file page in the cache. Ergo, the inconsistency problem goes away. 4. IIUC, the pieces like msync(MS_ASYNC) and msync(MS_INVALIDATE) exist only because of the bad old non-unified cache days. MS_INVALIDATE was a way of saying: make sure that writes to the file by other processes are visible in this mapping. msync() without the MS_INVALIDATE flags was a way of saying: make sure that read()s from the file see the changes made via this mapping. Using either MS_SYNC or MS_ASYNC was the way of saying: "I either want to wait until the file updates have been completed", or "please start the updates now, but I don't want to wait until they're completed". 5. On systems with a unified cache, msync(MS_INVALIDATE) is a no-op. (That is so on Linux.) 6. On Linux, MS_ASYNC is also a no-op. That's fine on a unified cache system. Filesystem I/O always sees a consistent view, and MS_ASYNC never undertook to give a guarantee about *when* the update would occur. (The Linux buffer cache logic will ensure that it is flushed out sometime in the near future.) 7. On Linux (and probably many other modern systems), the only call that has any real use is msync(MS_SYNC), meaning "flush the buffers *now*, and I want to wait for that to complete, so that I can then continue safe in the knowledge that my data has landed on a device". That's useful if we want insurance for our data in the event of a system crash. 8. POSIX make no mandate for a unified cache system. Thus, we have MS_ASYNC and MS_INVALIDATE in the standard, and the standard says nothing (AFAIK) about whether munmap() will flush data. On Linux (and probably most modern systems), we're fine. but portable applications that care about standards and nonunified caches need to use msync(). My advice: To ensure that the contents of a shared file mapping are written to the underlying file--even on bad old implementations--a call to msync() should be made before unmapping a mapping with munmap(). 9. The mmap() man page says this: MAP_SHARED Share this mapping. Updates to the mapping are vis‐ ible to other processes that map this file, and are carried through to the underlying file. The file may not actually be updated until msync(2) or mun‐ map() is called. I believe the piece "or munmap()" is misleading. It implies that munmap() must trigger a sync action. I don't think this is true. All that it is required to do is remove some range of pages from the process's virtual address space. I'm inclined to remove those words, but I'd like to see if any FS person has a correction to my understanding first. Cheers, Michael -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/ -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
