On Tue, 2019-08-06 at 01:25 -0700, Christoph Hellwig wrote: > On Mon, Aug 05, 2019 at 04:05:01PM -0400, Jeff Layton wrote: > > Instead, borrow the scheme used by nfs.ko. Buffered writes take the > > i_rwsem exclusively, but buffered reads and all O_DIRECT requests > > take a shared lock, allowing them to run in parallel. > > Note that you'll still need an exclusive lock to guard against cache > invalidation for direct writes. And instead of adding a new lock you > might want to look at the i_rwsem based scheme in XFS (whіch also > happens to be where O_DIRECT originally came from). Thanks, Christoph. That part of the patch description is unclear. I'll fix that up, but this patch does ensure that no buffered I/O can take place while any direct I/O is in flight. Only operations of the same "flavor" can run in parallel. Note that we _do_ use the i_rwsem here, but there is also an additional per-inode flag that handles the buffered read/direct I/O exclusion. I did take a look at the xfs_ilock* code this morning. That's quite a bit more complex than this. It's possible that ceph doesn't serialize mmap I/O and O_DIRECT properly. I'll have to look over xfstests and see whether there is a good test for that as well and whether it passes. Cheers, -- Jeff Layton <jlayton@xxxxxxxxxx>
