On Thu, Feb 1, 2024 at 6:33 PM Amir Goldstein <amir73il@xxxxxxxxx> wrote: > > On Thu, Feb 1, 2024 at 4:47 PM Miklos Szeredi <miklos@xxxxxxxxxx> wrote: > > > > On Thu, 1 Feb 2024 at 00:09, Bernd Schubert <bschubert@xxxxxxx> wrote: > > > > > > From: Amir Goldstein <amir73il@xxxxxxxxx> > > > > > > The fuse inode io mode is determined by the mode of its open files/mmaps > > > and parallel dio. > > > > > > - caching io mode - files open in caching mode or mmap on direct_io file > > > - direct io mode - no files open in caching mode and no files mmaped > > > - parallel dio mode - direct io mode with parallel dio in progress > > > > Specifically if iocachectr is: > > > > > 0 -> caching io > > == 0 -> direct io > > < 0 -> parallel io > > > > > > > > We use a new FOPEN_CACHE_IO flag to explicitly mark a file that was open > > > in caching mode. > > > > This is really confusing. FOPEN_CACHE_IO is apparently an internally > > used flag, but it's defined on the userspace API. > > > > a) what is the meaning of this flag on the external API? > > b) what is the purpose of this flag internally? > > The purpose is to annotate the state of direct io file that was mmaped > as FOPEN_DIRECT_IO | FOPEN_CACHE_IO. > An fd like this puts inode in caching mode and its release may get inode > out of caching mode. > > I did not manage to do refcoutning with fuse_vma_close(), because those > calls are not balances with fuse_file_mmap() calls. > > The first mmap() of an FOPEN_DIRECT_IO file may incur wait for completion > of parallel dio. > > The only use of exporting FOPEN_CACHE_IO to the server is that it could > force incurring this wait at open() time instead of mmap() time. > Miklos, I have played with this rebranding of FOPEN_CACHE_IO => FOPEN_NO_PARALLEL_DIO_WRITES The meaning of the rebranded flag is: Prevent parallel dio on inode for as long as this file is kept open. The io modes code sets this flag implicitly on the first shared mmap. Let me know if this makes the external flag easier to swallow. Of course I can make this flag internal and not and FOPEN_ flag at all, but IMO, the code is easier to understand when the type of iocachectl refcount held by any file is specified by its FOPEN_ flags. Let me know what you think. Thanks, Amir. https://github.com/amir73il/linux/commits/fuse_io_mode-wip/ --- a/include/uapi/linux/fuse.h +++ b/include/uapi/linux/fuse.h @@ -353,7 +353,7 @@ struct fuse_file_lock { * FOPEN_STREAM: the file is stream-like (no file position at all) * FOPEN_NOFLUSH: don't flush data cache on close (unless FUSE_WRITEBACK_CACHE) * FOPEN_PARALLEL_DIRECT_WRITES: Allow concurrent direct writes on the same inode - * FOPEN_CACHE_IO: using cache for this open file (incl. mmap on direct_io) + * FOPEN_NO_PARALLEL_DIO_WRITES: Deny concurrent direct writes on the same inode */ #define FOPEN_DIRECT_IO (1 << 0) #define FOPEN_KEEP_CACHE (1 << 1) @@ -362,7 +362,7 @@ struct fuse_file_lock { #define FOPEN_STREAM (1 << 4) #define FOPEN_NOFLUSH (1 << 5) #define FOPEN_PARALLEL_DIRECT_WRITES (1 << 6) -#define FOPEN_CACHE_IO (1 << 7) +#define FOPEN_NO_PARALLEL_DIO_WRITES (1 << 7) ... - /* Set explicit FOPEN_CACHE_IO flag for file open in caching mode */ - if (!fuse_file_is_direct_io(file)) - ff->open_flags |= FOPEN_CACHE_IO; + /* + * FOPEN_CACHE_IO is an internal flag that is set on file not open in + * direct io mode and it cannot be set explicitly by the server. + * FOPEN_NO_PARALLEL_DIO_WRITES is set on file open in caching mode and + * is not allowed together with FOPEN_PARALLEL_DIRECT_WRITES. + * This includes a file open with O_DIRECT, but server did not specify + * FOPEN_DIRECT_IO. In this case, a later fcntl() could remove O_DIRECT, + * so we put the inode in caching mode to prevent parallel dio. + * FOPEN_PARALLEL_DIRECT_WRITES requires FOPEN_DIRECT_IO. + */ + if (ff->open_flags & FOPEN_NO_PARALLEL_DIO_WRITES) { + if (ff->open_flags & FOPEN_PARALLEL_DIRECT_WRITES) + goto fail; + } else if (!(ff->open_flags & FOPEN_DIRECT_IO)) { + ff->open_flags |= FOPEN_NO_PARALLEL_DIO_WRITES; + ff->open_flags &= ~FOPEN_PARALLEL_DIRECT_WRITES; + }
