On Sat, Nov 28, 2020 at 10:10:37AM +0800, Peng Tao wrote: > On Tue, Oct 27, 2020 at 1:00 AM Alessio Balsini <balsini@xxxxxxxxxxx> wrote: > > > > This is the 10th version of the series. Please find the changelog at the > > bottom of this cover letter. > > > > Add support for file system passthrough read/write of files when enabled in > > userspace through the option FUSE_PASSTHROUGH. > > > > There are file systems based on FUSE that are intended to enforce special > > policies or trigger complicated decision makings at the file operations > > level. Android, for example, uses FUSE to enforce fine-grained access > > policies that also depend on the file contents. > > Sometimes it happens that at open or create time a file is identified as > > not requiring additional checks for consequent reads/writes, thus FUSE > > would simply act as a passive bridge between the process accessing the FUSE > > file system and the lower file system. Splicing and caching help reduce the > > FUSE overhead, but there are still read/write operations forwarded to the > > userspace FUSE daemon that could be avoided. > > > > This series has been inspired by the original patches from Nikhilesh Reddy, > > the idea and code of which has been elaborated and improved thanks to the > > community support. > > > > When the FUSE_PASSTHROUGH capability is enabled, the FUSE daemon may decide > > while handling the open/create operations, if the given file can be > > accessed in passthrough mode. This means that all the further read and > > write operations would be forwarded by the kernel directly to the lower > > file system using the VFS layer rather than to the FUSE daemon. > > All the requests other than reads or writes are still handled by the > > userspace FUSE daemon. > > This allows for improved performance on reads and writes, especially in the > > case of reads at random offsets, for which no (readahead) caching mechanism > > would help. > > Benchmarks show improved performance that is close to native file system > > access when doing massive manipulations on a single opened file, especially > > in the case of random reads, for which the bandwidth increased by almost 2X > > or sequential writes for which the improvement is close to 3X. > > > > The creation of this direct connection (passthrough) between FUSE file > > objects and file objects in the lower file system happens in a way that > > reminds of passing file descriptors via sockets: > > - a process requests the opening of a file handled by FUSE, so the kernel > > forwards the request to the FUSE daemon; > > - the FUSE daemon opens the target file in the lower file system, getting > > its file descriptor; > > - the FUSE daemon also decides according to its internal policies if > > passthrough can be enabled for that file, and, if so, can perform a > > FUSE_DEV_IOC_PASSTHROUGH_OPEN ioctl() on /dev/fuse, passing the file > > descriptor obtained at the previous step and the fuse_req unique > > identifier; > > - the kernel translates the file descriptor to the file pointer navigating > > through the opened files of the "current" process and temporarily stores > > it in the associated open/create fuse_req's passthrough_filp; > > - when the FUSE daemon has done with the request and it's time for the > > kernel to close it, it checks if the passthrough_filp is available and in > > case updates the additional field in the fuse_file owned by the process > > accessing the FUSE file system. > > From now on, all the read/write operations performed by that process will > > be redirected to the corresponding lower file system file by creating new > > VFS requests. > > Since the read/write operation to the lower file system is executed with > > the current process's credentials, it might happen that it does not have > > enough privileges to succeed. For this reason, the process temporarily > > receives the same credentials as the FUSE daemon, that are reverted as soon > > as the read/write operation completes, emulating the behavior of the > > request to be performed by the FUSE daemon itself. This solution has been > > inspired by the way overlayfs handles read/write operations. > > Asynchronous IO is supported as well, handled by creating separate AIO > > requests for the lower file system that will be internally tracked by FUSE, > > that intercepts and propagates their completion through an internal > > ki_completed callback similar to the current implementation of overlayfs. > > The ioctl() has been designed taking as a reference and trying to converge > > to the fuse2 implementation. For example, the fuse_passthrough_out data > > structure has extra fields that will allow for further extensions of the > > feature. > > > > > > Performance on SSD > > > > What follows has been performed with this change [V6] rebased on top of > > vanilla v5.8 Linux kernel, using a custom passthrough_hp FUSE daemon that > > enables pass-through for each file that is opened during both "open" and > > "create". Tests were run on an Intel Xeon E5-2678V3, 32GiB of RAM, with an > > ext4-formatted SSD as the lower file system, with no special tuning, e.g., > > all the involved processes are SCHED_OTHER, ondemand is the frequency > > governor with no frequency restrictions, and turbo-boost, as well as > > p-state, are active. This is because I noticed that, for such high-level > > benchmarks, results consistency was minimally affected by these features. > > The source code of the updated libfuse library and passthrough_hp is shared > > at the following repository: > > > > https://github.com/balsini/libfuse/tree/fuse-passthrough-stable-v.3.9.4 > The libfuse changes are not updated with the latest ioctl UAPI change yet. > > > * UAPI updated: ioctl() now returns an ID that will be used at > > open/create response time to reference the passthrough file > > Cheers, > Tao > -- > Into Sth. Rich & Strange Hi Tao, You are right, sorry, this is the correct branch that uses the FUSE_DEV_IOC_PASSTHROUGH_OPEN ioctl with the current patch set: https://github.com/balsini/libfuse/tree/fuse-passthrough-v10-linux-5.8-v.3.9.4 I think I'll stick to this libfuse branch naming pattern from now on. :) Thanks, Alessio
