On Mon, Oct 16, 2023 at 7:09 PM Amir Goldstein <amir73il@xxxxxxxxx> wrote: > > Miklos, > > I've shared several POC branches since the posting of v13 back in May > and played with several API choices. It is time to post v14. > > The API we converged to is server managed shared backing files that are > referenced by backing id plus per-file re-opened backing_file. > > This model looks coherent to me. I think that the example server [3] > demonstrates that this API is simple enough to work with. > > There is quite a bit of re-factored code in this version - I've actually > declared this common code as a new vfs subsystem [stackable filesystems] > in MAINTAINERS per Christian's request. > > The re-factored common code is based on overlayfs-next and Christian's > vfs.misc branch (for the backing_file changes). > > I am not posting performance numbers again. Alessio has already posted > performance numbers back in v12 and nothing has changed in this regard. > We are using a variant of v12 patches in production and the performance > improvement is very noticable. > > Bernd and Nikolaus have helped with improving running fstests on fuse > passthrough examples. > > I have ran the -g auto fstests with v14 patches with the example server. > Compared to the baseline test results with passthrough_hp, the backing > file passthrough passes several more test, mainly tests related to data > coherency, such as generic/451. FWIW, baseline passthrough_hp --nocache does pass generic/451. Not surprising considering that the test is for "Test data integrity when mixing buffered reads and asynchronous direct writes a file." > > The following tests are the only ones that pass on baseline passthtough_hp > and fail with my backing file passthrough example: > > generic/080 generic/120 generic/193 generic/215 generic/355 > > Those tests are failing because of missing mtime/atime/ctime updates Some more detailed analysis: generic/120 tests -o noatime and fails because atime is updated (on the backing file). This is a general FUSE issue and passthrough_hp --nocache fails the same test (i.e. it only passed because of attribute cache). generic/080, generic/215 both test for c/mtime updates after mapped writes. It is not surprising that backing file passthrough fails these tests - there is no "passthrough getattr" like overlayfs and there is no opportunity to invalidate the FUSE inode attribute cache. > in some use cases and failure to strip suid/sgid bits in some cases. The suid/sgid strip failures (generic/193, generic/355) were just a silly bug. Forgot to add file_remove_privs() in passthrough write. I now moved it from overlayfs into the common backing_file helpers. I don't think that the issue with mapped writes c/mtime update is a show stopper? Thanks, Amir. > > Changes from v13 [1]: > - rebase on 6.6-rc6 (and overlayfs and vfs next branches) > - server managed shared backing files without auto-close mode > - open a backing_file per fuse_file with fuse file's path and flags > - factor out common read/write/splice/mmap helpers from overlayfs > - factor out ioctl helpers > > [1] https://lore.kernel.org/r/20230519125705.598234-1-amir73il@xxxxxxxxx/ > [2] https://github.com/amir73il/linux/commits/fuse-backing-fd-v14 > [3] https://github.com/amir73il/libfuse/commits/fuse-backing-fd > > Amir Goldstein (12): > fs: prepare for stackable filesystems backing file helpers > fs: factor out backing_file_{read,write}_iter() helpers > fs: factor out backing_file_splice_{read,write}() helpers > fs: factor out backing_file_mmap() helper > fuse: factor out helper for FUSE_DEV_IOC_CLONE > fuse: introduce FUSE_PASSTHROUGH capability > fuse: pass optional backing_id in struct fuse_open_out > fuse: implement ioctls to manage backing files > fuse: implement read/write passthrough > fuse: implement splice_{read/write} passthrough > fuse: implement passthrough for mmap > fuse: implement passthrough for readdir > > MAINTAINERS | 9 + > fs/Kconfig | 4 + > fs/Makefile | 1 + > fs/backing-file.c | 319 ++++++++++++++++++++++++++++ > fs/fuse/Kconfig | 11 + > fs/fuse/Makefile | 1 + > fs/fuse/cuse.c | 3 +- > fs/fuse/dev.c | 98 ++++++--- > fs/fuse/dir.c | 2 +- > fs/fuse/file.c | 69 ++++-- > fs/fuse/fuse_i.h | 72 ++++++- > fs/fuse/inode.c | 25 +++ > fs/fuse/ioctl.c | 3 +- > fs/fuse/passthrough.c | 392 +++++++++++++++++++++++++++++++++++ > fs/fuse/readdir.c | 12 +- > fs/open.c | 38 ---- > fs/overlayfs/Kconfig | 1 + > fs/overlayfs/file.c | 246 ++++------------------ > fs/overlayfs/overlayfs.h | 8 +- > fs/overlayfs/super.c | 11 +- > include/linux/backing-file.h | 42 ++++ > include/linux/fs.h | 3 - > include/uapi/linux/fuse.h | 23 +- > 23 files changed, 1085 insertions(+), 308 deletions(-) > create mode 100644 fs/backing-file.c > create mode 100644 fs/fuse/passthrough.c > create mode 100644 include/linux/backing-file.h > > -- > 2.34.1 >
