Miklos, I've shared several POC branches since the posting of v13 back in May and played with several API choices. It is time to post v14. The API we converged to is server managed shared backing files that are referenced by backing id plus per-file re-opened backing_file. This model looks coherent to me. I think that the example server [3] demonstrates that this API is simple enough to work with. There is quite a bit of re-factored code in this version - I've actually declared this common code as a new vfs subsystem [stackable filesystems] in MAINTAINERS per Christian's request. The re-factored common code is based on overlayfs-next and Christian's vfs.misc branch (for the backing_file changes). I am not posting performance numbers again. Alessio has already posted performance numbers back in v12 and nothing has changed in this regard. We are using a variant of v12 patches in production and the performance improvement is very noticable. Bernd and Nikolaus have helped with improving running fstests on fuse passthrough examples. I have ran the -g auto fstests with v14 patches with the example server. Compared to the baseline test results with passthrough_hp, the backing file passthrough passes several more test, mainly tests related to data coherecy, such as generic/451. The following tests are the only ones that pass on baseline passthtough_hp and fail with my backing file passthrough example: generic/080 generic/120 generic/193 generic/215 generic/355 Those tests are failing because of missing mtime/atime/ctime updates in some use cases and failure to strip suid/sgid bits in some cases. The model of who is responsible for updating timestamps and stripping suid/sgid bits is not always clear when it comes to backing file passthrough. I tried to invalidate attr caches similar to how fuse read/write behaves, but it does not cover all cases correctly. Let me know what you think of this version and if you think there is anything else that we need to take care of before upstreaming. Thanks, Amir. Changes from v13 [1]: - rebase on 6.6-rc6 (and overlayfs and vfs next branches) - server managed shared backing files without auto-close mode - open a backing_file per fuse_file with fuse file's path and flags - factor out common read/write/splice/mmap helpers from overlayfs - factor out ioctl helpers [1] https://lore.kernel.org/r/20230519125705.598234-1-amir73il@xxxxxxxxx/ [2] https://github.com/amir73il/linux/commits/fuse-backing-fd-v14 [3] https://github.com/amir73il/libfuse/commits/fuse-backing-fd Amir Goldstein (12): fs: prepare for stackable filesystems backing file helpers fs: factor out backing_file_{read,write}_iter() helpers fs: factor out backing_file_splice_{read,write}() helpers fs: factor out backing_file_mmap() helper fuse: factor out helper for FUSE_DEV_IOC_CLONE fuse: introduce FUSE_PASSTHROUGH capability fuse: pass optional backing_id in struct fuse_open_out fuse: implement ioctls to manage backing files fuse: implement read/write passthrough fuse: implement splice_{read/write} passthrough fuse: implement passthrough for mmap fuse: implement passthrough for readdir MAINTAINERS | 9 + fs/Kconfig | 4 + fs/Makefile | 1 + fs/backing-file.c | 319 ++++++++++++++++++++++++++++ fs/fuse/Kconfig | 11 + fs/fuse/Makefile | 1 + fs/fuse/cuse.c | 3 +- fs/fuse/dev.c | 98 ++++++--- fs/fuse/dir.c | 2 +- fs/fuse/file.c | 69 ++++-- fs/fuse/fuse_i.h | 72 ++++++- fs/fuse/inode.c | 25 +++ fs/fuse/ioctl.c | 3 +- fs/fuse/passthrough.c | 392 +++++++++++++++++++++++++++++++++++ fs/fuse/readdir.c | 12 +- fs/open.c | 38 ---- fs/overlayfs/Kconfig | 1 + fs/overlayfs/file.c | 246 ++++------------------ fs/overlayfs/overlayfs.h | 8 +- fs/overlayfs/super.c | 11 +- include/linux/backing-file.h | 42 ++++ include/linux/fs.h | 3 - include/uapi/linux/fuse.h | 23 +- 23 files changed, 1085 insertions(+), 308 deletions(-) create mode 100644 fs/backing-file.c create mode 100644 fs/fuse/passthrough.c create mode 100644 include/linux/backing-file.h -- 2.34.1
