[PATCH v14 00/12] FUSE passthrough for file io

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Miklos,

I've shared several POC branches since the posting of v13 back in May
and played with several API choices. It is time to post v14.

The API we converged to is server managed shared backing files that are
referenced by backing id plus per-file re-opened backing_file.

This model looks coherent to me. I think that the example server [3]
demonstrates that this API is simple enough to work with.

There is quite a bit of re-factored code in this version - I've actually
declared this common code as a new vfs subsystem [stackable filesystems]
in MAINTAINERS per Christian's request.

The re-factored common code is based on overlayfs-next and Christian's
vfs.misc branch (for the backing_file changes).

I am not posting performance numbers again. Alessio has already posted
performance numbers back in v12 and nothing has changed in this regard.
We are using a variant of v12 patches in production and the performance
improvement is very noticable.

Bernd and Nikolaus have helped with improving running fstests on fuse
passthrough examples.

I have ran the -g auto fstests with v14 patches with the example server.
Compared to the baseline test results with passthrough_hp, the backing
file passthrough passes several more test, mainly tests related to data
coherecy, such as generic/451.

The following tests are the only ones that pass on baseline passthtough_hp
and fail with my backing file passthrough example:

  generic/080 generic/120 generic/193 generic/215 generic/355

Those tests are failing because of missing mtime/atime/ctime updates
in some use cases and failure to strip suid/sgid bits in some cases.

The model of who is responsible for updating timestamps and stripping
suid/sgid bits is not always clear when it comes to backing file
passthrough. I tried to invalidate attr caches similar to how fuse
read/write behaves, but it does not cover all cases correctly.

Let me know what you think of this version and if you think there is
anything else that we need to take care of before upstreaming.

Thanks,
Amir.

Changes from v13 [1]:
- rebase on 6.6-rc6 (and overlayfs and vfs next branches)
- server managed shared backing files without auto-close mode
- open a backing_file per fuse_file with fuse file's path and flags
- factor out common read/write/splice/mmap helpers from overlayfs
- factor out ioctl helpers

[1] https://lore.kernel.org/r/20230519125705.598234-1-amir73il@xxxxxxxxx/
[2] https://github.com/amir73il/linux/commits/fuse-backing-fd-v14
[3] https://github.com/amir73il/libfuse/commits/fuse-backing-fd

Amir Goldstein (12):
  fs: prepare for stackable filesystems backing file helpers
  fs: factor out backing_file_{read,write}_iter() helpers
  fs: factor out backing_file_splice_{read,write}() helpers
  fs: factor out backing_file_mmap() helper
  fuse: factor out helper for FUSE_DEV_IOC_CLONE
  fuse: introduce FUSE_PASSTHROUGH capability
  fuse: pass optional backing_id in struct fuse_open_out
  fuse: implement ioctls to manage backing files
  fuse: implement read/write passthrough
  fuse: implement splice_{read/write} passthrough
  fuse: implement passthrough for mmap
  fuse: implement passthrough for readdir

 MAINTAINERS                  |   9 +
 fs/Kconfig                   |   4 +
 fs/Makefile                  |   1 +
 fs/backing-file.c            | 319 ++++++++++++++++++++++++++++
 fs/fuse/Kconfig              |  11 +
 fs/fuse/Makefile             |   1 +
 fs/fuse/cuse.c               |   3 +-
 fs/fuse/dev.c                |  98 ++++++---
 fs/fuse/dir.c                |   2 +-
 fs/fuse/file.c               |  69 ++++--
 fs/fuse/fuse_i.h             |  72 ++++++-
 fs/fuse/inode.c              |  25 +++
 fs/fuse/ioctl.c              |   3 +-
 fs/fuse/passthrough.c        | 392 +++++++++++++++++++++++++++++++++++
 fs/fuse/readdir.c            |  12 +-
 fs/open.c                    |  38 ----
 fs/overlayfs/Kconfig         |   1 +
 fs/overlayfs/file.c          | 246 ++++------------------
 fs/overlayfs/overlayfs.h     |   8 +-
 fs/overlayfs/super.c         |  11 +-
 include/linux/backing-file.h |  42 ++++
 include/linux/fs.h           |   3 -
 include/uapi/linux/fuse.h    |  23 +-
 23 files changed, 1085 insertions(+), 308 deletions(-)
 create mode 100644 fs/backing-file.c
 create mode 100644 fs/fuse/passthrough.c
 create mode 100644 include/linux/backing-file.h

-- 
2.34.1





[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [NTFS 3]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [NTFS 3]     [Samba]     [Device Mapper]     [CEPH Development]

  Powered by Linux