On Mon, 2021-11-29 at 14:36 +0100, Christian Brauner wrote: > Hey Jeff, > Hey everyone, > > I have a question about the relationship between the cephfs kernel > client/filesystem and the code in [1]. > > I'm working on patches to let cephfs support mapped mounts. I've got > everything in place and I'm able to run the full mapped mounts/vfs > testsuite against a ceph cluster. So things like: > > mount -t ceph 1.2.3.4:6789:/ /mnt -o name=admin,secret=wootwoot > mount-idmapped --map-mount b:1000:0:1 /mnt /opt > Sounds like a cool feature. > (Where in /opt all inodes owned by id 1000 are owned by id 0 and callers > with id 0 write files to disk as id 1000.) > > My next step was to read to the ceph code in github.com/ceph/ceph to > make sure that I don't miss any corner cases that wouldn't work > correctly with mapped mounts after the relevant kernel changes. > > So I read through [1] and I could use some guidance if possible. The > code in there to me reads like it is a full implementation of cephfs in > userspace and I'm wondering how this related to the cephfs kernel > client/filesystem. > > The client code in [1] seems to implement full-blown userspace path > lookup and permission checking that seems to reimplement the vfs layer > in userspace. > A lot of the helpers such as ceph_ll_mknod() in addition expose a > UserPerms struct that seems to be settable by the caller which is used > during lookup. > > How does this permission/path lookup code relate to the cephfs kernel > client/filesystem? Do they interact with each other in any way? > > Thanks for taking the time to answer! > Christian > > [1]: https://github.com/ceph/ceph/tree/master/src/client/* There are really two "official" cephfs clients: the userland client which is (mostly) represented by src/client in the userland ceph tree, and the kernel cephfs client (aka the kclient), which is (mostly) fs/ceph and net/ceph in the Linux kernel tree. The userland code usually gets compiled into libcephfs.so and is the backend engine for ceph-fuse, the nfs-ganesha FSAL driver, and the samba vfs backend driver, as well as a bunch of testcases in the ceph tree. libcephfs communicates directly with the cluster without interacting with any of the kernel ceph code, so ceph-fuse, ganesha, etc. do use the userland code for lookups and permission checks. Technically, the kclient is the reimplentation. Most of the ceph and libceph code in the kernel has been copied/reimplemented from the ceph userland code. There is no real dependency between the kclient and libcephfs. Maintaining the kclient is a continual game of catchup, unfortunately, though in some cases we do leading edge client development there instead and play catchup in the userland client. Cheers, -- Jeff Layton <jlayton@xxxxxxxxxx>
