All, I'm not sure if this is relevant here, but I recently tried to use OverlayFS with an NFS share. It wouldn't work because NFS does not present to the kernel as a block device. OverlayFS requires a block device abstraction. If CephFS doesn't present as a block device you won't get it to work. Hope this helps. -Dave -- Dave Hall Binghamton University kdhall@xxxxxxxxxxxxxx 607-760-2328 (Cell) 607-777-4641 (Office) On Mon, Nov 9, 2020 at 9:08 AM Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx> wrote: > Hi Luis, > > Thanks for your help. Sorry I forgot about the kernel details. This is > latest RHEL 7.9. > > ~/ uname -r > 3.10.0-1160.2.2.el7.x86_64 > > ~/ grep CONFIG_TMPFS_XATTR /boot/config-3.10.0-1160.2.2.el7.x86_64 > CONFIG_TMPFS_XATTR=y > > upper directory /upperdir is using xattrs > > ~/ ls -l /dev/mapper/vg0-racine > lrwxrwxrwx 1 root root 7 6 mars 2020 /dev/mapper/vg0-racine -> ../dm-0 > > ~/ cat /proc/fs/ext4/dm-0/options | grep xattr > user_xattr > > ~/ setfattr -n user.name -v upperdir /upperdir > > ~/ getfattr -n user.name /upperdir > getfattr: Suppression des « / » en tête des chemins absolus > # file: upperdir > user.name="upperdir" > > Are you able to modify the content of a snapshot directory using > overlayfs on your side? > > Frédéric. > > Le 09/11/2020 à 12:39, Luis Henriques a écrit : > > Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx> writes: > > > >> Hello, > >> > >> I would like to use a cephfs snapshot as a read/write volume without > having to > >> clone it first as the cloning operation is - if I'm not mistaken - still > >> inefficient as of now. This is for a data restore use case with Moodle > >> application needing a writable data directory to start. > >> > >> The idea that came to mind was to use overlayFS with cephfs set up as a > >> read-only lower layer and a writable local directory set up as an upper > >> layer. With this set up, any modifications to the read-only > .snap/testsnap > >> directory would normally go to the upper directory making the snapshot > directory > >> somehow writable to the Moodle application. While this works fine when > a local > >> read-only filesystem is set up as the lower layer, it fails when cephfs > is set > >> up as the lower layer. Any modifications to the .snap/testsnap tree in > the > >> /cephfs-snap directory fails with an "Operation not supported". > >> > >> $ mkdir /cephfs /upperdir /workdir /cephfs-snap > >> > >> $ mount -t ceph 100.74.191.129:/volumes/group1/subvolume1/ /cephfs -o > >> name=admin,secretfile=/etc/ceph/admin.secret > >> > >> $ mount -t overlay overlay -o > >> > redirect_dir=on,lowerdir=/cephfs/.snap/testsnap,upperdir=/upperdir,workdir=/workdir > >> /cephfs-snap > >> > >> $ ls /cephfs-snap > >> usr > >> > >> $ touch /cephfs-snap/foo.txt <---- writing outside the > lowerdir > >> succeeds > >> > >> $ ls /cephfs-snap > >> foo.txt usr > >> > >> $ ls /usr/etc > >> > >> $ touch /cephfs-snap/usr/etc/foo <---- writing inside the > lowerdir > >> fails > >> touch: impossible de faire un touch « /cephfs-snap/usr/etc/foo »: > Opération non > >> supportée > >> > >> I tried to mount the whole cephfs tree read-only (-o ro), tried to > disable ACLs > >> (-o noacl) as seen here [1] but of no help. Mounting with ceph-fuse > didn't help > >> either. There's been a recent discussion about this here [2] between > Greg and > >> Robert but with no real solution. > > I just commented on that bug tracker and, although I'm not really 100% > > sure, I suspect that the tmpfs on that system has been compiled without > > xattr support. > > > >> Did someone manage to do this? > > I couldn't reproduce you're problem. Is it possible that your upper dir > > doesn't support xatttrs either? Also, kernel client details would help. > > > > Cheers, > > -- > > Luis > > > >> Regards, > >> > >> Frédéric. > >> > >> [1] https://blog.fai-project.org/posts/overlayfs/ > >> [2] https://tracker.ceph.com/issues/44821 > >> _______________________________________________ > >> ceph-users mailing list -- ceph-users@xxxxxxx > >> To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |