Re: OverlayFS with Cephfs to mount a snapshot read/write

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Luis,

I gave RHEL 8 and kernel 4.18 a try and it's working perfectly! \o/

Same commands, same mount options. Does anyone know why and if there's any chances I can have this working with CentOS/RHEL 7 and 3.10 kernel?

Best regards,

Frédéric.

Le 09/11/2020 à 15:04, Frédéric Nass a écrit :
Hi Luis,

Thanks for your help. Sorry I forgot about the kernel details. This is latest RHEL 7.9.

~/ uname -r
3.10.0-1160.2.2.el7.x86_64

~/ grep CONFIG_TMPFS_XATTR /boot/config-3.10.0-1160.2.2.el7.x86_64
CONFIG_TMPFS_XATTR=y

upper directory /upperdir is using xattrs

~/ ls -l /dev/mapper/vg0-racine
lrwxrwxrwx 1 root root 7  6 mars   2020 /dev/mapper/vg0-racine -> ../dm-0

~/ cat /proc/fs/ext4/dm-0/options | grep xattr
user_xattr

~/ setfattr -n user.name -v upperdir /upperdir

~/ getfattr -n user.name /upperdir
getfattr: Suppression des « / » en tête des chemins absolus
# file: upperdir
user.name="upperdir"

Are you able to modify the content of a snapshot directory using overlayfs on your side?

Frédéric.

Le 09/11/2020 à 12:39, Luis Henriques a écrit :
Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx> writes:

Hello,

I would like to use a cephfs snapshot as a read/write volume without having to clone it first as the cloning operation is - if I'm not mistaken - still
inefficient as of now. This is for a data restore use case with Moodle
application needing a writable data directory to start.

The idea that came to mind was to use overlayFS with cephfs set up as a
read-only lower layer and a writable local directory set up as an upper
layer. With this set up, any modifications to the read-only .snap/testsnap directory would normally go to the upper directory making the snapshot directory somehow writable to the Moodle application. While this works fine when a local read-only filesystem is set up as the lower layer, it fails when cephfs is set up as the lower layer. Any modifications to the .snap/testsnap tree in the
/cephfs-snap directory fails with an "Operation not supported".

$ mkdir /cephfs /upperdir /workdir /cephfs-snap

$ mount -t ceph 100.74.191.129:/volumes/group1/subvolume1/ /cephfs -o
name=admin,secretfile=/etc/ceph/admin.secret

$ mount -t overlay overlay -o
redirect_dir=on,lowerdir=/cephfs/.snap/testsnap,upperdir=/upperdir,workdir=/workdir
/cephfs-snap

$ ls /cephfs-snap
usr

$ touch /cephfs-snap/foo.txt            <---- writing outside the lowerdir
succeeds

$ ls /cephfs-snap
foo.txt  usr

$ ls /usr/etc

$ touch /cephfs-snap/usr/etc/foo            <---- writing inside the lowerdir
fails
touch: impossible de faire un touch « /cephfs-snap/usr/etc/foo »: Opération non
supportée

I tried to mount the whole cephfs tree read-only (-o ro), tried to disable ACLs (-o noacl) as seen here [1] but of no help. Mounting with ceph-fuse didn't help either. There's been a recent discussion about this here [2] between Greg and
Robert but with no real solution.
I just commented on that bug tracker and, although I'm not really 100%
sure, I suspect that the tmpfs on that system has been compiled without
xattr support.

Did someone manage to do this?
I couldn't reproduce you're problem.  Is it possible that your upper dir
doesn't support xatttrs either?  Also, kernel client details would help.

Cheers,
--
Luis

Regards,

Frédéric.

[1] https://blog.fai-project.org/posts/overlayfs/
[2] https://tracker.ceph.com/issues/44821
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux