Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx> writes:
> Hi Luis,
>
> Thanks for your help. Sorry I forgot about the kernel details. This is latest
> RHEL 7.9.
>
> ~/ uname -r
> 3.10.0-1160.2.2.el7.x86_64
>
> ~/ grep CONFIG_TMPFS_XATTR /boot/config-3.10.0-1160.2.2.el7.x86_64
> CONFIG_TMPFS_XATTR=y
>
> upper directory /upperdir is using xattrs
>
> ~/ ls -l /dev/mapper/vg0-racine
> lrwxrwxrwx 1 root root 7 6 mars 2020 /dev/mapper/vg0-racine -> ../dm-0
>
> ~/ cat /proc/fs/ext4/dm-0/options | grep xattr
> user_xattr
>
> ~/ setfattr -n user.name -v upperdir /upperdir
>
> ~/ getfattr -n user.name /upperdir
> getfattr: Suppression des « / » en tête des chemins absolus
> # file: upperdir
> user.name="upperdir"
>
> Are you able to modify the content of a snapshot directory using overlayfs on
> your side?
[ Cc'ing Jeff ]
Yes, I'm able to do that using a *recent* kernel. I got curious and after
some digging I managed to reproduce the issue with kernel 5.3. The
culprit was commit e09580b343aa ("ceph: don't list vxattrs in
listxattr()"), in 5.4.
Getting a bit more into the whole rabbit hole, it looks like
ovl_copy_xattr() will try to copy all the ceph-related vxattrs. And that
won't work (for ex. for ceph.dir.entries).
Can you try cherry-picking this commit into your kernel to see if that
fixes it for you?
Cheers,
--
Luis
> Frédéric.
>
> Le 09/11/2020 à 12:39, Luis Henriques a écrit :
>> Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx> writes:
>>
>>> Hello,
>>>
>>> I would like to use a cephfs snapshot as a read/write volume without having to
>>> clone it first as the cloning operation is - if I'm not mistaken - still
>>> inefficient as of now. This is for a data restore use case with Moodle
>>> application needing a writable data directory to start.
>>>
>>> The idea that came to mind was to use overlayFS with cephfs set up as a
>>> read-only lower layer and a writable local directory set up as an upper
>>> layer. With this set up, any modifications to the read-only .snap/testsnap
>>> directory would normally go to the upper directory making the snapshot directory
>>> somehow writable to the Moodle application. While this works fine when a local
>>> read-only filesystem is set up as the lower layer, it fails when cephfs is set
>>> up as the lower layer. Any modifications to the .snap/testsnap tree in the
>>> /cephfs-snap directory fails with an "Operation not supported".
>>>
>>> $ mkdir /cephfs /upperdir /workdir /cephfs-snap
>>>
>>> $ mount -t ceph 100.74.191.129:/volumes/group1/subvolume1/ /cephfs -o
>>> name=admin,secretfile=/etc/ceph/admin.secret
>>>
>>> $ mount -t overlay overlay -o
>>> redirect_dir=on,lowerdir=/cephfs/.snap/testsnap,upperdir=/upperdir,workdir=/workdir
>>> /cephfs-snap
>>>
>>> $ ls /cephfs-snap
>>> usr
>>>
>>> $ touch /cephfs-snap/foo.txt <---- writing outside the lowerdir
>>> succeeds
>>>
>>> $ ls /cephfs-snap
>>> foo.txt usr
>>>
>>> $ ls /usr/etc
>>>
>>> $ touch /cephfs-snap/usr/etc/foo <---- writing inside the lowerdir
>>> fails
>>> touch: impossible de faire un touch « /cephfs-snap/usr/etc/foo »: Opération non
>>> supportée
>>>
>>> I tried to mount the whole cephfs tree read-only (-o ro), tried to disable ACLs
>>> (-o noacl) as seen here [1] but of no help. Mounting with ceph-fuse didn't help
>>> either. There's been a recent discussion about this here [2] between Greg and
>>> Robert but with no real solution.
>> I just commented on that bug tracker and, although I'm not really 100%
>> sure, I suspect that the tmpfs on that system has been compiled without
>> xattr support.
>>
>>> Did someone manage to do this?
>> I couldn't reproduce you're problem. Is it possible that your upper dir
>> doesn't support xatttrs either? Also, kernel client details would help.
>>
>> Cheers,
>> --
>> Luis
>>
>>> Regards,
>>>
>>> Frédéric.
>>>
>>> [1] https://blog.fai-project.org/posts/overlayfs/
>>> [2] https://tracker.ceph.com/issues/44821
>>> _______________________________________________
>>> ceph-users mailing list -- ceph-users@xxxxxxx
>>> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
