Re: OverlayFS with Cephfs to mount a snapshot read/write

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx> writes:

> Hello,
>
> I would like to use a cephfs snapshot as a read/write volume without having to
> clone it first as the cloning operation is - if I'm not mistaken - still
> inefficient as of now. This is for a data restore use case with Moodle
> application needing a writable data directory to start.
>
> The idea that came to mind was to use overlayFS with cephfs set up as a
> read-only lower layer and a writable local directory set up as an upper 
> layer. With this set up, any modifications to the read-only .snap/testsnap
> directory would normally go to the upper directory making the snapshot directory
> somehow writable to the Moodle application. While this works fine when a local
> read-only filesystem is set up as the lower layer, it fails when cephfs is set
> up as the lower layer. Any modifications to the .snap/testsnap tree in the
> /cephfs-snap directory fails with an "Operation not supported".
>
> $ mkdir /cephfs /upperdir /workdir /cephfs-snap
>
> $ mount -t ceph 100.74.191.129:/volumes/group1/subvolume1/ /cephfs -o
> name=admin,secretfile=/etc/ceph/admin.secret
>
> $ mount -t overlay overlay -o
> redirect_dir=on,lowerdir=/cephfs/.snap/testsnap,upperdir=/upperdir,workdir=/workdir 
> /cephfs-snap
>
> $ ls /cephfs-snap
> usr
>
> $ touch /cephfs-snap/foo.txt            <---- writing outside the lowerdir
> succeeds
>
> $ ls /cephfs-snap
> foo.txt  usr
>
> $ ls /usr/etc
>
> $ touch /cephfs-snap/usr/etc/foo            <---- writing inside the lowerdir
> fails
> touch: impossible de faire un touch « /cephfs-snap/usr/etc/foo »: Opération non
> supportée
>
> I tried to mount the whole cephfs tree read-only (-o ro), tried to disable ACLs
> (-o noacl) as seen here [1] but of no help. Mounting with ceph-fuse didn't help
> either. There's been a recent discussion about this here [2] between Greg and
> Robert but with no real solution.

I just commented on that bug tracker and, although I'm not really 100%
sure, I suspect that the tmpfs on that system has been compiled without
xattr support.

> Did someone manage to do this?

I couldn't reproduce you're problem.  Is it possible that your upper dir
doesn't support xatttrs either?  Also, kernel client details would help.

Cheers,
--
Luis

>
> Regards,
>
> Frédéric.
>
> [1] https://blog.fai-project.org/posts/overlayfs/
> [2] https://tracker.ceph.com/issues/44821
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux