Hello Gürkan, Indeed, by design, snapshots are only possible at the top of a subvolume (which is invisible in a pod). This is deliberate, as otherwise they would become a mechanism for quota evasion. You can try CSI-level snapshots, which use cephfs-level snapshots under the hood. Or, you can even try giving your pod a permission to talk to the CSI driver, which would then make snapshots on its behalf, but I don't know offhand how to do it. In any case, please also try asking in Kubernetes forums. On Ceph side, unfortunately, everything works as intended. On Sat, Mar 1, 2025 at 2:58 AM <ceph@xxxxxxxxx> wrote: > > Hello all, > > We're getting a "Operation not permitted" error while trying to create a snapshot on the client. It is somehow related to previously-asked Pacific issue mentioned here: https://www.spinics.net/lists/ceph-users/msg67908.html > > We are on squid (19.2.1) and the given workaround seems to be only temporary. Anyway, here are some details about the issue: > > Cephx side: > > ``` > client.mount-update-production > key: *** > caps: [mds] allow rws fsname=production-cephfs path=/volumes/_nogroup/update-production > caps: [mon] allow r fsname=production-cephfs > caps: [osd] allow rw tag cephfs data=production-cephfs > ``` > > FS flags: > ``` > ~> sudo ceph fs lsflags production-cephfs > joinable allow_snaps allow_multimds_snaps > ``` > > I got the path via following command: > ``` > ~> sudo ceph fs subvolume getpath production-cephfs update-production > /volumes/_nogroup/update-production/e155ba30-d201-4653-be3f-86533324ee5f > ``` > > And mounted to client with mentioned key: > ``` > ~> mount | grep ceph > mount-update-production@00000000-0000-0000-0000-000000000000.production-cephfs=/volumes/_nogroup/update-production/e155ba30-d201-4653-be3f-86533324ee5f on /opt/aptly/.aptly type ceph (rw,noatime,name=mount-update-production,secret=<hidden>,acl,mon_addr=[multiple mon addresses here]) > ``` > > Everything works flawlessly. Yet, when we want to get a snapshot: > ``` > /opt/aptly/.aptly/pool/9c/0b ~> sudo mkdir .snap/test > mkdir: cannot create directory ‘.snap/test’: Operation not permitted > ``` > > If I mount the whole FS with admin rights and run `setfattr -n ceph.dir.subvolume -v 0` for _all_ directories of following path one by one: /volumes/_nogroup/update-production/e155ba30-d201-4653-be3f-86533324ee5f, then it is solving the issue. But it does not last; not sure what is triggering it again, but if I try snapshotting anything next day, getting the same "Operation not permitted" error again. > > Is there something I can try, or am I missing something obvious? > > Thanks, > Gürkan > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx -- Alexander Patrakov _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |