Cephfs cannot create snapshots in subdirs of / with mds = "allow *"

[David Prude <david@xxxxxxxxxxxxxxxx>]

Hello,

   We have a cluster initially installed with pacific 16.2.5 within
which we have a single cephfs volume created as such:

          ceph fs volume create dncephfs

We have a client.admin client:

          [client.admin]
             key = REDACTED
             caps mds = "allow *"
             caps mgr = "allow *"
             caps mon = "allow *"
             caps osd = "allow *"

We have dncephfs mounted as this client:

          mount | grep dncephfs
10.30.0.145:6789,10.30.0.146:6789,10.30.0.147:6789,10.30.0.148:6789,10.30.0.149:6789:/
on /mnt/dncephfs type ceph (rw,noatime,name=admin,secret=<hidden>,acl)

We can create and delete snapshots at the / of this cephfs:

          root@ceph-01:~# cd /mnt/dncephfs/
          root@ceph-01:/mnt/dncephfs# cd .snap
          root@ceph-01:/mnt/dncephfs/.snap# mkdir testsnapshot1

And confirm that the snapshot is seen by the mds:

         root@ceph-01:/# ceph daemon mds.dncephfs.ceph-01.zmlkdd dump snaps
          {
              "last_created": 14,
              "last_destroyed": 13,
              "snaps": [
                  {
                      "snapid": 14,
                      "ino": 1,
                      "stamp": "2021-08-20T11:05:17.129353+0000",
                      "name": "testsnapshot1",
                      "metadata": {}
                  }
              ]
          }

However, we are unable to create snapshots within any sub-directories of / :

          root@ceph-01:/mnt/dncephfs# mkdir exampledir
          root@ceph-01:/mnt/dncephfs# cd exampledir/.snap
          root@ceph-01:/mnt/dncephfs/exampledir/.snap# mkdir examplesnapshot
          mkdir: cannot create directory ‘examplesnapshot’: Operation
not permitted

We initially were mounting this volume under a different client which
did not have rwps. We tried explicitly providing rwps to that client and
then moved on to testing with our client.admin (with auth listed above).

We have tried explicitly setting "ceph fs set dncephfs allow_new_snaps
true" which had no effect. We have search the mds logs and no entries
appear on the snapshot creation failure.

Does anyone have any idea what may be going on or further information we
should be looking at to resolve this?

Thank you,

-David

-- 
David Prude
Systems Administrator
PGP Fingerprint: 1DAA 4418 7F7F B8AA F50C  6FDF C294 B58F A286 F847
Democracy Now!
www.democracynow.org

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx