Dear Dan, thanks for the quick reply! Am 27.05.19 um 11:44 schrieb Dan van der Ster:
Hi Oliver, We saw the same issue after upgrading to mimic. IIRC we could make the max_bytes xattr visible by touching an empty file in the dir (thereby updating the dir inode). e.g. touch /cephfs/user/freyermu/.quota; rm /cephfs/user/freyermu/.quota
sadly, no, not even with sync's in between: ----- $ touch /cephfs/user/freyermu/.quota; sync; rm -f /cephfs/user/freyermu/.quota; sync; getfattr --absolute-names --only-values -n ceph.quota.max_bytes /cephfs/user/freyermu/ /cephfs/user/freyermu/: ceph.quota.max_bytes: No such attribute ----- Also restarting the FUSE client after that does not change it. Maybe this requires the rest of the cluster to be upgraded to work? I'm just guessing here, but maybe the MDS needs the file creation / update of the directory inode to "update" the way the quota attributes are exported. If something changed here with Mimic, this would explain why the "touch" is needed. And this would also explain why this might only help if the MDS is upgraded to Mimic, too. We have scheduled the remaining parts of the upgrade for Wednesday, and worst case could survive until then without quota enforcement, but it's a really strange and unexpected incompatibility. Cheers, Oliver
Does that work? -- dan On Mon, May 27, 2019 at 11:36 AM Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx> wrote:Dear Cephalopodians, in the process of migrating a cluster from Luminous (12.2.12) to Mimic (13.2.5), we have upgraded the FUSE clients first (we took the chance during a time of low activity), thinking that this should not cause any issues. All MDS+MON+OSDs are still on Luminous, 12.2.12. However, it seems quotas have stopped working - with a (FUSE) Mimic client (13.2.5), I see: $ getfattr --absolute-names --only-values -n ceph.quota.max_bytes /cephfs/user/freyermu/ /cephfs/user/freyermu/: ceph.quota.max_bytes: No such attribute A Luminous client (12.2.12) on the same cluster sees: $ getfattr --absolute-names --only-values -n ceph.quota.max_bytes /cephfs/user/freyermu/ 5000000000000 It does not seem as if the attribute has been renamed (e.g. https://github.com/ceph/ceph/blob/mimic/qa/tasks/cephfs/test_quota.py still references it, same for the docs), and I have to assume the clients also do not enforce quota if they do not see it. Is this a known incompatibility between Mimic clients and a Luminous cluster? The release notes of Mimic only mention that quota support was added to the kernel client, but nothing else quota related catches my eye. Cheers, Oliver _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
-- Oliver Freyermuth Universität Bonn Physikalisches Institut, Raum 1.047 Nußallee 12 53115 Bonn -- Tel.: +49 228 73 2367 Fax: +49 228 73 7869 --
Attachment:
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
