Re: df shows wrong size of cephfs share when a subdirectory is mounted

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, Apr 14, 2022 at 8:07 PM Ryan Taylor <rptaylor@xxxxxxx> wrote:
>
> Hello,
>
>
> I am using cephfs via Openstack Manila (Ussuri I think).
>
> The cephfs cluster is v14.2.22 and my client has kernel  4.18.0-348.20.1.el8_5.x86_64
>
>
> I have a Manila share
>
> /volumes/_nogroup/55e46a89-31ff-4878-9e2a-81b4226c3cb2
>
>
> that is 5000 GB in size. When I mount it the size is reported correctly:
>
>
> # df -h /cephfs
> Filesystem                                                                                                 Size  Used Avail Use% Mounted on
> 10.30.201.3:6789,10.30.202.3:6789,10.30.203.3:6789:/volumes/_nogroup/55e46a89-31ff-4878-9e2a-81b4226c3cb2  4.9T  278G  4.7T   6% /cephfs
>
>
> However when I mount a subpath /test1 of my share, then both the size and usage are showing the size of the whole cephfs filesystem rather than my private share.
>
>
> # df -h /cephfs
> Filesystem                                                                                                       Size  Used Avail Use% Mounted on
> 10.30.201.3:6789,10.30.202.3:6789,10.30.203.3:6789:/volumes/_nogroup/55e46a89-31ff-4878-9e2a-81b4226c3cb2/test1  4.0P  277T  3.7P   7% /cephfs
>

What are the capabilities of the ceph client user ID that you used to
mount "/volumes/_nogroup/55e46a89-31ff-4878-9e2a-81b4226c3cb2/test1" ?
Maybe you're hitting this limitation in
https://docs.ceph.com/en/latest/cephfs/quota/#limitations ,
"Quotas must be configured carefully when used with path-based mount
restrictions. The client needs to have access to the directory inode
on which quotas are configured in order to enforce them. If the client
has restricted access to a specific path (e.g., /home/user) based on
the MDS capability, and a quota is configured on an ancestor directory
they do not have access to (e.g., /home), the client will not enforce
it. When using path-based access restrictions be sure to configure the
quota on the directory the client is restricted too (e.g., /home/user)
or something nested beneath it. "

>
> I tried setting the  ceph.quota.max_bytes  xattr on a subdirectory but it did not help.
>

You can't set quota xattr if your ceph client user ID doesn't have 'p'
flag in its MDS capabilities,
https://docs.ceph.com/en/latest/cephfs/client-auth/#layout-and-quota-restriction-the-p-flag
.

-Ramana

> I'm not sure if the issue is in cephfs or Manila, but what would be required to get the right size and usage stats to be reported by df when a subpath of a share is mounted?
>
>
> Thanks!
>
> -rt
>
>
> Ryan Taylor
> Research Computing Specialist
> Research Computing Services, University Systems
> University of Victoria
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux