Re: Ceph MDS - busy?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Things to check:

* metadata is on SSD?
* try multiple active MDS servers
* try a larger cache for the MDS
* try a recent version of Ceph

Paul

-- 
Paul Emmerich

Looking for help with your Ceph cluster? Contact us at https://croit.io

croit GmbH
Freseniusstr. 31h
81247 München
www.croit.io
Tel: +49 89 1896585 90


On Thu, Apr 30, 2020 at 10:02 AM <jesper@xxxxxxxx> wrote:

> Hi.
>
> How do I find out if the MDS is "busy" - being the one limiting CephFS
> metadata throughput. (12.2.8).
>
> $ time find . | wc -l
> 1918069
>
> real    8m43.008s
> user    0m2.689s
> sys     0m7.818s
>
> or 3.667ms per file.
> In the light of "potentially batching" and a network latency of ~0.20ms to
> the MDS - I have a feeling that this could be significantly improved.
>
> Then I additionally tried to do the same through the NFS -ganesha gateway.
>
> For reference:
> Same - but on "local DAS - xfs".
> $ time find . | wc -l
> 1918061
>
> real    0m4.848s
> user    0m2.360s
> sys     0m2.816s
>
> Same but "above local DAS over NFS":
> $ time find . | wc -l
> 1918061
>
> real    5m56.546s
> user    0m2.903s
> sys     0m34.381s
>
>
> jk@ceph-mon1:~$ sudo ceph fs status
> [sudo] password for jk:
> cephfs - 84 clients
> ======
> +------+----------------+-----------+---------------+-------+-------+
> | Rank |     State      |    MDS    |    Activity   |  dns  |  inos |
> +------+----------------+-----------+---------------+-------+-------+
> |  0   |     active     | ceph-mds2 | Reqs: 1369 /s | 11.3M | 11.3M |
> | 0-s  | standby-replay | ceph-mds1 | Evts:    0 /s |    0  |    0  |
> +------+----------------+-----------+---------------+-------+-------+
> +------------------+----------+-------+-------+
> |       Pool       |   type   |  used | avail |
> +------------------+----------+-------+-------+
> | cephfs_metadata  | metadata |  226M | 16.4T |
> |   cephfs_data    |   data   |  164T |  132T |
> | cephfs_data_ec42 |   data   |  180T |  265T |
> +------------------+----------+-------+-------+
>
> +-------------+
> | Standby MDS |
> +-------------+
> +-------------+
> MDS version: ceph version 12.2.5-45redhat1xenial
> (d4b9f17b56b3348566926849313084dd6efc2ca2) luminous (stable)
>
> How can we asses where the bottleneck is and what to do to speed it up?
>
>
>
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux