Hi, I am currently running a ceph cluster running in CEPHFS with 3 nodes each have 6 osd’s except 1 who got 5. I got 3 mds : 2 active and 1 standby, 3 mon. [root@ceph-n1 ~]# ceph -s cluster: id: 1d97aa70-2029-463a-b6fa-20e98f3e21fb health: HEALTH_WARN 3 clients failing to respond to capability release 2 MDSs report slow requests services: mon: 3 daemons, quorum ceph-n1,ceph-n2,ceph-n3 mgr: ceph-n1(active), standbys: ceph-n2, ceph-n3 mds: cephfs-2/2/2 up {0=ceph-n1=up:active,1=ceph-n2=up:active}, 1 up:standby osd: 17 osds: 17 up, 17 in data: pools: 2 pools, 1024 pgs objects: 541k objects, 42006 MB usage: 143 GB used, 6825 GB / 6969 GB avail pgs: 1024 active+clean io: client: 32980 B/s rd, 77295 B/s wr, 5 op/s rd, 14 op/s wr
infrastructure so around 30 000 mailbox. I’m now facing a problem : mdsceph-n1(mds.0): Client mda3.sogetel.net failing to respond to capability releaseclient_id: 1134426 mdsceph-n1(mds.0): Client mda2.sogetel.net failing to respond to capability releaseclient_id: 1172391 mdsceph-n2(mds.1): Client mda3.sogetel.net failing to respond to capability releaseclient_id: 1134426 MDS_SLOW_REQUEST 2 MDSs report slow requests mdsceph-n1(mds.0): 112 slow requests are blocked > 30 sec mdsceph-n2(mds.1): 323 slow requests are blocked > 30 sec I can’t figure out how to fix this… Here some information’s about my cluster : I’m running ceph luminous 12.2.5 on my 3 ceph nodes : ceph-n1, ceph-n2, ceph-n3.
LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: Fedora Description: Fedora release 25 (Twenty Five) Release: 25 Codename: TwentyFive My ceph nodes : CentOS Linux release 7.5.1804 (Core) NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" CentOS Linux release 7.5.1804 (Core) CentOS Linux release 7.5.1804 (Core) ceph daemon mds.ceph-n1 perf dump mds : "mds": { "request": 21968558, "reply": 21954801, "reply_latency": { "avgcount": 21954801, "sum": 100879.560315258, "avgtime": 0.004594874 }, "forward": 13627, "dir_fetch": 3327, "dir_commit": 162830, "dir_split": 1, "dir_merge": 0, "inode_max": 2147483647, "inodes": 68767, "inodes_top": 4524, "inodes_bottom": 56697, "inodes_pin_tail": 7546, "inodes_pinned": 62304, "inodes_expired": 1640159, "inodes_with_caps": 62192, "caps": 114126, "subtrees": 14, "traverse": 38309963, "traverse_hit": 37606227, "traverse_forward": 12189, "traverse_discover": 6634, "traverse_dir_fetch": 1769, "traverse_remote_ino": 6, "traverse_lock": 7731, "load_cent": 2196856701, "q": 0, "exported": 143, "exported_inodes": 291372, "imported": 125, "imported_inodes": 176509 Thanks for your help… Regards Marc-Antoine |
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
