Hi,
On 09/10/2018 02:40 PM, marc-antoine desrochers wrote:
Hi,
I am currently running a ceph cluster running in CEPHFS with 3 nodes each
have 6 osd's except 1 who got 5. I got 3 mds : 2 active and 1 standby, 3
mon.
[root@ceph-n1 ~]# ceph -s
cluster:
id: 1d97aa70-2029-463a-b6fa-20e98f3e21fb
health: HEALTH_WARN
3 clients failing to respond to capability release
2 MDSs report slow requests
*snipsnap*
I'm now facing a problem :
MDS_CLIENT_LATE_RELEASE 3 clients failing to respond to capability release
mdsceph-n1(mds.0): Client mda3.sogetel.net failing to respond to
capability releaseclient_id: 1134426
mdsceph-n1(mds.0): Client mda2.sogetel.net failing to respond to
capability releaseclient_id: 1172391
mdsceph-n2(mds.1): Client mda3.sogetel.net failing to respond to
capability releaseclient_id: 1134426
MDS_SLOW_REQUEST 2 MDSs report slow requests
mdsceph-n1(mds.0): 112 slow requests are blocked > 30 sec
mdsceph-n2(mds.1): 323 slow requests are blocked > 30 sec
The messages indicate that clients do not release capabilities for
opened/cached files. These files are either accessed by other clients
(and thus these other clients need to acquire the capabilities), or the
MDS runs out of memory and tries to reduce the number of capabilities in
his book keeping to reduce the memory footprint. In both cases the
client request to open a file is blocked.
In case of the second problem, you can increase the mds cache size to
allow it to store more inode and capability entries
(mds_cache_memory_limit in ceph.conf). You should also try to figure out
why the clients do not release the capabilities, e.g. whether they
really have a large number of open/cached files.
Do you use ceph-fuse or the kernel based implementation?
Regards,
Burkhard
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
