> On Feb 3, 2016, at 21:50, Michael Metz-Martini | SpeedPartner GmbH <metz@xxxxxxxxxxxxxxx> wrote: > > Hi, > > Am 03.02.2016 um 12:11 schrieb Yan, Zheng: >>> On Feb 3, 2016, at 17:39, Michael Metz-Martini | SpeedPartner GmbH <metz@xxxxxxxxxxxxxxx> wrote: >>> Am 03.02.2016 um 10:26 schrieb Gregory Farnum: >>>> On Tue, Feb 2, 2016 at 10:09 PM, Michael Metz-Martini | SpeedPartner >>>> GmbH <metz@xxxxxxxxxxxxxxx> wrote: >>>> Or maybe your kernels are too old; Zheng would know. >>> We're already far away from centos-Dist-Kernel. but upgrading to 4.4.x >>> for the clients should be possible if that might help. >> mds log should contain messages like: >> >> client.XXXX isn't responding to mclientcaps(revoke) >> >> please send these messages to us. > 2016-02-03 14:42:25.568800 7fadfd280700 2 mds.0.cache > check_memory_usage total 17302804, rss 16604996, heap 42916, malloc > -1008738 mmap 0, baseline 39844, buffers 0, max 1048576, 881503 / > 3999988 inodes have caps, 882499 caps, 0.220625 caps per inode > 2016-02-03 14:42:25.581494 7fadfd280700 0 log_channel(default) log > [WRN] : client.10199852 isn't responding to mclientcaps(revoke), ino > 100815bd349 pending pAsLsXsFsc issued pAsLsXsFscb, sent 62.127500 > seconds ago > 2016-02-03 14:42:25.581519 7fadfd280700 0 log_channel(default) log > [WRN] : client.10199852 isn't responding to mclientcaps(revoke), ino > 100815bf1af pending pAsLsXsFsc issued pAsLsXsFscb, sent 62.085996 > seconds ago > 2016-02-03 14:42:25.581527 7fadfd280700 0 log_channel(default) log > [WRN] : client.10199852 isn't responding to mclientcaps(revoke), ino > 100815bf4d3 pending pAsLsXsFsc issued pAsLsXsFscb, sent 62.084284 > seconds ago > 2016-02-03 14:42:25.581534 7fadfd280700 0 log_channel(default) log > [WRN] : client.10199852 isn't responding to mclientcaps(revoke), ino > 100815d2500 pending pAsLsXsFsc issued pAsLsXsFscb, sent 61.731320 > seconds ago > 2016-02-03 14:42:25.581840 7fadfd280700 0 log_channel(default) log > [WRN] : 7 slow requests, 6 included below; oldest blocked for > > 62.125785 secs > 2016-02-03 14:42:25.581849 7fadfd280700 0 log_channel(default) log > [WRN] : slow request 62.125785 seconds old, received at 2016-02-03 > 14:41:23.455812: client_request(client.10199855:1313157 getattr > pAsLsXsFs #100815bd349 2016-02-03 14:41:23.452386) currently failed to > rdlock, waiting This seems like dirty page writeback is too slow. Is there any hung OSD request in /sys/kernel/debug/ceph/xxx/osdc? > > -- > Kind regards > Michael _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |