On Wed, Oct 18, 2017 at 8:12 AM, Ольга Ухина <olga.uhina@xxxxxxxxx> wrote: > I have a problem with ceph luminous 12.2.1. > […] > I have slow requests on different OSDs on random time (for example at night, > but I don’t see any problems at the time of problem > […] > 2017-10-18 01:20:38.187326 mon.st3 mon.0 10.192.1.78:6789/0 22689 : cluster > [WRN] Health check update: 49 slow requests are blocked > 32 sec > (REQUEST_SLOW) This looks almost exactly like what we have been experiencing, and your use-case (Proxmox client using rbd) is the same as ours as well. Unfortunately we were not able to find the source of the issue so far, and haven’t gotten much feedback from the list. Extensive testing of every component has ruled out any hardware issue we can think of. Originally we thought our issue was related to deep-scrub, but that now appears not to be the case, as it happens even when nothing is being deep-scrubbed. Nonetheless, although they aren’t the cause, they definitely make the problem much worse. So you may want to check to see if deep-scrub operations are happening at the times where you see issues and (if so) whether the OSDs participating in the deep-scrub are the same ones reporting slow requests. Hopefully you have better luck finding/fixing this than we have! It’s definitely been a very frustrating issue for us. Thanks! _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |