Hi! Thanks for your help.
How can I increase interval of history for command ceph daemon osd.<id> dump_historic_ops? It shows only for several minutes.
I see slow requests on random osds each time and on different hosts (there are three). As I see in logs the problem doesn't relate to scrubbing.
Regards,
Olga Ukhina
Olga Ukhina
2017-10-20 4:42 GMT+03:00 Brad Hubbard <bhubbard@xxxxxxxxxx>:
I guess you have both read and followed
http://docs.ceph.com/docs/master/rados/troubleshooting/ troubleshooting-osd/? highlight=backfill#debugging- slow-requests
What was the result?
On Fri, Oct 20, 2017 at 2:50 AM, J David <j.david.lists@xxxxxxxxx> wrote:
> On Wed, Oct 18, 2017 at 8:12 AM, Ольга Ухина <olga.uhina@xxxxxxxxx> wrote:
>> I have a problem with ceph luminous 12.2.1.
>> […]
>> I have slow requests on different OSDs on random time (for example at night,
>> but I don’t see any problems at the time of problem
>> […]
>> 2017-10-18 01:20:38.187326 mon.st3 mon.0 10.192.1.78:6789/0 22689 : cluster
>> [WRN] Health check update: 49 slow requests are blocked > 32 sec
>> (REQUEST_SLOW)
>
> This looks almost exactly like what we have been experiencing, and
> your use-case (Proxmox client using rbd) is the same as ours as well.
>
> Unfortunately we were not able to find the source of the issue so far,
> and haven’t gotten much feedback from the list. Extensive testing of
> every component has ruled out any hardware issue we can think of.
>
> Originally we thought our issue was related to deep-scrub, but that
> now appears not to be the case, as it happens even when nothing is
> being deep-scrubbed. Nonetheless, although they aren’t the cause,
> they definitely make the problem much worse. So you may want to check
> to see if deep-scrub operations are happening at the times where you
> see issues and (if so) whether the OSDs participating in the
> deep-scrub are the same ones reporting slow requests.
>
> Hopefully you have better luck finding/fixing this than we have! It’s
> definitely been a very frustrating issue for us.
>
> Thanks!
--> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph. com
Cheers,
Brad
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
