> Op 6 mei 2017 om 9:55 schreef Christian Balzer <chibi@xxxxxxx>: > > > > Hello, > > On Sat, 6 May 2017 09:25:15 +0200 (CEST) Wido den Hollander wrote: > > > > Op 5 mei 2017 om 10:33 schreef Sam Wouters <sam@xxxxxxxxx>: > > > > > > > > > Hi, > > > > > > we have a small cluster running on jewel 10.2.7; NL-SAS disks only, osd > > > data and journal co located on the disks; main purpose rgw secondary zone. > > > > > > Since the upgrade to jewel, whenever a deep scrub starts on one of the > > > rgw index pool pg's, slow requests start piling up and rgw requests are > > > blocked after some hours. > > > The deep-scrub doesn't seem to finish (still running after +11 hours) > > > and only escape I found so far is a restart of the primary osd holding > > > the pg. > > > > > > Maybe important to know, we have some large rgw buckets regarding > > > #objects (+ 3 million) with only index sharding of 8. > > > > > > scrub related settings: > > > osd scrub sleep = 0.1 > > > > Try removing this line, it can block threads under Jewel. > > > I'd really REALLY wish that would get fixed properly, as in the original > functionality restored. Afaik new work is being done on this. There was a recent thread on the ceph-users or devel (can't find it) that new code is out there to fix this. Wido > Because as we've learned entrusting everything into internal Ceph queues > with priorities isn't working as expected in all cases. > > For a second, very distant option, turn it into a NOP for the time being. > As it stands now, it's another self-made, Jewel introduced bug... > > Christian > > > See how that works out. > > > > Wido > > > > > osd scrub during recovery = False > > > osd scrub priority = 1 > > > osd deep scrub stride = 1048576 > > > osd scrub chunk min = 1 > > > osd scrub chunk max = 1 > > > > > > Any help on debugging / resolving would be very much appreciated... > > > > > > regards, > > > Sam > > > > > > _______________________________________________ > > > ceph-users mailing list > > > ceph-users@xxxxxxxxxxxxxx > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > _______________________________________________ > > ceph-users mailing list > > ceph-users@xxxxxxxxxxxxxx > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > -- > Christian Balzer Network/Systems Engineer > chibi@xxxxxxx Global OnLine Japan/Rakuten Communications > http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |