Without knowing more about the underlying hardware, you likely are reaching some type of IO resource constraint. Are your journals colocated or non-colocated? How fast is your backend OSD storage device? You may also want to look at setting the norebalance flag. Good luck! > On Sep 20, 2018, at 19:52, Chen Allen <uilcxr@xxxxxxxxx> wrote: > > Hi there, > > Has anyone experienced below? > 2 of OSD server was down, after bring up 2 of servers, I brought 52 OSD's in with just weight of 0.05, but it causing huge backfilling load, I saw so many blocked requests and a number of pg stuck inactive. some of servers was impact. so I stopped backfilling by mark nobackfill flag. everything back to normal. > But the most strange thing happens after 2 hours, the backfilling suddenly start again despite of nobackfill flag marked and causing so many blocked requests then we have to reweight 52 OSD's to 0 to stabilize storage. > > Not sure why backfill start again. Anyone has any idea about that please comments. > > Thanks so much. > Allen > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |