Yes, you’re right but I have 0 object/s recovery last night. when I changed pg/pgp from 1400 to 2048, rebalancing speeded up but the percentage of rebalancing backed to 53%. I have this situation again n again since I dropped out failed OSD when I increase pg/pgp but each time rebalancing stopped at 0 objects/s and low speed transfer. Thanks > On Jul 3, 2016, at 1:25 PM, Wido den Hollander <wido@xxxxxxxx> wrote: > > >> Op 3 juli 2016 om 10:50 schreef Roozbeh Shafiee <roozbeh.shafiee@xxxxxxxxx>: >> >> >> Thanks for quick response, Wido >> >> the "ceph -s" output has pasted here: >> http://pastie.org/10897747 >> >> and this is output of “ceph health detail”: >> http://pastebin.com/vMeURWC9 >> > > It seems the cluster is still backfilling PGs and you 'ceph -s' shows so: 'recovery io 62375 kB/s, 15 objects/s' > > It will just take some time before it finishes. > > Wido > >> Thank you >> >>> On Jul 3, 2016, at 1:10 PM, Wido den Hollander <wido@xxxxxxxx> wrote: >>> >>> >>>> Op 3 juli 2016 om 10:34 schreef Roozbeh Shafiee <roozbeh.shafiee@xxxxxxxxx>: >>>> >>>> >>>> Hi list, >>>> >>>> A few days ago one of my OSDs failed and I dropped out that but afterwards I got >>>> HEALTH_WARN until now. After turing off the OSD, the self-healing system started >>>> to rebalance data between other OSDs. >>>> >>>> My question is: At the end of rebalancing, the process doesn’t complete and I get this message >>>> at the end of “ceph -s” output: >>>> >>>> recovery io 1456 KB/s, 0 object/s >>>> >>> >>> Could you post the exact output of 'ceph -s'? >>> >>> There is something more which needs to be shown. >>> >>> 'ceph health detail' also might tell you more. >>> >>> Wido >>> >>>> how can I back to HEALTH_OK situation again? >>>> >>>> My cluster details are: >>>> >>>> - 27 OSDs >>>> - 3 MONs >>>> - 2048 pg/pgs >>>> - Each OSD has 4 TB of space >>>> - CentOS 7.2 with 3.10 linux kernel >>>> - Ceph Hammer version >>>> >>>> Thank you, >>>> Roozbeh_______________________________________________ >>>> ceph-users mailing list >>>> ceph-users@xxxxxxxxxxxxxx >>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |