On 11/07/17 20:05, Eino Tuominen wrote: > Hi Richard, > > Thanks for the explanation, that makes perfect sense. I've missed the difference between ceph osd reweight and ceph osd crush reweight. I have to study that better. > > Is there a way to get ceph to prioritise fixing degraded objects over fixing misplaced ones? The difference between out/reweight and crush weight caught me out recently as well, I was seeing a lot more data movement that I expected during the process of replacing old disks until someone explained the difference to me. As I understand it, ceph already prioritises fixing things which are degraded over things which are misplaced - the problem is that this is granular at the level of PGs, then objects. It will try and choose to recover PGs with degraded objects before PGs which only have misplaced objects, and I think that within a specific PG recovery it will try to fix degraded objects before misplaced objects. However it will still finish recovering the whole PG, misplaced objects and all - it's got no mechanism to put a recovery on hold once it's fixed the degraded objects to free capacity for recovering a different PG. The default recovery limits are really conservative though, you can probably increase the rate quite a lot by increasing osd_max_backfills above the default limit of 1 - "ceph tell osd.X injectargs '--osd_max_backfills Y'" to update a running config, "osd_max_backfills = Y" in the [osd] section of your ceph.conf to make it persist. Rich
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
