Den tis 19 juli 2022 kl 13:09 skrev Jesper Lykkegaard Karlsen <jelka@xxxxxxxxx>: > > Hi all, > Setup: Octopus - erasure 8-3 > I had gotten to the point where I had some rather old OSD nodes, that I wanted to replace with new ones. > The procedure was planned like this: > > * add new replacement OSD nodes > * set all OSDs on the retiring nodes to out. > * wait for everything to rebalance > * remove retiring nodes > After around 50% misplaced objects remaining, the OSDs started to complain about backfillfull OSDs and nearfull OSDs. > A bit of a surprise to me, as RAW size is only 47% used. > It seems that rebalancing does not happen in a prioritized manner, where planed backfill starts with the OSD with most space available space, but "alphabetically" according to pg-name. > Is this really true? I don't know if it does it in any particular order, just that it certainly doesn't fire off requests to the least filled OSD to receive data first, so when I have gotten into similar situations, it just tried to run as many moves as possible given max_backfill and all that, then some/most might get stuck in toofull, but as the rest of the slots progress, space gets available and at some point those toofull ones get handled. It delays the completion but hasn't caused me any other specific problems. Though I will admit I have used "ceph osd reweight osd.123 <less-than-1.0>" at times to force emptying of some OSDs, but that was more my impatience than anything else. -- May the most significant bit of your life be positive. _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |