On Sun, Sep 24, 2017 at 3:02 AM, Elita Lobo <loboelita@xxxxxxxxx> wrote: > Hi , > > We are trying to simulate repair of single/multiple erasures using cl > msr. When we remove one or more osds , we see that other osds shift > positions within the placement group. > > Eg - This is how our original PG looked - > [10,8,11,6,12,13,15,14,3,4,0,24,21,19] > we had 25 osds available in total and k=10 and m=4 (n=14) . When we > removed osd 14 and osd 15, some of the osds changed their positions > within the PG. > > Resulting PG: [10,8,11,6,21,19,22,23,3,4,0,24,12,13] > Original PG: [10,8,11,6,12,13,15,14,3,4,0,24,21,19] > > As you can see osds 12,13 have exchanged positions with osds 21,19. > > Can we avoid this and only let the removed osd be replaced by a > different available osd? > Also can you please explain under what conditions osds will shift within the PG? This is generally not supposed to happen, but you're forcing a pseudo-random algorithm to pick 14 out of 25 target devices. You can probably reduce the odds by changing the shape of your CRUSH map, but in general running with each object on that high a percentage of your cluster is going to result in mapping oddities like you've seen. -Greg > > Thanks, > Elita > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
