On Tue, 22 Nov 2016, Dan Van Der Ster wrote: > Hi, > > I have a couple questions about http://tracker.ceph.com/issues/15653 > > In the ticket Sage discusses small/big drives, and the small drives get > more data than expected. > > But we observe this at the rack level: our cluster has four racks, with > 7, 8, 8, 4 hosts respectively. The rack with 4 hosts is ~35% more full > than the others. > > So AFAICT, because of #15653, CRUSH does not currently work well if you > try to build a pool which is replicated rack/host-wise when your > rack/hosts are not all ~identical in size. Right--it's not about devices, but items within a CRUSH bucket. Unfortunately we don't have a good technical solution for this yet. The best proposal so far is Adam's PR at https://github.com/ceph/ceph/pull/10218 but it leaves much to be desired. I think we can do better, hopefully in time for lumninous. In the meantime, you can underweight (devices in) small racks. :( sage > Are others noticing this pattern? Or are we unusual in that our clusters > are not flat/uniform in structure? > > Cheers, Dan > _______________________________________________ Ceph-large mailing list Ceph-large@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-large-ceph.com
