> On 22 Nov 2016, at 16:17, Sage Weil <sage@xxxxxxxxxxxx> wrote: > > On Tue, 22 Nov 2016, Dan Van Der Ster wrote: >> Hi, >> >> I have a couple questions about http://tracker.ceph.com/issues/15653 >> >> In the ticket Sage discusses small/big drives, and the small drives get >> more data than expected. >> >> But we observe this at the rack level: our cluster has four racks, with >> 7, 8, 8, 4 hosts respectively. The rack with 4 hosts is ~35% more full >> than the others. >> >> So AFAICT, because of #15653, CRUSH does not currently work well if you >> try to build a pool which is replicated rack/host-wise when your >> rack/hosts are not all ~identical in size. > > Right--it's not about devices, but items within a CRUSH bucket. > Unfortunately we don't have a good technical solution for this yet. The > best proposal so far is Adam's PR at > > https://github.com/ceph/ceph/pull/10218 > > but it leaves much to be desired. I think we can do better, hopefully in > time for lumninous. > > In the meantime, you can underweight (devices in) small racks. :( Thanks Sage. So you confirm that reweighting alone won't solve this? -- dan > > sage > > > >> Are others noticing this pattern? Or are we unusual in that our clusters >> are not flat/uniform in structure? >> >> Cheers, Dan >> _______________________________________________ Ceph-large mailing list Ceph-large@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-large-ceph.com
