Re: OSDs not balanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]


> I think the short answer is "because you have so wildly varying sizes
> both for drives and hosts".

Arguably OP's OSDs *are* balanced in that their PGs are roughly in line with their sizes, but indeed the size disparity is problematic in some ways.

Notably, the 500GB OSD should just be removed.  I think balancing doesn't account for WAL/DB/other overhead, so it won't be accurately accounted for and can't hold much data nyway.

This cluster shows evidence of reweight-by-utilization having been run, but only on two of the hosts.  If the balancer module is active, those override weights will confound it.

> If your drive sizes span from 0.5 to 9.5, there will naturally be
> skewed data, and it is not a huge surprise that the automation has
> some troubles getting it "good". When the balancer places a PG on a
> 0.5-sized drive compared to a 9.5-sized one, it eats up 19x more of
> the "free space" on the smaller one, so there are very few good
> options when the sizes are so different. Even if you placed all PGs
> correctly due to size, the 9.5-sized disk would end up getting 19x
> more IO than the small drive and for hdd, it seldom is possible to
> gracefully handle a 19-fold increase in IO, most of the time will
> probably be spent on seeks.

ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]

  Powered by Linux