>> There is no way to fill up all disks evenly with the same number of
>> Bytes and then stop filling the small disks when they're full and
>> only continue filling the larger disks.
>This is possible with adjusting crush weights. Initially the smaller
>drives are weighted more highly than larger drives. As data gets added
>the weights are changed so that larger drives continue to fill while no
>drives becomes overfull.
So IMHO when you diverge from the default crush weight (e.g. for performance) it is something you should do permanently or
have a very clear path to what the next steps will be (e.g. we need to do this temporarily until the new hardware comes in).
You gain some short-term performance while you have the space.
However, as the cluster gets fuller:
You will need to change the weight which results in a lot of data movement which is a pain in itself especially when the cluster is near its IO limits.
Finally, you still end up with the exact same (bad) performance but it might now be a performance cliff instead of a gradual worsening over time.
A possible way to solve this would be to implement some mechanism to reshuffle the data based on the IO-patterns so each OSD get the same IO-pressure (or even better based on a new "IOPS" weight you can set).
So you could give each OSD the proper "size weight" and ceph would make sure you get the optimal IO performance by making sure each disk has the proper amount of "hot" data where the IOs happen.
But I guess that’s a very hard thing to build properly.
Final note: if you only have SSDs in the cluster the problem might not be there because usually bigger SSDs are also faster :)
Cheers,
Robert van Leeuwen
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
