from our experience even 80% is a 'dangerous zone' ( unfortunately this is how it goes ... and it is quiet wasteful comparing to other solutions )
On Thu, May 4, 2017 at 3:57 PM, David Turner <drakonstein@xxxxxxxxx> wrote:
The Ceph Enterprise default is 65% nearfull. Do not go above 85% nearfull unless you are stuck while backfilling and need to increase it to add/remove storage. Ceph needs overhead to be able to recover from situations where disks are lost. I always take into account what would happen to the %full if I lost a full storage node and used that as my target size.The first place to look when trying to use more space is to balance your cluster to get all disks closer to how fill the cluster is as a whole. By default osds can be as far apart as 50% while others are 85% and your cluster % used is right around upper 60's-low 70's.
The most full I've ever maintained a cluster was 77-80% full and it took so much more babysitting as any time a disk died or we added storage, it required constant babysitting to make sure things would finish backfilling. This cluster has 50 nodes.
On Thu, May 4, 2017, 8:30 AM Loic Dachary <loic@xxxxxxxxxxx> wrote:Hi,
In a cluster where the failure domain is the host and dozens of hosts, the 85% default for nearfull ratio is fine. A host failing won't suddenly make the cluster 99% full. In smaller clusters, with 10 hosts or less, it is likely to not be enough. And in larger clusters 85% may be too much to reserve and 90% could be more than enough.
Is there a way to calculate the optimum nearfull ratio for a given crushmap ?
Cheers
--
Loïc Dachary, Artisan Logiciel Libre
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph. com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph. com
This e-mail, as well as any attached document, may contain material which is confidential and privileged and may include trademark, copyright and other intellectual property rights that are proprietary to Kenshoo Ltd, its subsidiaries or affiliates ("Kenshoo"). This e-mail and its attachments may be read, copied and used only by the addressee for the purpose(s) for which it was disclosed herein. If you have received it in error, please destroy the message and any attachment, and contact us immediately. If you are not the intended recipient, be aware that any review, reliance, disclosure, copying, distribution or use of the contents of this message without Kenshoo's express permission is strictly prohibited.
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
