Hello, Thank you all for sharing your experiences, thoughts. One more question: regarding the pool used for the measurement (the -p option of the script), is it recommended to create a new pool for this? Or I could use one of our already existing pools in the cluster? Thank you, Laszlo On 07.08.2017 19:07, Bryan Stillwell wrote:
Dan, We recently went through an expansion of an RGW cluster and found that we needed 'norebalance' set whenever making CRUSH weight changes to avoid slow requests. We were also increasing the CRUSH weight by 1.0 each time which seemed to reduce the extra data movement we were seeing with smaller weight increases. Maybe something to try out next time? Bryan From: ceph-users <ceph-users-bounces@xxxxxxxxxxxxxx> on behalf of Dan van der Ster <dan@xxxxxxxxxxxxxx> Date: Friday, August 4, 2017 at 1:59 AM To: Laszlo Budai <laszlo@xxxxxxxxxxxxxxxx> Cc: ceph-users <ceph-users@xxxxxxxxxxxxxx> Subject: Re: expanding cluster with minimal impact Hi Laszlo, The script defaults are what we used to do a large intervention (the default delta weight is 0.01). For our clusters going any faster becomes disruptive, but this really depends on your cluster size and activity. BTW, in case it wasn't clear, to use this script for adding capacity you need to create the new OSDs to your cluster with initial crush weight = 0.0 osd crush initial weight = 0 osd crush update on start = true -- Dan On Thu, Aug 3, 2017 at 8:12 PM, Laszlo Budai <laszlo@xxxxxxxxxxxxxxxx> wrote: Dear all, I need to expand a ceph cluster with minimal impact. Reading previous threads on this topic from the list I've found the ceph-gentle-reweight script (https://github.com/cernceph/ceph-scripts/blob/master/tools/ceph-gentle-reweight) created by Dan van der Ster (Thank you Dan for sharing the script with us!). I've done some experiments, and it looks promising, but it is needed to properly set the parameters. Did any of you tested this script before? what is the recommended delta_weight to be used? From the default parameters of the script I can see that the default delta weight is .5% of the target weight that means 200 reweighting cycles. I have experimented with a reweight ratio of 5% while running a fio test on a client. The results were OK (I mean no slow requests), but my test cluster was a very small one. If any of you has done some larger experiments with this script I would be really interested to read about your results. Thank you! Laszlo _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
