Hi Casey,
Thank you for the follow-up. I had just found that one while searching in the tracker. I probably should have done that first(though I guess I was hoping/assuming google would have brought it up(but of course it didn't).
Anyway, it is working nicely for me now with the newer version of curl, so that is what really matters. Still, kind of a pain having to go build the new package.
Have a good day,
Lewis George
From: "Casey Bodley" <cbodley@xxxxxxxxxx>
Sent: Friday, September 16, 2016 11:02 AM
To: ceph-users@xxxxxxxxxxxxxx
Subject: Re: High CPU load with radosgw instances
In the meantime, we've made changes to radosgw so that it can detect and
work around this libcurl bug. You can track the progress of this
workaround (currently in master and pending backport to jewel) at
http://tracker.ceph.com/issues/16695.
Casey
On 09/16/2016 01:38 PM, Ken Dreyer wrote:
> Hi Lewis,
>
> This sounds a lot like https://bugzilla.redhat.com/1347904 , currently
> slated for the upcoming RHEL 7.3 (and CentOS 7.3).
>
> There's an SRPM in that BZ that you can rebuild and test out. This
> method won't require you to keep chasing upstream curl versions
> forever (curl has a lot of CVEs).
>
> Mind testing that out and reporting back?
>
> - Ken
>
>
> On Fri, Sep 16, 2016 at 11:06 AM, lewis.george@xxxxxxxxxxxxx
> <lewis.george@xxxxxxxxxxxxx> wrote:
>> Hi Yehuda,
>> Well, again, thank you!
>>
>> I was able to get a package built from the latest curl release, and after
>> upgrading on my radosgw hosts, the load is no longer running high. The load
>> is just sitting at almost nothing and I only see the radosgw process using
>> CPU when it is actually doing something now.
>>
>> So, I am still curious if this would be considered a bug or not, since the
>> curl version from the base CentOS repo seems to have an issue.
>>
>> Have a good day,
>>
>> Lewis George
>>
>>
>>
>> ________________________________
>> From: "lewis.george@xxxxxxxxxxxxx" <lewis.george@xxxxxxxxxxxxx>
>> Sent: Friday, September 16, 2016 7:28 AM
>> To: "Yehuda Sadeh-Weinraub" <yehuda@xxxxxxxxxx>
>>
>> Cc: "ceph-users@xxxxxxxxxxxxxx" <ceph-users@xxxxxxxxxxxxxx>
>> Subject: Re: High CPU load with radosgw instances
>>
>> Hi Yehuda,
>> Thank you for the idea. I will try to test that and see if it helps.
>>
>> If that is the case, would that be considered a bug with radosgw? I ask
>> because, that version of curl seems to be what is currently standard on
>> RHEL/CentOS 7 (fully updated). I will have to either compile it or search
>> 3rd-party repos for newer version, which is not usually something that is
>> great.
>>
>> Have a good day,
>>
>> Lewis George
>>
>>
>> ________________________________
>> From: "Yehuda Sadeh-Weinraub" <yehuda@xxxxxxxxxx>
>> Sent: Thursday, September 15, 2016 10:42 PM
>> To: lewis.george@xxxxxxxxxxxxx
>> Cc: "ceph-users@xxxxxxxxxxxxxx" <ceph-users@xxxxxxxxxxxxxx>
>> Subject: Re: High CPU load with radosgw instances
>>
>> On Thu, Sep 15, 2016 at 4:53 PM, lewis.george@xxxxxxxxxxxxx
>> <lewis.george@xxxxxxxxxxxxx> wrote:
>>> Hi,
>>> So, maybe someone has an idea of where to go on this.
>>>
>>> I have just setup 2 rgw instances in a multisite setup. They are working
>>> nicely. I have add a couple of test buckets and some files to make sure it
>>> works is all. The status shows both are caught up. Nobody else is
>>> accessing
>>> or using them.
>>>
>>> However, the CPU load on both hosts is sitting at like 3.00, with the
>>> radosgw process taking up 99% CPU constantly. I do not see anything in the
>>> logs happening at all.
>>>
>>> Any thoughts or direction?
>>>
>> We've seen that happening when running on a system with older version
>> of libcurl (e.g., 7.29). If that's the case upgrading to a newer
>> version should fix it for you.
>>
>> Yehuda
>>
>>
>>> Have a good day,
>>>
>>> Lewis George
>>>
>>>
>>> _______________________________________________
>>> ceph-users mailing list
>>> ceph-users@xxxxxxxxxxxxxx
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>
>>
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Sent: Friday, September 16, 2016 11:02 AM
To: ceph-users@xxxxxxxxxxxxxx
Subject: Re: High CPU load with radosgw instances
work around this libcurl bug. You can track the progress of this
workaround (currently in master and pending backport to jewel) at
http://tracker.ceph.com/issues/16695.
Casey
On 09/16/2016 01:38 PM, Ken Dreyer wrote:
> Hi Lewis,
>
> This sounds a lot like https://bugzilla.redhat.com/1347904 , currently
> slated for the upcoming RHEL 7.3 (and CentOS 7.3).
>
> There's an SRPM in that BZ that you can rebuild and test out. This
> method won't require you to keep chasing upstream curl versions
> forever (curl has a lot of CVEs).
>
> Mind testing that out and reporting back?
>
> - Ken
>
>
> On Fri, Sep 16, 2016 at 11:06 AM, lewis.george@xxxxxxxxxxxxx
> <lewis.george@xxxxxxxxxxxxx> wrote:
>> Hi Yehuda,
>> Well, again, thank you!
>>
>> I was able to get a package built from the latest curl release, and after
>> upgrading on my radosgw hosts, the load is no longer running high. The load
>> is just sitting at almost nothing and I only see the radosgw process using
>> CPU when it is actually doing something now.
>>
>> So, I am still curious if this would be considered a bug or not, since the
>> curl version from the base CentOS repo seems to have an issue.
>>
>> Have a good day,
>>
>> Lewis George
>>
>>
>>
>> ________________________________
>> From: "lewis.george@xxxxxxxxxxxxx" <lewis.george@xxxxxxxxxxxxx>
>> Sent: Friday, September 16, 2016 7:28 AM
>> To: "Yehuda Sadeh-Weinraub" <yehuda@xxxxxxxxxx>
>>
>> Cc: "ceph-users@xxxxxxxxxxxxxx" <ceph-users@xxxxxxxxxxxxxx>
>> Subject: Re: High CPU load with radosgw instances
>>
>> Hi Yehuda,
>> Thank you for the idea. I will try to test that and see if it helps.
>>
>> If that is the case, would that be considered a bug with radosgw? I ask
>> because, that version of curl seems to be what is currently standard on
>> RHEL/CentOS 7 (fully updated). I will have to either compile it or search
>> 3rd-party repos for newer version, which is not usually something that is
>> great.
>>
>> Have a good day,
>>
>> Lewis George
>>
>>
>> ________________________________
>> From: "Yehuda Sadeh-Weinraub" <yehuda@xxxxxxxxxx>
>> Sent: Thursday, September 15, 2016 10:42 PM
>> To: lewis.george@xxxxxxxxxxxxx
>> Cc: "ceph-users@xxxxxxxxxxxxxx" <ceph-users@xxxxxxxxxxxxxx>
>> Subject: Re: High CPU load with radosgw instances
>>
>> On Thu, Sep 15, 2016 at 4:53 PM, lewis.george@xxxxxxxxxxxxx
>> <lewis.george@xxxxxxxxxxxxx> wrote:
>>> Hi,
>>> So, maybe someone has an idea of where to go on this.
>>>
>>> I have just setup 2 rgw instances in a multisite setup. They are working
>>> nicely. I have add a couple of test buckets and some files to make sure it
>>> works is all. The status shows both are caught up. Nobody else is
>>> accessing
>>> or using them.
>>>
>>> However, the CPU load on both hosts is sitting at like 3.00, with the
>>> radosgw process taking up 99% CPU constantly. I do not see anything in the
>>> logs happening at all.
>>>
>>> Any thoughts or direction?
>>>
>> We've seen that happening when running on a system with older version
>> of libcurl (e.g., 7.29). If that's the case upgrading to a newer
>> version should fix it for you.
>>
>> Yehuda
>>
>>
>>> Have a good day,
>>>
>>> Lewis George
>>>
>>>
>>> _______________________________________________
>>> ceph-users mailing list
>>> ceph-users@xxxxxxxxxxxxxx
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>
>>
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
