Re: High CPU load with radosgw instances

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Casey,

That warning message tells users to upgrade to a new version of
libcurl. Telling users to upgrade to a newer version of a base system
package like that sets the user on a trajectory to have to maintain
their own curl packages forever, decreasing the security of their
overall system in the long run. For example ceph.com itself shipped a
newer el6 curl package for a while in "ceph-extras", until it fell of
everyone's radar, no one updated it, and it had many outstanding CVEs
until we finally dropped el6 support altogether.

Would you please remove this last sentence from the RGW log message?

- Ken


On Fri, Sep 16, 2016 at 12:02 PM, Casey Bodley <cbodley@xxxxxxxxxx> wrote:
> In the meantime, we've made changes to radosgw so that it can detect and
> work around this libcurl bug. You can track the progress of this workaround
> (currently in master and pending backport to jewel) at
> http://tracker.ceph.com/issues/16695.
>
> Casey
>
>
>
> On 09/16/2016 01:38 PM, Ken Dreyer wrote:
>>
>> Hi Lewis,
>>
>> This sounds a lot like https://bugzilla.redhat.com/1347904 , currently
>> slated for the upcoming RHEL 7.3 (and CentOS 7.3).
>>
>> There's an SRPM in that BZ that you can rebuild and test out. This
>> method won't require you to keep chasing upstream curl versions
>> forever (curl has a lot of CVEs).
>>
>> Mind testing that out and reporting back?
>>
>> - Ken
>>
>>
>> On Fri, Sep 16, 2016 at 11:06 AM, lewis.george@xxxxxxxxxxxxx
>> <lewis.george@xxxxxxxxxxxxx> wrote:
>>>
>>> Hi Yehuda,
>>> Well, again, thank you!
>>>
>>> I was able to get a package built from the latest curl release, and after
>>> upgrading on my radosgw hosts, the load is no longer running high. The
>>> load
>>> is just sitting at almost nothing and I only see the radosgw process
>>> using
>>> CPU when it is actually doing something now.
>>>
>>> So, I am still curious if this would be considered a bug or not, since
>>> the
>>> curl version from the base CentOS repo seems to have an issue.
>>>
>>> Have a good day,
>>>
>>> Lewis George
>>>
>>>
>>>
>>> ________________________________
>>> From: "lewis.george@xxxxxxxxxxxxx" <lewis.george@xxxxxxxxxxxxx>
>>> Sent: Friday, September 16, 2016 7:28 AM
>>> To: "Yehuda Sadeh-Weinraub" <yehuda@xxxxxxxxxx>
>>>
>>> Cc: "ceph-users@xxxxxxxxxxxxxx" <ceph-users@xxxxxxxxxxxxxx>
>>> Subject: Re:  High CPU load with radosgw instances
>>>
>>> Hi Yehuda,
>>> Thank you for the idea. I will try to test that and see if it helps.
>>>
>>> If that is the case, would that be considered a bug with radosgw? I ask
>>> because, that version of curl seems to be what is currently standard on
>>> RHEL/CentOS 7 (fully updated). I will have to either compile it or search
>>> 3rd-party repos for newer version, which is not usually something that is
>>> great.
>>>
>>> Have a good day,
>>>
>>> Lewis George
>>>
>>>
>>> ________________________________
>>> From: "Yehuda Sadeh-Weinraub" <yehuda@xxxxxxxxxx>
>>> Sent: Thursday, September 15, 2016 10:42 PM
>>> To: lewis.george@xxxxxxxxxxxxx
>>> Cc: "ceph-users@xxxxxxxxxxxxxx" <ceph-users@xxxxxxxxxxxxxx>
>>> Subject: Re:  High CPU load with radosgw instances
>>>
>>> On Thu, Sep 15, 2016 at 4:53 PM, lewis.george@xxxxxxxxxxxxx
>>> <lewis.george@xxxxxxxxxxxxx> wrote:
>>>>
>>>> Hi,
>>>> So, maybe someone has an idea of where to go on this.
>>>>
>>>> I have just setup 2 rgw instances in a multisite setup. They are working
>>>> nicely. I have add a couple of test buckets and some files to make sure
>>>> it
>>>> works is all. The status shows both are caught up. Nobody else is
>>>> accessing
>>>> or using them.
>>>>
>>>> However, the CPU load on both hosts is sitting at like 3.00, with the
>>>> radosgw process taking up 99% CPU constantly. I do not see anything in
>>>> the
>>>> logs happening at all.
>>>>
>>>> Any thoughts or direction?
>>>>
>>> We've seen that happening when running on a system with older version
>>> of libcurl (e.g., 7.29). If that's the case upgrading to a newer
>>> version should fix it for you.
>>>
>>> Yehuda
>>>
>>>
>>>> Have a good day,
>>>>
>>>> Lewis George
>>>>
>>>>
>>>> _______________________________________________
>>>> ceph-users mailing list
>>>> ceph-users@xxxxxxxxxxxxxx
>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>>
>>>
>>> _______________________________________________
>>> ceph-users mailing list
>>> ceph-users@xxxxxxxxxxxxxx
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux