Re: Radosgw (civetweb) hangs once around 850 established connections

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Sean,

Did you check that the process isn't hitting some ulimits? cat
/proc/`pidof radosgw`/limits and compare with the num processes/num
FDs in use.

Cheers, Dan


On Tue, Mar 29, 2016 at 8:35 PM, seapasulli@xxxxxxxxxxxx
<seapasulli@xxxxxxxxxxxx> wrote:
> So an update for anyone else having this issue. It looks like radosgw either
> has a memory leak or it spools the whole object into ram or something.
>
> root@kh11-9:/etc/apt/sources.list.d# free -m
>              total       used       free     shared    buffers cached
> Mem:         64397      63775        621          0 3         46
> -/+ buffers/cache:      63725        671
> Swap:        65499      17630      47869
>
> root@kh11-9:/etc/apt/sources.list.d# ps faux | grep -iE "USE[R]|radosg[w]"
> USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> root      269910  134 95.2 90622120 62819128 ?   Ssl  12:31  79:37
> /usr/bin/radosgw --cluster=ceph --id rgw.kh11-9 -f
>
> The odd things are 1.) the disk is fine. 2.) the rest of the server seems
> very responsive. I can ssh into the server without any problems, curl out,
> wget, etc but radosgw is stuck in the mud
>
> This is after 150-300 wget requests to public objects, 2 radosgws freeze
> like this.  The cluster is health okay as well::
>
> root@kh11-9:~# grep -iE "health" ceph_report.json
>     "health": {
>         "health": {
>             "health_services": [
>                             "health": "HEALTH_OK"
>                             "health": "HEALTH_OK"
>                             "health": "HEALTH_OK"
>                     "health": "HEALTH_OK"
>                     "health": "HEALTH_OK"
>                     "health": "HEALTH_OK"
>         "overall_status": "HEALTH_OK",
>
> Has anyone seen this behavior before?
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux