Re: suddendly high cpu load because of googlebot

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



What are your TimeOut and KeepAliveTimeout values?

On Mar 9, 2012 7:08 PM, "Simone Frattegiani" <simone.frattegiani@xxxxxxxxx> wrote:
Hello,

i suddendly started having CPU load issues, like this:

top - 03:52:04 up 3 days, 8:52, 0 users, load average: 74.35, 73.66, 71.41
Tasks: 304 total, 75 running, 229 sleeping, 0 stopped, 0 zombie
Cpu(s): 72.1%us, 2.0%sy, 0.1%ni, 23.9%id, 1.2%wa, 0.4%hi, 0.4%si, 0.0%st
Mem: 3115464k total, 2840484k used, 274980k free, 197472k buffers
Swap: 5177336k total, 76k used, 5177260k free, 1702148k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19392 apache 25 0 34096 15m 3548 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
start -DSSL
19480 apache 24 0 34172 15m 3788 R 19.6 0.5 0:04.79 /usr/sbin/httpd -k
start -DSSL
19481 apache 25 0 34048 14m 3436 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
start -DSSL
19482 apache 25 0 34180 14m 3500 R 19.6 0.5 0:06.63 /usr/sbin/httpd -k
start -DSSL
19881 apache 25 0 34048 14m 3436 R 19.6 0.5 0:10.49 /usr/sbin/httpd -k
start -DSSL
20262 root 20 0 2424 1032 696 R 2.0 0.0 0:00.01 /usr/bin/top -c -b -n 1
1 root 15 0 2160 596 516 S 0.0 0.0 0:00.62 init [3]
2 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [migration/0]
3 root 34 19 0 0 0 R 0.0 0.0 0:00.00 [ksoftirqd/0]
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/0]

[...]


I can also see from my panel that apache spawned like 50 children,
which is very unusual.


Server-status shows this:

Total Accesses: 292068
Total kBytes: 5540956
CPULoad: .0884054
Uptime: 45823
ReqPerSec: 6.37383
BytesPerSec: 123823
BytesPerReq: 19426.8
BusyWorkers: 99
IdleWorkers: 10
Scoreboard: WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
WWWWWWWWWWWWWWWWWWWWWWWWWWKWWWWWCWWWWWWWWWKWWWWWWWWWWKWWW_W_CWWW__WS__K_WWK__K_S........
........................................................
...................................................

With hundreds of lines like this (66.249.66.44 is googlebot):

24-0 - 0/0/8 . 0.02 661 34 0.0 0.00 0.00 66.249.66.44
www.mydomain.com        GET /mypage

So, please correct me if i'm wrong, that 661 in the SS column means
that 661 seconds have passed since the last time Googlebot used that
slot to make a request... so basically the server is keeping alive a
connection that has been idle for like 11 minutes!!
Did i understand correctly? Is it normal? Is it what is causing the problem?

The commands "netstat -a | grep 66.249" shows 20 connections from
googlebot ip, 1 in ESTABILISHEd status, the others in TIME WAIT.

If i restart apache, everything gets back to normal.

Any suggestions?
THanks!

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx


[Index of Archives]     [Open SSH Users]     [Linux ACPI]     [Linux Kernel]     [Linux Laptop]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Squid]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]

  Powered by Linux