Re: memory problems, I lose control of the entire server

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sun, Feb 28, 2010 at 5:24 PM, Alvise Nicoletti
<lists@xxxxxxxxxxxxxxxxxxx> wrote:
> Hi.
> I searched help from several months ago with no success.
> So I'll repeat again my problem hoping that this time someone will answer.
>
> I often (3 times in a month) control of my virtual server, so I have to call
> the provider to soft-reset it.
> The problem is that apache takes all the memory and the cpu usage of the
> server, and sometimes after that I have also kernel panic errors.
>
> Even with the server working, I have the apache2 daemons big from 30 to 50
> megabytes each one (now I use 9).
>
>
> In the logs I found stuff like this:
> Feb 28 12:54:09 utopia kernel: [408986.594006] apache2 invoked oom-killer:
> gfp_mask=0x1201d2, order=0, oomkilladj=0
> Feb 28 12:54:09 utopia kernel: [408986.594022] Pid: 6817, comm: apache2 Not
> tainted 2.6.28-18-server #59-Ubuntu
> Feb 28 12:54:09 utopia kernel: [408986.594026] Call Trace:
> Feb 28 12:54:09 utopia kernel: [408986.594080]  [<ffffffff802b3825>]
> oom_kill_process+0x95/0x240
> Feb 28 12:54:09 utopia kernel: [408986.594087]  [<ffffffff802b3fdf>] ?
> select_bad_process+0xef/0x130
> Feb 28 12:54:09 utopia kernel: [408986.594091]  [<ffffffff802b40d4>]
> out_of_memory+0xb4/0x150
> Feb 28 12:54:09 utopia kernel: [408986.594095]  [<ffffffff802b6c49>]
> __alloc_pages_internal+0x4a9/0x4f0
> Feb 28 12:54:09 utopia kernel: [408986.594101]  [<ffffffff802b9a7a>]
> __do_page_cache_readahead+0xda/0x210
> Feb 28 12:54:09 utopia kernel: [408986.594105]  [<ffffffff802b9c0e>]
> do_page_cache_readahead+0x5e/0x90
> Feb 28 12:54:09 utopia kernel: [408986.594108]  [<ffffffff802b201a>]
> filemap_fault+0x34a/0x430
> Feb 28 12:54:09 utopia kernel: [408986.594113]  [<ffffffff802c6010>]
> __do_fault+0x50/0x520
> Feb 28 12:54:09 utopia kernel: [408986.594117]  [<ffffffff802c7199>]
> handle_mm_fault+0x1e9/0x470
> Feb 28 12:54:09 utopia kernel: [408986.594134]  [<ffffffff802f3a1f>] ?
> path_walk+0xbf/0xd0
> Feb 28 12:54:09 utopia kernel: [408986.594158]  [<ffffffff8022f669>] ?
> default_spin_lock_flags+0x9/0x10
> Feb 28 12:54:09 utopia kernel: [408986.594177]  [<ffffffff8069e0d7>]
> do_page_fault+0x307/0x780
> Feb 28 12:54:09 utopia kernel: [408986.594183]  [<ffffffff802eb9f3>] ?
> cp_new_stat+0xe3/0xf0
> Feb 28 12:54:09 utopia kernel: [408986.594186]  [<ffffffff802ebee6>] ?
> sys_newstat+0x36/0x50
> Feb 28 12:54:09 utopia kernel: [408986.594198]  [<ffffffff8069b6ba>]
> error_exit+0x0/0x70
> Feb 28 12:54:09 utopia kernel: [408986.594202] Mem-Info:
> Feb 28 12:54:09 utopia kernel: [408986.594205] DMA per-cpu:
> Feb 28 12:54:09 utopia kernel: [408986.594207] CPU    0: hi:    0, btch:   1
> usd:   0
> Feb 28 12:54:09 utopia kernel: [408986.594210] CPU    1: hi:    0, btch:   1
> usd:   0
> Feb 28 12:54:09 utopia kernel: [408986.594212] DMA32 per-cpu:
> Feb 28 12:54:09 utopia kernel: [408986.594214] CPU    0: hi:  186, btch:  31
> usd: 183
> Feb 28 12:54:09 utopia kernel: [408986.594216] CPU    1: hi:  186, btch:  31
> usd: 172
> Feb 28 12:54:09 utopia kernel: [408986.594226] Active_anon:363780
> active_file:19 inactive_anon:121250
> Feb 28 12:54:09 utopia kernel: [408986.594228]  inactive_file:91
> unevictable:0 dirty:0 writeback:0 unstable:0
> Feb 28 12:54:09 utopia kernel: [408986.594229]  free:3066 slab:4774 mapped:1
> pagetables:7473 bounce:0
> Feb 28 12:54:09 utopia kernel: [408986.594234] DMA free:6644kB min:12kB
> low:12kB high:16kB active_anon:0kB inactive_anon:0kB acti$
> Feb 28 12:54:09 utopia kernel: [408986.594237] lowmem_reserve[]: 0 2004 2004
> 2004
> Feb 28 12:54:09 utopia kernel: [408986.594244] DMA32 free:5620kB min:5720kB
> low:7148kB high:8580kB active_anon:1455120kB inactive$
> Feb 28 12:54:09 utopia kernel: [408986.594247] lowmem_reserve[]: 0 0 0 0
> Feb 28 12:54:09 utopia kernel: [408986.594259] DMA: 3*4kB 3*8kB 3*16kB
> 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 1*1024kB 0*2048kB 1*$
> Feb 28 12:54:09 utopia kernel: [408986.594272] DMA32: 155*4kB 1*8kB 0*16kB
> 0*32kB 0*64kB 1*128kB 1*256kB 1*512kB 0*1024kB 0*2048k$
> Feb 28 12:54:09 utopia kernel: [408986.594284] 828 total pagecache pages
> Feb 28 12:54:09 utopia kernel: [408986.594286] 0 pages in swap cache
> Feb 28 12:54:09 utopia kernel: [408986.594288] Swap cache stats: add 4551,
> delete 4551, find 160864/160937
> Feb 28 12:54:09 utopia kernel: [408986.594297] Free swap  = 0kB
> Feb 28 12:54:09 utopia kernel: [408986.594299] Total swap = 0kB
> Feb 28 12:54:09 utopia kernel: [408986.608274] 524272 pages RAM
> Feb 28 12:54:09 utopia kernel: [408986.608283] 15596 pages reserved
> Feb 28 12:54:09 utopia kernel: [408986.608284] 63040 pages shared
> Feb 28 12:54:09 utopia kernel: [408986.608286] 500299 pages non-shared
> Feb 28 12:54:09 utopia kernel: [408986.608292] Out of memory: kill process
> 6817 (apache2) score 63892 or a child
> Feb 28 12:54:09 utopia kernel: [408986.609426] Killed process 6817 (apache2)
> Feb 28 12:54:17 utopia kernel: [409094.110911] pickup invoked oom-killer:
> gfp_mask=0x1201d2, order=0, oomkilladj=0
> Feb 28 12:54:17 utopia kernel: [409094.110925] Pid: 5668, comm: pickup Not
> tainted 2.6.28-18-server #59-Ubuntu
> Feb 28 12:54:17 utopia kernel: [409094.110929] Call Trace:
> (and so on)
>
> This is my configuration:
> Linux utopia 2.6.28-18-server #59-Ubuntu SMP Thu Jan 28 02:25:03 UTC 2010
> x86_64 GNU/Linux
> Server version: Apache/2.2.11 (Ubuntu)
> Server built:   Nov 13 2009 22:06:57
> Server's Module Magic Number: 20051115:21
> Server loaded:  APR 1.2.12, APR-Util 1.2.12
> Compiled using: APR 1.2.12, APR-Util 1.2.12
> Architecture:   64-bit
> Server MPM:     Prefork
>  threaded:     no
>   forked:     yes (variable process count)
> Server compiled with....
> -D APACHE_MPM_DIR="server/mpm/prefork"
> -D APR_HAS_SENDFILE
> -D APR_HAS_MMAP
> -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
> -D APR_USE_SYSVSEM_SERIALIZE
> -D APR_USE_PTHREAD_SERIALIZE
> -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
> -D APR_HAS_OTHER_CHILD
> -D AP_HAVE_RELIABLE_PIPED_LOGS
> -D DYNAMIC_MODULE_LIMIT=128
> -D HTTPD_ROOT=""
> -D SUEXEC_BIN="/usr/lib/apache2/suexec"
> -D DEFAULT_PIDLOG="/var/run/apache2.pid"
> -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
> -D DEFAULT_LOCKFILE="/var/run/apache2/accept.lock"
> -D DEFAULT_ERRORLOG="logs/error_log"
> -D AP_TYPES_CONFIG_FILE="/etc/apache2/mime.types"
> -D SERVER_CONFIG_FILE="/etc/apache2/apache2.conf"
>
> This is the main configuration of apache2:
> <IfModule mpm_prefork_module>
>   StartServers          5
>   MinSpareServers       5
>   MaxSpareServers      9
>   MaxClients          80
>   MaxRequestsPerChild   400
>   MaxMemFree  200000
> </IfModule>
>
> KeepAliveTimeout 3
> MaxKeepAliveRequests 50
> KeepAlive On
> Timeout 200
>

Add more memory/reduce number of clients/disable KeepAlive.

30-50 MB per child sounds large, unless you are using PHP, in which
case it sounds about right.

You also seem to allow a very large number of children - MaxClients
80. If you are running out of memory, and can't add any more, don't
configure apache so that it can use 2.4 GB - 4 GB of RAM.

KeepAlive is good for clients, less good for pre-fork webservers
consuming lots of resources. Each client you KeepAlive adds one more
unusable child, another 30-50MB of memory used.

Cheers

Tom

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
   "   from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx



[Index of Archives]     [Open SSH Users]     [Linux ACPI]     [Linux Kernel]     [Linux Laptop]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Squid]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]

  Powered by Linux