Hello @All,
We have a problem on our production machine with high CPU utilization
caused by kswapd3 daemon. Server is 128GB of physical memory and 81GB
of SWAP.
# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.6 (Tikanga)
#
# uptime
09:52:13 up 38 days, 17:01, 14 users, load average: 43.93, 50.02, 51.36
#
# free -m
total used free shared buffers cached
Mem: 128989 75577 53412 0 416 57131
-/+ buffers/cache: 18029 110960
Swap: 81919 31310 50609
#
### from TOP:
Tasks: 1590 total, 3 running, 1578 sleeping, 1 stopped, 8 zombie
Cpu(s): 1.1%us, 1.9%sy, 0.0%ni, 74.4%id, 22.4%wa, 0.0%hi, 0.2%si, 0.0%st
Mem: 132085528k total, 77397544k used, 54687984k free, 426472k buffers
Swap: 83886064k total, 32043168k used, 51842896k free, 58475692k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1816 root 10 -5 0 0 0 D 203.0 0.0 1124:39 kswapd3
10801 root 15 0 113m 23m 1680 S 158.6 0.0 1694:13 vxconfigd
3925 root RT 0 162m 15m 3368 S 140.1 0.0 1313:32 multipathd
21399 wsy5067 16 0 22768 3192 1704 S 50.3 0.0 0:12.96 top
10583 root 15 0 0 0 0 S 43.8 0.0 58:56.55 dmp_daemon
16673 root 16 0 17720 1220 792 S 37.3 0.0 732:23.49 cmafcad
24636 oracle 15 0 3314m 41m 40m S 26.6 0.0 29:31.84 oracle
487 oracle 15 0 3301m 90m 88m S 21.4 0.1 1:05.68 oracle
14197 root 18 0 315m 15m 3984 S 21.1 0.0 346:15.70 vxsvc
26071 root 15 0 23900 6788 2092 S 19.8 0.0 12:31.44 MountAgent
16418 root 15 0 5308 3656 592 S 19.1 0.0 544:54.78 cmahostd
485 oracle 15 0 3301m 87m 87m S 18.8 0.1 1:03.92 oracle
21456 root 15 0 0 0 0 S 18.5 0.0 14:16.09 pdflush
4699 root 22 0 65204 8420 1568 S 15.6 0.0 137:15.75 vxdclid
16603 root 18 0 8820 6808 332 D 14.3 0.0 1290:32 cmaperfd
22047 tlmcron1 18 0 1227m 115m 6812 S 12.3 0.1 554:55.86 java
6157 oracle 16 0 3302m 118m 115m D 11.4 0.1 0:01.29 oracle
30669 oracle 16 0 2066m 141m 134m D 10.7 0.1 0:20.19 oracle
357 tlmtst3 18 0 101m 15m 9292 S 9.7 0.0 0:29.43 stlfetch
15005 oracle 16 0 2046m 84m 75m D 9.7 0.1 0:01.75 oracle
32744 tlmtst3 18 0 101m 15m 9292 S 8.8 0.0 0:31.52 stlfetch
26077 root 15 0 12116 4228 1936 S 7.8 0.0 4:26.71 VolumeAgent
17955 oracle 16 0 3301m 28m 25m D 7.1 0.0 0:00.22 oracle
26076 root 15 0 23268 5284 2224 S 6.8 0.0 4:00.50 OracleAgent
111 root 10 -5 0 0 0 S 6.5 0.0 4:57.32 events/13
5112 oracle 16 0 3314m 35m 35m S 6.5 0.0 10:38.55 oracle
10440 oracle 16 0 3314m 36m 36m S 6.5 0.0 10:54.65 oracle
26074 root 15 0 23012 4904 2136 S 6.5 0.0 3:09.46 NetlsnrAgent
8631 oracle 16 0 3315m 37m 36m S 6.2 0.0 12:04.57 oracle
26636 oracle 15 0 3316m 40m 39m S 6.2 0.0 13:07.75 oracle
10780 oracle 16 0 6386m 39m 39m D 5.8 0.0 10:57.01 oracle
13091 tlmtst5 15 0 423m 18m 13m S 5.8 0.0 0:02.76 gedit
17212 oracle 15 0 2050m 39m 39m S 5.8 0.0 11:30.82 oracle
20283 oracle 15 0 3314m 41m 40m S 5.8 0.0 10:49.00 oracle
13147 oracle 15 0 3314m 43m 43m S 5.5 0.0 10:43.12 oracle
16875 oracle 15 0 3316m 42m 40m S 5.5 0.0 22:52.66 oracle
28143 root 15 0 111m 6700 3396 S 5.5 0.0 143:25.46 opcmona
5718 oracle 16 0 3314m 38m 36m S 5.2 0.0 11:30.95 oracle
22163 oracle 15 0 3316m 39m 38m S 5.2 0.0 32:51.94 oracle
26065 root 15 0 22960 4724 2164 S 4.5 0.0 1:53.23 ApplicationAgen
26612 oracle 18 0 3302m 22m 19m S 4.2 0.0 23:31.48 oracle
8597 oracle 18 0 3302m 18m 16m S 3.9 0.0 23:30.62 oracle
10690 oracle 18 0 6373m 20m 16m S 3.9 0.0 23:28.74 oracle
11822 oracle 18 0 2038m 29m 25m S 3.9 0.0 0:05.17 oracle
Why the kswapd daemon causing so high CPU utilization? Can it be
caused by some process which try to access memory pages which are not
in memory, but in swap? Or can it be cause by swapping for exapmple
some inactive oracle database SGA from memory to swap?
Thank you for help.
Regards,
Karol Sebesta
--
To unsubscribe from this list: send the line "unsubscribe linux-admin" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
