Hi , PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 23921 kapil 25 0 811m 510m 6260 R 99.9 14.5 9:50.62 java Here , the process 23921 consumes all the cpu ( but in summary , cpu is shown idle , this is strange) Why does this process consume all the cpu? What does it do? Can you check: Strace -p 23291 ? What happens when you stop this ? Does the system work faster? Kind Regards, Tolga Evren -----Original Message----- From: redhat-list-bounces@xxxxxxxxxx [mailto:redhat-list-bounces@xxxxxxxxxx] On Behalf Of unix syzadmin Sent: Thursday, December 22, 2005 1:51 PM To: General Red Hat Linux discussion list Subject: Re: All RAM is being used,swap not being used at all. Linux running slow. Hi, top - 11:35:37 up 23:43, 10 users, load average: 2.45, 2.56, 2.63 Tasks: 249 total, 2 running, 247 sleeping, 0 stopped, 0 zombie Cpu(s): 5.9% us, 0.1% sy, 0.0% ni, 93.9% id, 0.0% wa, 0.1% hi, 0.1% si Mem: 3595048k total, 3508176k used, 86872k free, 10508k buffers Swap: 3148700k total, 144k used, 3148556k free, 1006012k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 23921 kapil 25 0 774m 358m 6248 R 99.1 10.2 6:34.45 java 24146 root 16 0 2428 1108 792 R 5.0 0.0 0:04.65 top 1826 root 16 0 9588 3492 2388 S 0.3 0.1 0:03.90 sendmail 1 root 16 0 1860 584 496 S 0.0 0.0 0:03.91 init 2 root RT 0 0 0 0 S 0.0 0.0 0:01.64 migration/0 3 root 34 19 0 0 0 S 0.0 0.0 0:00.06 ksoftirqd/0 4 root RT 0 0 0 0 S 0.0 0.0 0:00.52 migration/1 5 root 34 19 0 0 0 S 0.0 0.0 0:00.04 ksoftirqd/1 6 root RT 0 0 0 0 S 0.0 0.0 0:01.85 migration/2 7 root 34 19 0 0 0 S 0.0 0.0 0:00.01 ksoftirqd/2 8 root RT 0 0 0 0 S 0.0 0.0 0:00.44 migration/3 9 root 34 19 0 0 0 S 0.0 0.0 0:00.02 ksoftirqd/3 10 root RT 0 0 0 0 S 0.0 0.0 0:02.11 migration/4 11 root 34 19 0 0 0 S 0.0 0.0 0:00.04 ksoftirqd/4 12 root RT 0 0 0 0 S 0.0 0.0 0:00.71 migration/5 13 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/5 14 root RT 0 0 0 0 S 0.0 0.0 0:02.16 migration/6 15 root 34 19 0 0 0 S 0.0 0.0 0:00.03 ksoftirqd/6 16 root RT 0 0 0 0 S 0.0 0.0 0:00.57 migration/7 17 root 34 19 0 0 0 S 0.0 0.0 0:00.04 ksoftirqd/7 18 root RT 0 0 0 0 S 0.0 0.0 0:02.27 migration/8 19 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/8 ====================================================================== top - 11:37:16 up 23:45, 10 users, load average: 2.36, 2.50, 2.60 Tasks: 249 total, 4 running, 245 sleeping, 0 stopped, 0 zombie Cpu(s): 5.3% us, 0.8% sy, 0.0% ni, 93.7% id, 0.0% wa, 0.2% hi, 0.0% si Mem: 3595048k total, 3503328k used, 91720k free, 10540k buffers Swap: 3148700k total, 144k used, 3148556k free, 1009592k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 23921 kapil 25 0 786m 370m 6260 R 99.9 10.5 8:07.53 java 24146 root 16 0 2428 1108 792 R 5.3 0.0 0:10.07 top 24248 subreddy 18 0 0 0 0 R 3.8 0.0 0:00.12 java 158 root 15 0 0 0 0 S 0.6 0.0 0:26.06 pdflush 1 root 16 0 1860 584 496 S 0.0 0.0 0:03.91 init 2 root RT 0 0 0 0 S 0.0 0.0 0:01.65 migration/0 3 root 34 19 0 0 0 S 0.0 0.0 0:00.06 ksoftirqd/0 4 root RT 0 0 0 0 S 0.0 0.0 0:00.52 migration/1 5 root 34 19 0 0 0 S 0.0 0.0 0:00.04 ksoftirqd/1 6 root RT 0 0 0 0 S 0.0 0.0 0:01.85 migration/2 7 root 34 19 0 0 0 S 0.0 0.0 0:00.01 ksoftirqd/2 8 root RT 0 0 0 0 S 0.0 0.0 0:00.44 migration/3 9 root 34 19 0 0 0 S 0.0 0.0 0:00.02 ksoftirqd/3 10 root RT 0 0 0 0 S 0.0 0.0 0:02.11 migration/4 11 root 34 19 0 0 0 S 0.0 0.0 0:00.04 ksoftirqd/4 12 root RT 0 0 0 0 S 0.0 0.0 0:00.71 migration/5 13 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/5 14 root RT 0 0 0 0 S 0.0 0.0 0:02.18 migration/6 15 root 34 19 0 0 0 S 0.0 0.0 0:00.03 ksofti ====================================================================== top - 11:39:00 up 23:46, 10 users, load average: 2.50, 2.48, 2.58 Tasks: 248 total, 3 running, 245 sleeping, 0 stopped, 0 zombie Cpu0 : 0.0% us, 0.0% sy, 0.0% ni, 99.7% id, 0.0% wa, 0.3% hi, 0.0% si Cpu1 : 0.0% us, 0.6% sy, 0.0% ni, 98.8% id, 0.0% wa, 0.6% hi, 0.0% si Cpu2 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0%si Cpu3 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0%si Cpu4 : 0.3% us, 0.6% sy, 0.0% ni, 99.1% id, 0.0% wa, 0.0% hi, 0.0% si Cpu5 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0%si Cpu6 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0%si Cpu7 : 0.0% us, 0.0% sy, 0.0% ni, 99.7% id, 0.0% wa, 0.3% hi, 0.0% si Cpu8 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0%si Cpu9 : 0.0% us, 0.0% sy, 0.0% ni, 99.1% id, 0.0% wa, 0.9% hi, 0.0% si Cpu10 : 0.0% us, 0.0% sy, 0.0% ni, 99.7% id, 0.0% wa, 0.3% hi, 0.0% si Cpu11 : 94.1% us, 5.6% sy, 0.0% ni, 0.0% id, 0.0% wa, 0.3% hi, 0.0% si Cpu12 : 0.0% us, 0.0% sy, 0.0% ni, 99.4% id, 0.0% wa, 0.6% hi, 0.0% si Cpu13 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0%si Cpu14 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0%si Cpu15 : 0.0% us, 0.0% sy, 0.0% ni, 99.4% id, 0.0% wa, 0.6% hi, 0.0% si Cpu16 : 0.0% us, 0.0% sy, 0.0% ni, 99.7% id, 0.0% wa, 0.3% hi, 0.0% si Cpu17 : 0.0% us, 0.0% sy, 0.0% ni, 99.7% id, 0.0% wa, 0.3% hi, 0.0% si Cpu18 : 3.4% us, 0.9% sy, 0.0% ni, 95.6% id, 0.0% wa, 0.0% hi, 0.0% si Cpu19 : 0.0% us, 0.0% sy, 0.0% ni, 99.4% id, 0.0% wa, 0.3% hi, 0.3% si Cpu20 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0%si Cpu21 : 95.3% us, 4.7% sy, 0.0% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.0% si Cpu22 : 0.0% us, 0.0% sy, 0.0% ni, 99.7% id, 0.0% wa, 0.3% hi, 0.0% si Cpu23 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0%si Cpu24 : 0.3% us, 0.9% sy, 0.0% ni, 98.8% id, 0.0% wa, 0.0% hi, 0.0% si Cpu25 : 0.0% us, 0.0% sy, 0.0% ni, 99.7% id, 0.0% wa, 0.0% hi, 0.3% si Cpu26 : 0.0% us, 0.0% sy, 0.0% ni, 99.7% id, 0.0% wa, 0.3% hi, 0.0% si Cpu27 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0%si Cpu28 : 0.0% us, 0.0% sy, 0.0% ni, 99.1% id, 0.0% wa, 0.3% hi, 0.6% si Cpu29 : 0.0% us, 0.6% sy, 0.0% ni, 99.4% id, 0.0% wa, 0.0% hi, 0.0% si Cpu30 : 0.0% us, 0.0% sy, 0.0% ni, 99.4% id, 0.0% wa, 0.3% hi, 0.3% si Cpu31 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0%si Cpu32 : 0.0% us, 0.0% sy, 0.0% ni, 99.7% id, 0.0% wa, 0.0% hi, 0.3% si Cpu33 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0%si Mem: 3595048k total, 3589936k used, 5112k free, 10704k buffers Swap: 3148700k total, 144k used, 3148556k free, 955764k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 23921 kapil 25 0 811m 510m 6260 R 99.9 14.5 9:50.62 java 24146 root 16 0 2428 1108 792 R 6.2 0.0 0:13.87 top 159 root 15 0 0 0 0 S 0.6 0.0 0:44.49 kswapd0 158 root 15 0 0 0 0 S 0.3 0.0 0:26.18 pdflush The users over here install weblogic. They update a property file with the details of weblogic. They then start an ant script which will run the weblogic and runs test on weblogic. Thanks & Regards, -GnanaShekar- On 12/22/05, Tolga Evren <tolgae@xxxxxxxxxxx> wrote: > > Hi , > > Your server is idle. Can you explain the problem further? Is there any > program running?( Database , mail server etc?) What dou you see in top ? > > > > > Kind Regards, > Tolga Evren > > -----Original Message----- > From: redhat-list-bounces@xxxxxxxxxx > [mailto:redhat-list-bounces@xxxxxxxxxx] On Behalf Of unix syzadmin > Sent: Thursday, December 22, 2005 11:49 AM > To: General Red Hat Linux discussion list > Subject: Re: All RAM is being used,swap not being used at all. Linux > running slow. > > Hi, > > [root@bangpcplin3 network-scripts]# vmstat 1 10 > procs -----------memory---------- ---swap-- -----io---- --system-- > ----cpu---- > r b swpd free buff cache si so bi bo in cs us sy > id > wa > 3 0 144 237088 16676 1997272 0 0 1 1 0 4 2 > 0 > 97 0 > 2 0 144 232928 16676 1997272 0 0 80 0 26 167 5 > 1 > 94 0 > 1 1 144 231776 16676 1997272 0 0 128 168 48 155 4 > 2 > 93 1 > 1 1 144 230544 16676 1997272 0 0 104 0 27 166 5 > 1 > 94 0 > 2 0 144 228512 16676 1997272 0 0 72 0 23 169 5 > 1 > 94 0 > 2 0 144 224976 16676 1997272 0 0 88 0 26 244 5 > 1 > 94 0 > 2 0 144 221584 16676 1997272 0 0 56 0 16 137 5 > 1 > 94 0 > 2 0 144 219536 16676 1997272 0 0 40 0 16 145 5 > 1 > 94 0 > 2 0 144 217040 16676 1997272 0 0 96 0 26 149 5 > 1 > 94 0 > 1 1 144 214544 16676 1997788 0 0 124 0 34 182 5 > 1 > 94 1 > [root@bangpcplin3 network-scripts]# > > > [root@bangpcplin3 network-scripts]# sar -B 1 10 > Linux 2.6.9-5.EL (bangpcplin3) 12/22/2005 > > 09:45:53 AM pgpgin/s pgpgout/s fault/s majflt/s > 09:45:54 AM 0.00 0.00 725.25 0.00 > 09:45:55 AM 0.00 0.00 208.91 0.00 > > 09:45:55 AM pgpgin/s pgpgout/s fault/s majflt/s > 09:45:56 AM 20.00 4.00 518.00 0.00 > > 09:45:56 AM pgpgin/s pgpgout/s fault/s majflt/s > 09:45:57 AM 16.00 0.00 178.00 0.00 > > 09:45:57 AM pgpgin/s pgpgout/s fault/s majflt/s > 09:45:58 AM 15.84 0.00 193.07 0.00 > > 09:45:58 AM pgpgin/s pgpgout/s fault/s majflt/s > 09:45:59 AM 0.00 0.00 228.71 0.00 > > 09:45:59 AM pgpgin/s pgpgout/s fault/s majflt/s > 09:46:00 AM 4.00 0.00 436.00 0.00 > 09:46:01 AM 8.00 0.00 523.00 0.00 > > 09:46:01 AM pgpgin/s pgpgout/s fault/s majflt/s > 09:46:02 AM 83.17 0.00 447.52 0.00 > > 09:46:02 AM pgpgin/s pgpgout/s fault/s majflt/s > 09:46:03 AM 11.88 0.00 433.66 0.00 > Average: 15.94 0.40 388.45 0.00 > > > [root@bangpcplin3 network-scripts]# sar -W 1 10 > Linux 2.6.9-5.EL (bangpcplin3) 12/22/2005 > > 09:46:26 AM pswpin/s pswpout/s > 09:46:27 AM 0.00 0.00 > 09:46:28 AM 0.00 0.00 > > 09:46:28 AM pswpin/s pswpout/s > 09:46:29 AM 0.00 0.00 > > 09:46:29 AM pswpin/s pswpout/s > 09:46:30 AM 0.00 0.00 > > 09:46:30 AM pswpin/s pswpout/s > 09:46:31 AM 0.00 0.00 > > 09:46:31 AM pswpin/s pswpout/s > 09:46:32 AM 0.00 0.00 > > 09:46:32 AM pswpin/s pswpout/s > 09:46:33 AM 0.00 0.00 > 09:46:34 AM 0.00 0.00 > > 09:46:34 AM pswpin/s pswpout/s > 09:46:35 AM 0.00 0.00 > > 09:46:35 AM pswpin/s pswpout/s > 09:46:36 AM 0.00 0.00 > Average: 0.00 0.00 > > > > [root@bangpcplin3 network-scripts]# iostat -x 1 > Linux 2.6.9-5.EL (bangpcplin3) 12/22/2005 > > avg-cpu: %user %nice %sys %iowait %idle > 2.46 0.00 0.36 0.02 97.15 > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s > avgrq-sz avgqu-sz await svctm %util > sda 0.04 0.97 1.27 1.10 42.28 94.26 21.14 47.13 > 57.75 0.10 41.43 5.81 1.37 > sdb 0.06 0.06 0.14 0.22 5.44 2.39 2.72 1.20 > 21.47 0.01 28.94 7.62 0.28 > > avg-cpu: %user %nice %sys %iowait %idle > 5.51 0.00 0.52 0.23 93.74 > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s > avgrq-sz avgqu-sz await svctm %util > sda 0.00 0.00 7.92 0.00 63.37 0.00 31.68 0.00 > 8.00 0.10 12.12 12.12 9.60 > sdb 0.00 0.00 0.99 0.00 0.00 0.00 0.00 0.00 > 0.00 0.02 39.00 20.00 1.98 > > avg-cpu: %user %nice %sys %iowait %idle > 5.59 0.00 0.44 0.21 93.76 > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s > avgrq-sz avgqu-sz await svctm %util > sda 0.00 0.00 5.00 10.00 40.00 112.00 20.00 56.00 > 10.13 0.31 20.93 6.87 10.30 > sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 > 0.00 0.00 0.00 0.00 0.00 > [root@bangpcplin3 network-scripts]# cat /proc/meminfo > MemTotal: 3595048 kB > MemFree: 128072 kB > Buffers: 16740 kB > Cached: 2017848 kB > SwapCached: 0 kB > Active: 1340824 kB > Inactive: 1906264 kB > HighTotal: 0 kB > HighFree: 0 kB > LowTotal: 3595048 kB > LowFree: 128072 kB > SwapTotal: 3148700 kB > SwapFree: 3148556 kB > Dirty: 6504 kB > Writeback: 0 kB > Mapped: 1227656 kB > Slab: 198172 kB > Committed_AS: 5080108 kB > PageTables: 5248 kB > VmallocTotal: 2147483647 kB > VmallocUsed: 3308 kB > VmallocChunk: 2147479631 kB > HugePages_Total: 0 > HugePages_Free: 0 > Hugepagesize: 16384 kB > > > [root@bangpcplin3 network-scripts]# sar -r 1 10 > Linux 2.6.9-5.EL (bangpcplin3) 12/22/2005 > > 09:48:07 AM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree > kbswpused %swpused kbswpcad > 09:48:08 AM 116336 3478712 96.76 16740 2017848 > 3148556 144 0.00 0 > 09:48:09 AM 114736 3480312 96.81 16740 2017848 > 3148556 144 0.00 0 > > 09:48:09 AM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree > kbswpused %swpused kbswpcad > 09:48:10 AM 113584 3481464 96.84 16740 2017848 > 3148556 144 0.00 0 > > 09:48:10 AM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree > kbswpused %swpused kbswpcad > 09:48:11 AM 113264 3481784 96.85 16740 2017848 > 3148556 144 0.00 0 > > 09:48:11 AM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree > kbswpused %swpused kbswpcad > 09:48:12 AM 113024 3482024 96.86 16740 2017848 > 3148556 144 0.00 0 > > 09:48:12 AM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree > kbswpused %swpused kbswpcad > 09:48:13 AM 111872 3483176 96.89 16740 2017848 > 3148556 144 0.00 0 > > 09:48:13 AM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree > kbswpused %swpused kbswpcad > 09:48:14 AM 111552 3483496 96.90 16740 2017848 > 3148556 144 0.00 0 > 09:48:15 AM 110176 3484872 96.94 16740 2018364 > 3148556 144 0.00 0 > > 09:48:15 AM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree > kbswpused %swpused kbswpcad > 09:48:16 AM 102176 3492872 97.16 16740 2018364 > 3148556 144 0.00 0 > > 09:48:16 AM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree > kbswpused %swpused kbswpcad > 09:48:17 AM 100832 3494216 97.20 16740 2018364 > 3148556 144 0.00 0 > Average: 110755 3484293 96.92 16740 2018003 > 3148556 144 0.00 0 > > > > [root@bangpcplin3 network-scripts]# sar 1 10 > Linux 2.6.9-5.EL (bangpcplin3) 12/22/2005 > > 09:48:36 AM CPU %user %nice %system %iowait %idle > 09:48:37 AM all 4.77 0.00 1.23 0.20 93.79 > 09:48:38 AM all 4.89 0.00 1.11 0.15 93.85 > > 09:48:38 AM CPU %user %nice %system %iowait %idle > 09:48:39 AM all 5.12 0.00 0.99 0.00 93.89 > > 09:48:39 AM CPU %user %nice %system %iowait %idle > 09:48:40 AM all 7.83 0.00 0.96 0.00 91.21 > > 09:48:40 AM CPU %user %nice %system %iowait %idle > 09:48:41 AM all 4.57 0.00 1.52 0.09 93.81 > > 09:48:41 AM CPU %user %nice %system %iowait %idle > 09:48:42 AM all 8.34 0.00 1.20 0.06 90.40 > > 09:48:42 AM CPU %user %nice %system %iowait %idle > 09:48:43 AM all 4.67 0.00 1.44 0.44 93.45 > 09:48:44 AM all 4.02 0.00 1.85 0.35 93.78 > > 09:48:44 AM CPU %user %nice %system %iowait %idle > 09:48:45 AM all 4.91 0.00 1.14 0.20 93.75 > > 09:48:45 AM CPU %user %nice %system %iowait %idle > 09:48:46 AM all 3.81 0.00 2.46 0.44 93.29 > Average: all 5.29 0.00 1.39 0.19 93.12 > > > Thanks & Regards, > -GnanaShekar- > > On 12/22/05, Tolga Evren <tolgae@xxxxxxxxxxx> wrote: > > > > Can you send : > > > > Vmstat 1 10 > > > > sar -B 1 10 > > > > sar -W 1 10 > > > > iostat -x 1 > > > > cat /proc/meminfo > > > > sar -r 1 10 > > > > sar 1 10 > > > > ? > > > > Kind Regards, > > Tolga Evren > > > > > > > > -----Original Message----- > > From: redhat-list-bounces@xxxxxxxxxx > > [mailto:redhat-list-bounces@xxxxxxxxxx] On Behalf Of unix syzadmin > > Sent: Thursday, December 22, 2005 11:33 AM > > To: redhat-list@xxxxxxxxxx > > Subject: All RAM is being used,swap not being used at all. Linux > running > > slow. > > > > Hi, > > > > All RAM is being used, swap not being used at all. Linux running > slow, > > Please suggest. > > > > # free -m > > total used free shared buffers > > cached > > Mem: 3510 3506 4 0 15 > > 2511 > > -/+ buffers/cache: 979 2531 > > Swap: 3074 0 3074 > > > > Thanks & Regards, > > -GnanaShekar- > > -- > > redhat-list mailing list > > unsubscribe mailto:redhat-list-request@xxxxxxxxxx?subject=subscribe > > https://www.redhat.com/mailman/listinfo/redhat-list > > > > -- > > redhat-list mailing list > > unsubscribe mailto:redhat-list-request@xxxxxxxxxx?subject=unsubscribe > > https://www.redhat.com/mailman/listinfo/redhat-list > > > -- > redhat-list mailing list > unsubscribe mailto:redhat-list-request@xxxxxxxxxx?subject=subscribe > https://www.redhat.com/mailman/listinfo/redhat-list > > -- > redhat-list mailing list > unsubscribe mailto:redhat-list-request@xxxxxxxxxx?subject=unsubscribe > https://www.redhat.com/mailman/listinfo/redhat-list > -- redhat-list mailing list unsubscribe mailto:redhat-list-request@xxxxxxxxxx?subject=subscribe https://www.redhat.com/mailman/listinfo/redhat-list -- redhat-list mailing list unsubscribe mailto:redhat-list-request@xxxxxxxxxx?subject=unsubscribe https://www.redhat.com/mailman/listinfo/redhat-list