On Thu, 10 May 2012 15:39:04 +0530, Nikunj A Dadhania <nikunj@xxxxxxxxxxxxxxxxxx> wrote: > On Thu, 10 May 2012 12:22:00 +0300, Avi Kivity <avi@xxxxxxxxxx> wrote: > > On 05/10/2012 11:15 AM, Nikunj A Dadhania wrote: > > > I am running a 3.4.0-rc4 based kernel(commit: af3a3ab2), guest config > > > attached. > > > > > > During my tests, I saw few softlockups inside the guests while running > > > kernbench inside the guest. I can reproduce this repeatedly. > > > > > > Test Setup: > > > > > > - Create 4 VMs (8vcpu, 1GB RAM) > > > - Run kernbench inside each guests (kernbench -f -M -H -o 16) in parallel > > > > > > > How overcommitted are you? > 8 Physical CPU and 32 vCPUs > > > What does the host hardware look like? > Machine : IBM xSeries with Intel(R) Xeon(R) x5570 2.93GHz CPU with 8 core , 64GB RAM > Threading is disabled. > > > Is there any load on the host? > No > > > > > The traces themselves always point a couple of instructions after > > interrupts are enabled. Can you post a few more? > > I had a discussion with Avi on IRC, he suggested running a trace on the host for the sched: tracepoints So when I see the console of the guest, i can see the soft-lockup messages. When I start tracing, soft-lockups disappear. Kernbench is started from the host; ps output: 13359 pts/1 S+ 0:00 perf stat -a -- ssh root@192.168.123.11 cd /root/linux-src; kernbench -f -M -H -o 16 > /root/kbench.log 13362 pts/1 S+ 0:00 ssh root@192.168.123.11 cd /root/linux-src; kernbench -f -M -H -o 16 > /root/kbench.log 13373 pts/1 S+ 0:00 perf stat -a -- ssh root@192.168.123.12 cd /root/linux-src; kernbench -f -M -H -o 16 > /root/kbench.log 13376 pts/1 S+ 0:00 ssh root@192.168.123.12 cd /root/linux-src; kernbench -f -M -H -o 16 > /root/kbench.log 13387 pts/1 S+ 0:00 perf stat -a -- ssh root@192.168.123.13 cd /root/linux-src; kernbench -f -M -H -o 16 > /root/kbench.log 13390 pts/1 S+ 0:00 ssh root@192.168.123.13 cd /root/linux-src; kernbench -f -M -H -o 16 > /root/kbench.log 13401 pts/1 S+ 0:00 perf stat -a -- ssh root@192.168.123.14 cd /root/linux-src; kernbench -f -M -H -o 16 > /root/kbench.log 13402 pts/1 S+ 0:00 ssh root@192.168.123.14 cd /root/linux-src; kernbench -f -M -H -o 16 > /root/kbench.log I have tried following commands trace-cmd: 1) trace-cmd record -e kvm -e sched -b 100000 -p 13362 2) trace-cmd record -e sched -b 100000 -p 13362 3) trace-cmd record -e kvm -b 100000 -p 13362 I have tried trace-cmd on qemu pid as well, I see similar behaviour in that case. Peterz/Ingo, Any clue on debugging this would help. Regards Nikunj -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
