Hi Jan,
Thanks for your help.
On Thu, Feb 24, 2011 at 2:26 AM, Jan Kiszka <jan.kiszka@xxxxxxxxxxx> wrote:
> On 2011-02-23 16:53, Ryousei Takano wrote:
>> Hi Jan,
>>
>> On Wed, Feb 23, 2011 at 11:47 PM, Jan Kiszka <jan.kiszka@xxxxxxxxxxx> wrote:
>>> On 2011-02-23 15:38, Ryousei Takano wrote:
>>>> This reverts commit c65827770b5e7282b5fb9da29c1de85ed08e816e
>>>
>>> Reasoning would belong here, not below the separator.
>>>
>>>>
>>>> Signed-off-by: Ryousei Takano <takano-ryousei@xxxxxxxxxx>
>>>> ---
>>>> I am trying to run OpenMP (i.e., multi-threaded) programs using Intel
>>>> Math Kernel Library (MKL) on a guest OS. In the qemu-kvm-0.12.5,
>>>> my program works fine. In the qemu-kvm-0.13.0, however, it does not.
>>>
>>> What about 0.14?
>>>
>> 0.14-rc1 also has problem.
>
> Not automagically healed - what a pity.
>
>>
>>>> The Linux kernel is 2.6.32.28. The outputs of 'ps -L' are shown as follows:
>>>>
>>>> qemu-kvm-0.12.5:
>>>> PID LWP TTY TIME CMD
>>>> 2731 2731 pts/0 00:00:00 bash
>>>> 2799 2799 pts/0 00:00:19 a.out
>>>> 2799 2803 pts/0 00:00:00 a.out
>>>> 2799 2804 pts/0 00:00:04 a.out <-
>>>> 2799 2805 pts/0 00:00:03 a.out <- these threads are working properly.
>>>> 2799 2806 pts/0 00:00:04 a.out <-
>>>> 2807 2807 pts/0 00:00:00 ps
>>>>
>>>> qemu-0.13.0:
>>>> PID LWP TTY TIME CMD
>>>> 2725 2725 pts/0 00:00:00 bash
>>>> 2748 2748 pts/0 00:00:19 a.out
>>>> 2748 2752 pts/0 00:00:00 a.out
>>>> 2748 2753 pts/0 00:00:00 a.out <-
>>>> 2748 2754 pts/0 00:00:00 a.out <- these threads are blocked by
>>>> pthread_cond_wait(3).
>>>> 2748 2755 pts/0 00:00:00 a.out <-
>>>> 2756 2756 pts/0 00:00:00 ps
>>>>
>>>> This problem seems to be introduced by the commit
>>>> c65827770b5e7282b5fb9da29c1de85ed08e816e.
>>>> The attached revert patch solves it. But I do not understand what is
>>>> happened;-)
>>>> Any comments and suggestions will be appreciated.
>>>>
>>>> Regards,
>>>> Ryousei
>>>>
>>>> hw/pc.c | 6 ++++++
>>>> target-i386/helper.c | 2 --
>>>> 2 files changed, 6 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/hw/pc.c b/hw/pc.c
>>>> index ec97181..577abe7 100644
>>>> --- a/hw/pc.c
>>>> +++ b/hw/pc.c
>>>> @@ -937,6 +937,12 @@ CPUState *pc_new_cpu(const char *cpu_model)
>>>> }
>>>> qemu_register_reset(pc_cpu_reset, env);
>>>> pc_cpu_reset(env);
>>>> +
>>>> + /* kvm needs this to run after the apic is initialized. Otherwise,
>>>> + * it can access invalid state and crash.
>>>> + */
>>>> + qemu_init_vcpu(env);
>>>> +
>>>> return env;
>>>> }
>>>>
>>>> diff --git a/target-i386/helper.c b/target-i386/helper.c
>>>> index 86202e6..69cda5e 100644
>>>> --- a/target-i386/helper.c
>>>> +++ b/target-i386/helper.c
>>>> @@ -1224,8 +1224,6 @@ CPUX86State *cpu_x86_init(const char *cpu_model)
>>>> }
>>>> mce_init(env);
>>>>
>>>> - qemu_init_vcpu(env);
>>>> -
>>>> return env;
>>>> }
>>>>
>>>
>>> I don't believe yet that the patch is the reason, specifically as it
>>> targets the setup phase (except for CPU hotplug, but that's broken for
>>> ages), not the operational phase of the VM.
>>>
>>> What's your qemu-kvm command line?
>>>
>> Here is my command line.
>>
>> sudo qemu-system-x86_64 \
>> -cpu host \
>> -smp 8 \
>> -m 46080 \
>> -net nic,model=virtio,macaddr=00:16:3e:1d:ff:00 \
>> -net tap,ifname=tap0,script=no,downscript=no \
>> -monitor telnet::5963,server,nowait \
>> -daemonize \
>> disk.img
>
> How many cores are available on the host? >= 8 or less?
>
8 cores. All cores are allocated to the VM.
If 7 cores are allocated, the problem is reproduced.
>>
>>> Also, please attach gdb to the stuck VM and print the backtraces of all
>>> threads ("thread apply all bt").
>>>
>> (gdb) thread apply all bt
>> Thread 9 (Thread 0x41129940 (LWP 8118)):
>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
>> #1 0x000000000042e56d in pthread_attr_setdetachstate ()
>> #2 0x000000000042ea99 in pthread_attr_setdetachstate ()
>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate ()
>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
>>
>> Thread 8 (Thread 0x41ca8940 (LWP 8119)):
>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
>> #1 0x000000000042e56d in pthread_attr_setdetachstate ()
>> #2 0x000000000042ea99 in pthread_attr_setdetachstate ()
>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate ()
>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
>>
>> Thread 7 (Thread 0x425f2940 (LWP 8120)):
>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
>> #1 0x000000000042e56d in pthread_attr_setdetachstate ()
>> #2 0x000000000042ea99 in pthread_attr_setdetachstate ()
>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate ()
>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
>>
>> Thread 6 (Thread 0x42df3940 (LWP 8121)):
>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
>> #1 0x000000000042e56d in pthread_attr_setdetachstate ()
>> #2 0x000000000042ea99 in pthread_attr_setdetachstate ()
>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate ()
>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
>>
>> Thread 5 (Thread 0x435f4940 (LWP 8122)):
>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
>> #1 0x000000000042e56d in pthread_attr_setdetachstate ()
>> #2 0x000000000042ea99 in pthread_attr_setdetachstate ()
>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate ()
>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
>>
>> Thread 4 (Thread 0x43df5940 (LWP 8123)):
>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
>> #1 0x000000000042e56d in pthread_attr_setdetachstate ()
>> #2 0x000000000042ea99 in pthread_attr_setdetachstate ()
>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate ()
>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
>>
>> Thread 3 (Thread 0x445f6940 (LWP 8124)):
>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
>> #1 0x000000000042e56d in pthread_attr_setdetachstate ()
>> #2 0x000000000042ea99 in pthread_attr_setdetachstate ()
>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate ()
>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
>>
>> Thread 2 (Thread 0x44df7940 (LWP 8125)):
>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
>> #1 0x000000000042e56d in pthread_attr_setdetachstate ()
>> #2 0x000000000042ea99 in pthread_attr_setdetachstate ()
>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate ()
>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
>>
>> Thread 1 (Thread 0x7fe796ec1740 (LWP 8117)):
>> #0 0x0000003dd38ccfc2 in select () from /lib64/libc.so.6
>> #1 0x0000000000419620 in pthread_attr_setdetachstate ()
>> #2 0x000000000042be07 in pthread_attr_setdetachstate ()
>> #3 0x000000000041ae20 in pthread_attr_setdetachstate ()
>> #4 0x0000003dd381d994 in __libc_start_main () from /lib64/libc.so.6
>> #5 0x00000000004063d9 in pthread_attr_setdetachstate ()
>> #6 0x00007fff8050c988 in ?? ()
>> #7 0x0000000000000000 in ?? ()
>>
>
> Can you retry with --enable-debug set during qemu-kvm configure? We need
> the symbols of the qemu functions. Despite the lacking symbols, this
> rather looks like we are stuck in kernel space, not on some condition
> variable (ie. futex).
>
Here is the result with --enable-debug.
Thread 9 (Thread 0x410c5940 (LWP 19303)):
#0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
#1 0x000000000043b85b in kvm_run (env=0x284c7c0) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582
#2 0x000000000043cf7f in kvm_cpu_exec (env=0x284c7c0) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233
#3 0x000000000043d73e in kvm_main_loop_cpu (env=0x284c7c0) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419
#4 0x000000000043d8ac in ap_main_loop (_env=0x284c7c0) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466
#5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
#6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
Thread 8 (Thread 0x418c6940 (LWP 19304)):
#0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
#1 0x000000000043b85b in kvm_run (env=0x2868180) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582
#2 0x000000000043cf7f in kvm_cpu_exec (env=0x2868180) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233
#3 0x000000000043d73e in kvm_main_loop_cpu (env=0x2868180) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419
#4 0x000000000043d8ac in ap_main_loop (_env=0x2868180) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466
#5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
#6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
Thread 7 (Thread 0x420c7940 (LWP 19305)):
#0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
#1 0x000000000043b85b in kvm_run (env=0x2877c70) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582
#2 0x000000000043cf7f in kvm_cpu_exec (env=0x2877c70) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233
#3 0x000000000043d73e in kvm_main_loop_cpu (env=0x2877c70) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419
#4 0x000000000043d8ac in ap_main_loop (_env=0x2877c70) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466
#5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
#6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
Thread 6 (Thread 0x428c8940 (LWP 19306)):
#0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
#1 0x000000000043b85b in kvm_run (env=0x2887740) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582
#2 0x000000000043cf7f in kvm_cpu_exec (env=0x2887740) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233
#3 0x000000000043d73e in kvm_main_loop_cpu (env=0x2887740) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419
#4 0x000000000043d8ac in ap_main_loop (_env=0x2887740) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466
#5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
#6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x430c9940 (LWP 19307)):
#0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
#1 0x000000000043b85b in kvm_run (env=0x2897210) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582
#2 0x000000000043cf7f in kvm_cpu_exec (env=0x2897210) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233
#3 0x000000000043d73e in kvm_main_loop_cpu (env=0x2897210) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419
#4 0x000000000043d8ac in ap_main_loop (_env=0x2897210) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466
#5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
#6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x438ca940 (LWP 19308)):
#0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
#1 0x000000000043b85b in kvm_run (env=0x28a6ce0) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582
#2 0x000000000043cf7f in kvm_cpu_exec (env=0x28a6ce0) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233
#3 0x000000000043d73e in kvm_main_loop_cpu (env=0x28a6ce0) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419
#4 0x000000000043d8ac in ap_main_loop (_env=0x28a6ce0) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466
#5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
#6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x440cb940 (LWP 19309)):
#0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
#1 0x000000000043b85b in kvm_run (env=0x28b67b0) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582
#2 0x000000000043cf7f in kvm_cpu_exec (env=0x28b67b0) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233
#3 0x000000000043d73e in kvm_main_loop_cpu (env=0x28b67b0) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419
#4 0x000000000043d8ac in ap_main_loop (_env=0x28b67b0) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466
#5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
#6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x448cc940 (LWP 19310)):
#0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6
#1 0x000000000043b85b in kvm_run (env=0x28c6280) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582
#2 0x000000000043cf7f in kvm_cpu_exec (env=0x28c6280) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233
#3 0x000000000043d73e in kvm_main_loop_cpu (env=0x28c6280) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419
#4 0x000000000043d8ac in ap_main_loop (_env=0x28c6280) at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466
#5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0
#6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7fd303b9c740 (LWP 19302)):
#0 0x0000003dd38ccfc2 in select () from /lib64/libc.so.6
#1 0x00000000004207a5 in main_loop_wait (nonblocking=0) at
/home/takano/qemu-kvm-0.14.0-rc1/vl.c:1376
#2 0x000000000043dd48 in kvm_main_loop () at
/home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1589
#3 0x0000000000420ad2 in main_loop () at
/home/takano/qemu-kvm-0.14.0-rc1/vl.c:1429
#4 0x0000000000424b39 in main (argc=15, argv=0x7fffeb76db58,
envp=0x7fffeb76dbd8) at /home/takano/qemu-kvm-0.14.0-rc1/vl.c:3201
> Then probably also interesting would be a trace of the host kernel
> (trace-cmd record -e kvm, see e.g. [1]).
>
OK. I will try trace as soon as possible.
Regards,
Ryousei
> Thanks,
> Jan
>
> [1] http://lwn.net/Articles/341902/
>
> --
> Siemens AG, Corporate Technology, CT T DE IT 1
> Corporate Competence Center Embedded Linux
>
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
