Re: libvirt hang on domain attemp to create domain

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 20.06.2013 10:44, Michal Privoznik wrote:
On 20.06.2013 09:40, Александр wrote:
В письме от Четверг, 20-июн-2013 09:15:14 Вы написали:
On 19.06.2013 21:24, Александр wrote:
В письме от Среда, 19-июн-2013 09:35:27 пользователь Michal Privoznik
написал:
On 18.06.2013 22:21, Александр wrote:
В письме от Вторник, 18-июн-2013 12:10:41 пользователь Michal Privoznik

написал:
On 18.06.2013 07:14, Александр wrote:
good day, i am trying to create domain via virDomainDefineXML. and my
program hanging, also i failed to caonnect to libvirt daemon with
virt-manager after this, only killing libvirtd via SIGKILL and
restarting
it solveing hang (i mean after restart i able to connect with
virt-manager), i using following domain xml

<?xml version="1.0" encoding="UTF-8"?>
<domain type="kvm">

   <tittle>Automatically created domain with default
   configuration</tittle>
   <name>vm1</name>
   <description>no description</description>
   <os>
<type>hvm</type>
     <boot dev="cdrom"/>
     <boot dev="hd"/>
</os>
   <vcpu>1</vcpu>
   <memory unit="MB">512</memory>
   <cpu model="host-model">
<topology sockets="1" cores="1" threads="1"/> </cpu>
   <clock offset="localtime"/>
   <devices>
<graphics type="vnc" autoport="yes" password="admin"/>
     <channel type="unix">
<source mode="bind" path="/var/run/vm1.sock"/>
       <target type="virtio" name="org.guest_agent.0"/>
</channel> </devices>

</domain>

i have this calls on stack

0  0x00007ffff48c1750  __poll_nocancel
1  0x00007ffff649e5ca  virNetClientIOEventLoop
This is just probably a client waiting for the corresponding libvirtd
to
provide a reply message. Are you able to get libvirtd debug logs so we
can see what's going on?

http://wiki.libvirt.org/page/DebugLogs

BTW, what's the libvirt version you are running?

Michal

2  0x00007ffff649ed96  virNetClientSendInternal
3  0x00007ffff64a0443  virNetClientSendWithReply
4  0x00007ffff64a0daa  virNetClientProgramCall
5  0x00007ffff6478566  callFull.isra.2
6  0x00007ffff6487fa2  remoteDomainDefineXML
7  0x00007ffff6449d2f  virDomainDefineXML
8  0x0000000000426941  virt_domain::create
/home/sss/install/git/work/avirt/vm_pool/vm_pool/libvirt.cpp  295
9  0x000000000040c1a9  main
/home/sss/install/git/work/avirt/vm_pool/vm_pool/main.cpp  40

what i am doing wrong ?



_______________________________________________
libvirt-users mailing list
libvirt-users@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvirt-users
i have libvirt version 1.0.6, debug log in attachment.
Yuck, this is 1.0.6; Well, from the logs it seems like the daemon is
stuck. Can you attach to the libvirtd while it's being stuck and run 't
a a bt' and post the result? I wonder if we have a deadlock somewhere.

Michal
if i correctly understand you, you need debugger backtrace after libvirtd
stuck ?, if so, here is bt from gdb
You understand correctly. However, simple 'bt' doesn't show what's going
on in other threads - libvirtd is multi threaded.

(gdb) bt
#0  0x00007f00c1ae876d in poll () from /lib64/libc.so.6
#1  0x00007f00c203cfcb in virEventPollRunOnce () from
/usr/lib64/libvirt.so.0 #2  0x00007f00c203ba4d in virEventRunDefaultImpl
() from
/usr/lib64/libvirt.so.0
#3  0x00007f00c214a5cd in virNetServerRun () from /usr/lib64/libvirt.so.0
#4  0x00007f00c2aea08a in main ()
So all we see here is the libvirtd event loop. That's why we need 't a a
bt' which is 'thread apply all backtrace' which will run 'bt' over all
threads.

Michal
how can i make it ?

The very same way as you did 'bt'. Just instead of typing 'bt' you'll
type 't a a bt'.

Michal
here is backtrace from all threads:

(gdb) t a a bt

Thread 11 (Thread 0x7fdd8db82700 (LWP 20321)):
#0 0x00007fdd944b990c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007fdd9476515a in virCondWait () from /usr/lib64/libvirt.so.0
#2 0x00007fdd9476582b in virThreadPoolWorker () from /usr/lib64/libvirt.so.0
#3  0x00007fdd94764dfe in virThreadHelper () from /usr/lib64/libvirt.so.0
#4  0x00007fdd944b5d63 in start_thread () from /lib64/libpthread.so.0
#5  0x00007fdd941ed04d in clone () from /lib64/libc.so.6

Thread 10 (Thread 0x7fdd8d381700 (LWP 20322)):
#0 0x00007fdd944b990c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007fdd9476515a in virCondWait () from /usr/lib64/libvirt.so.0
#2 0x00007fdd9476582b in virThreadPoolWorker () from /usr/lib64/libvirt.so.0
#3  0x00007fdd94764dfe in virThreadHelper () from /usr/lib64/libvirt.so.0
#4  0x00007fdd944b5d63 in start_thread () from /lib64/libpthread.so.0
#5  0x00007fdd941ed04d in clone () from /lib64/libc.so.6

Thread 9 (Thread 0x7fdd8cb80700 (LWP 20323)):
#0  0x00007fdd944bc344 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007fdd944b7e56 in _L_lock_511 () from /lib64/libpthread.so.0
#2  0x00007fdd944b7caa in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007fdd94773861 in virDomainObjListSearchName () from /usr/lib64/libvirt.so.0
#4  0x00007fdd9473c8a5 in virHashSearch () from /usr/lib64/libvirt.so.0
#5 0x00007fdd9478cc3d in virDomainObjListAddLocked.isra.43 () from /usr/lib64/libvirt.so.0 #6 0x00007fdd9478cd47 in virDomainObjListAdd () from /usr/lib64/libvirt.so.0 #7 0x00007fdd875a311c in qemuDomainDefineXML () from /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so
#8  0x00007fdd947ead2f in virDomainDefineXML () from /usr/lib64/libvirt.so.0
#9  0x00007fdd95206b4d in remoteDispatchDomainDefineXMLHelper ()
#10 0x00007fdd94848d27 in virNetServerProgramDispatch () from /usr/lib64/libvirt.so.0 #11 0x00007fdd94844068 in virNetServerHandleJob () from /usr/lib64/libvirt.so.0 #12 0x00007fdd94765795 in virThreadPoolWorker () from /usr/lib64/libvirt.so.0
#13 0x00007fdd94764dfe in virThreadHelper () from /usr/lib64/libvirt.so.0
#14 0x00007fdd944b5d63 in start_thread () from /lib64/libpthread.so.0
#15 0x00007fdd941ed04d in clone () from /lib64/libc.so.6

Thread 8 (Thread 0x7fdd8c37f700 (LWP 20324)):
#0 0x00007fdd944b990c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007fdd9476515a in virCondWait () from /usr/lib64/libvirt.so.0
#2 0x00007fdd9476582b in virThreadPoolWorker () from /usr/lib64/libvirt.so.0
#3  0x00007fdd94764dfe in virThreadHelper () from /usr/lib64/libvirt.so.0
#4  0x00007fdd944b5d63 in start_thread () from /lib64/libpthread.so.0
#5  0x00007fdd941ed04d in clone () from /lib64/libc.so.6

Thread 7 (Thread 0x7fdd8bb7e700 (LWP 20325)):
#0 0x00007fdd944b990c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007fdd9476515a in virCondWait () from /usr/lib64/libvirt.so.0
#2 0x00007fdd9476582b in virThreadPoolWorker () from /usr/lib64/libvirt.so.0
#3  0x00007fdd94764dfe in virThreadHelper () from /usr/lib64/libvirt.so.0
#4  0x00007fdd944b5d63 in start_thread () from /lib64/libpthread.so.0
#5  0x00007fdd941ed04d in clone () from /lib64/libc.so.6

Thread 6 (Thread 0x7fdd8b37d700 (LWP 20326)):
#0 0x00007fdd944b990c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007fdd9476515a in virCondWait () from /usr/lib64/libvirt.so.0
#2 0x00007fdd9476584b in virThreadPoolWorker () from /usr/lib64/libvirt.so.0
#3  0x00007fdd94764dfe in virThreadHelper () from /usr/lib64/libvirt.so.0
#4  0x00007fdd944b5d63 in start_thread () from /lib64/libpthread.so.0
#5  0x00007fdd941ed04d in clone () from /lib64/libc.so.6

Thread 5 (Thread 0x7fdd8ab7c700 (LWP 20327)):
#0 0x00007fdd944b990c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007fdd9476515a in virCondWait () from /usr/lib64/libvirt.so.0
#2 0x00007fdd9476584b in virThreadPoolWorker () from /usr/lib64/libvirt.so.0
#3  0x00007fdd94764dfe in virThreadHelper () from /usr/lib64/libvirt.so.0
#4  0x00007fdd944b5d63 in start_thread () from /lib64/libpthread.so.0
#5  0x00007fdd941ed04d in clone () from /lib64/libc.so.6

Thread 4 (Thread 0x7fdd8a37b700 (LWP 20328)):
#0 0x00007fdd944b990c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
---Type <return> to continue, or q <return> to quit---
#1  0x00007fdd9476515a in virCondWait () from /usr/lib64/libvirt.so.0
#2 0x00007fdd9476584b in virThreadPoolWorker () from /usr/lib64/libvirt.so.0
#3  0x00007fdd94764dfe in virThreadHelper () from /usr/lib64/libvirt.so.0
#4  0x00007fdd944b5d63 in start_thread () from /lib64/libpthread.so.0
#5  0x00007fdd941ed04d in clone () from /lib64/libc.so.6

Thread 3 (Thread 0x7fdd89b7a700 (LWP 20329)):
#0 0x00007fdd944b990c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007fdd9476515a in virCondWait () from /usr/lib64/libvirt.so.0
#2 0x00007fdd9476584b in virThreadPoolWorker () from /usr/lib64/libvirt.so.0
#3  0x00007fdd94764dfe in virThreadHelper () from /usr/lib64/libvirt.so.0
#4  0x00007fdd944b5d63 in start_thread () from /lib64/libpthread.so.0
#5  0x00007fdd941ed04d in clone () from /lib64/libc.so.6

Thread 2 (Thread 0x7fdd89379700 (LWP 20330)):
#0 0x00007fdd944b990c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007fdd9476515a in virCondWait () from /usr/lib64/libvirt.so.0
#2 0x00007fdd9476584b in virThreadPoolWorker () from /usr/lib64/libvirt.so.0
#3  0x00007fdd94764dfe in virThreadHelper () from /usr/lib64/libvirt.so.0
#4  0x00007fdd944b5d63 in start_thread () from /lib64/libpthread.so.0
#5  0x00007fdd941ed04d in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7fdd95166740 (LWP 20320)):
#0  0x00007fdd941e376d in poll () from /lib64/libc.so.6
#1 0x00007fdd94737fcb in virEventPollRunOnce () from /usr/lib64/libvirt.so.0 #2 0x00007fdd94736a4d in virEventRunDefaultImpl () from /usr/lib64/libvirt.so.0
#3  0x00007fdd948455cd in virNetServerRun () from /usr/lib64/libvirt.so.0
#4  0x00007fdd951e508a in main ()

and also from my program if it important:

0  0x00007ffff48c1750  __poll_nocancel
1  0x00007ffff649e5ca  virNetClientIOEventLoop
2  0x00007ffff649ed96  virNetClientSendInternal
3  0x00007ffff64a0443  virNetClientSendWithReply
4  0x00007ffff64a0daa  virNetClientProgramCall
5  0x00007ffff6478566  callFull.isra.2
6  0x00007ffff6487fa2  remoteDomainDefineXML
7  0x00007ffff6449d2f  virDomainDefineXML
8 0x0000000000426941 virt_domain::create /home/sss/install/git/work/avirt/vm_pool/vm_pool/libvirt.cpp 295 9 0x000000000040c1a9 main /home/sss/install/git/work/avirt/vm_pool/vm_pool/main.cpp 40

_______________________________________________
libvirt-users mailing list
libvirt-users@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvirt-users





[Index of Archives]     [Virt Tools]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Yosemite News]     [KDE Users]

  Powered by Linux