On Fri, Jun 08, 2018 at 12:49:07PM +0200, Marc Hartmayer wrote:
> On start up of libvirtd the worker pool of the QEMU driver must be
> initialized before trying to reconnect to all the running QEMU
> instances. Otherwise segmentation faults can occur if there are QEMU
> monitor events emitted.
>
> #0 __GI___pthread_mutex_lock (mutex=mutex@entry=0x40) at ../nptl/pthread_mutex_lock.c:67
> #1 0x000003fffdba9e62 in virMutexLock (m=m@entry=0x40) at ../../src/util/virthread.c:89
> #2 0x000003fffdbab2dc in virThreadPoolSendJob (pool=0x0, priority=priority@entry=0, jobData=0x1000b7210) at ../../src/util/virthreadpool.c:386
> #3 0x000003ffd8343b70 in qemuProcessHandleSerialChanged (mon=<optimized out>, vm=0x3ff70074340, devAlias=<optimized out>, connected=<optimized out>, opaque=0x3ff70013f70) at ../../src/qemu/qemu_process.c:1480
> #4 0x000003ffd836a776 in qemuMonitorEmitSerialChange (mon=mon@entry=0x3ff500013f0, devAlias=devAlias@entry=0x1000b6940 "channel0", connected=<optimized out>) at ../../src/qemu/qemu_monitor.c:1558
> #5 0x000003ffd8378e52 in qemuMonitorJSONHandleSerialChange (mon=0x3ff500013f0, data=0x1000b79a0) at ../../src/qemu/qemu_monitor_json.c:1133
> #6 0x000003ffd8378930 in qemuMonitorJSONIOProcessEvent (mon=<optimized out>, mon@entry=0x3ff500013f0, obj=obj@entry=0x1000b6780) at ../../src/qemu/qemu_monitor_json.c:182
> #7 0x000003ffd837edee in qemuMonitorJSONIOProcessLine (mon=0x3ff500013f0, mon@entry=<error reading variable: value has been optimized out>, line=0x1000b8760 "{\"timestamp\": {\"seconds\": 1528444997, \"microseconds\": 125231}, \"event\": \"VSERPORT_CHANGE\", \"data\": {\"open\": false, \"id\": \"channel0\"}}", msg=0x3ff917f9850, msg@entry=<error reading variable: value has been optimized out>) at ../../src/qemu/qemu_monitor_json.c:211
> #8 0x000003ffd837ef86 in qemuMonitorJSONIOProcess (mon=<optimized out>, mon@entry=0x3ff500013f0, data=0x10008c4c0 "{\"return\": [{\"arch\": \"other\", \"current\": true, \"props\": {\"core-id\": 0}, \"CPU\": 0, \"qom_path\": \"/machine/unattached/device[0]\", \"halted\": false, \"thread_id\": 131501}, {\"arch\": \"other\", \"current\": false"..., len=477, msg=<optimized out>, msg@entry=0x3ff917f9850) at ../../src/qemu/qemu_monitor_json.c:253
> #9 0x000003ffd836757a in qemuMonitorIOProcess (mon=mon@entry=0x3ff500013f0) at ../../src/qemu/qemu_monitor.c:447
> #10 0x000003ffd836863e in qemuMonitorIO (watch=<error reading variable: value has been optimized out>, fd=<optimized out>, events=0, events@entry=<error reading variable: value has been optimized out>, opaque=0x3ff500013f0, opaque@entry=<error reading variable: value has been optimized out>) at ../../src/qemu/qemu_monitor.c:702
> #11 0x000003fffdb4033a in virEventPollDispatchHandles (nfds=<optimized out>, fds=0x1000b7160) at ../../src/util/vireventpoll.c:508
> #12 0x000003fffdb4055e in virEventPollRunOnce () at ../../src/util/vireventpoll.c:657
> #13 0x000003fffdb3e782 in virEventRunDefaultImpl () at ../../src/util/virevent.c:327
> #14 0x000003fffdc89400 in virNetDaemonRun (dmn=0x1000b2510) at ../../src/rpc/virnetdaemon.c:850
> #15 0x000000010002a816 in main (argc=<optimized out>, argv=<optimized out>) at ../../src/remote/remote_daemon.c:1460
>
> Signed-off-by: Marc Hartmayer <mhartmay@xxxxxxxxxxxxx>
> Reviewed-by: Bjoern Walk <bwalk@xxxxxxxxxxxxx>
> Reviewed-by: Boris Fiuczynski <fiuczy@xxxxxxxxxxxxx>
> ---
> src/qemu/qemu_driver.c | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
> index 971cf3c2750a..28769878cc58 100644
> --- a/src/qemu/qemu_driver.c
> +++ b/src/qemu/qemu_driver.c
> @@ -932,12 +932,15 @@ qemuStateInitialize(bool privileged,
> qemuDomainManagedSaveLoad,
> qemu_driver);
>
> - qemuProcessReconnectAll(qemu_driver);
> -
> + /* must be initialized before trying to reconnect to all the
> + * running domains since there might occur some QEMU monitor
> + * events that will be dispatched to the worker pool */
> qemu_driver->workerPool = virThreadPoolNew(0, 1, 0, qemuProcessEventHandler, qemu_driver);
> if (!qemu_driver->workerPool)
> goto error;
>
> + qemuProcessReconnectAll(qemu_driver);
> +
> virNWFilterRegisterCallbackDriver(&qemuCallbackDriver);
> return 0;
Wow, how could we have missed this for so long...
Reviewed-by: Erik Skultety <eskultet@xxxxxxxxxx>
PS: I'll strip some bits of the backtrace so it looks more compact in the
history before pushing.
--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list
