Bjoern Walk <bwalk@xxxxxxxxxxxxx> [2018-09-12, 01:17PM +0200]: > Michal Privoznik <mprivozn@xxxxxxxxxx> [2018-09-12, 11:32AM +0200]: > > On 09/12/2018 07:19 AM, Bjoern Walk wrote: > > > Michal Privoznik <mprivozn@xxxxxxxxxx> [2018-09-10, 11:36AM +0200]: > > >> Technically, this is v4 of: > > >> > > >> https://www.redhat.com/archives/libvir-list/2018-August/msg01627.html > > >> > > >> However, this is implementing different approach than any of the > > >> previous versions. > > >> > > >> One of the problems with previous version was that it was too > > >> complicated. The main reason for that was that we could not close the > > >> connection whilst there was a file locked. So we had to invent a > > >> mechanism that would prevent that (on the client side). > > >> > > >> These patches implement different approach. They rely on secdriver's > > >> transactions which bring all the paths we want to label into one place > > >> so that they can be relabelled within different namespace. > > >> I'm extending this idea so that transactions run all the time > > >> (regardless of domain namespacing) and only at the very last moment is > > >> decided which namespace would the relabeling run in. > > >> > > >> Metadata locking is then as easy as putting lock/unlock calls around one > > >> function. > > >> > > >> You can find the patches at my github too: > > >> > > >> https://github.com/zippy2/libvirt/tree/disk_metadata_lock_v4_alt > > > > > > Hey Michal, > > > > > > is was running a quick test with this patch series with two domains > > > sharing a disk image without <shareable/> and SELinux enabled. When > > > starting the second domain, the whole libvirtd daemon hangs for almost a > > > minute until giving the error that the image is locked. I haven't > > > debugged it yet to figure out what happens. > > > > Is this on two different hosts or one? > > On the same host. > > > I'm unable to reproduce, so can you please attach debugger and share 't > > a a bt' output with me? > > (gdb) thread apply all bt > > Thread 17 (Thread 0x3ff48dff910 (LWP 193353)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff6678c5a8 in udevEventHandleThread (opaque=<optimized out>) at node_device/node_device_udev.c:1603 > #3 0x000003ff8c6bad16 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 16 (Thread 0x3ff4acfe910 (LWP 193312)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbbe0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 15 (Thread 0x3ff4b4ff910 (LWP 193311)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbbe0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 14 (Thread 0x3ff64efd910 (LWP 193310)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbbe0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 13 (Thread 0x3ff656fe910 (LWP 193309)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbbe0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 12 (Thread 0x3ff65eff910 (LWP 193308)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbbe0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 11 (Thread 0x3ff67fff910 (LWP 193307)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbba0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 10 (Thread 0x3ff84bf7910 (LWP 193306)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbba0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 9 (Thread 0x3ff853f8910 (LWP 193305)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbba0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 8 (Thread 0x3ff85bf9910 (LWP 193304)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbba0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 7 (Thread 0x3ff863fa910 (LWP 193303)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbba0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 6 (Thread 0x3ff86bfb910 (LWP 193302)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbbe0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 5 (Thread 0x3ff873fc910 (LWP 193301)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbbe0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 4 (Thread 0x3ff87bfd910 (LWP 193300)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbbe0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 3 (Thread 0x3ff883fe910 (LWP 193299)): > #0 0x000003ff8b7113d4 in read () from /lib64/libpthread.so.0 > #1 0x000003ff8c65a648 in saferead () from /lib64/libvirt.so.0 > #2 0x000003ff8c65a718 in saferead_lim () from /lib64/libvirt.so.0 > #3 0x000003ff8c65abe4 in virFileReadHeaderFD () from /lib64/libvirt.so.0 > #4 0x000003ff8c69cb5a in virProcessRunInMountNamespace () from /lib64/libvirt.so.0 > #5 0x000003ff8c767bd6 in virSecuritySELinuxTransactionCommit () from /lib64/libvirt.so.0 > #6 0x000003ff8c75fb1c in virSecurityManagerTransactionCommit () from /lib64/libvirt.so.0 > #7 0x000003ff8c75bb70 in virSecurityStackTransactionCommit () from /lib64/libvirt.so.0 > #8 0x000003ff8c75fb1c in virSecurityManagerTransactionCommit () from /lib64/libvirt.so.0 > #9 0x000003ff6612c492 in qemuSecuritySetAllLabel () from /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so > #10 0x000003ff660bcfd6 in qemuProcessLaunch () from /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so > #11 0x000003ff660c0916 in qemuProcessStart () from /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so > #12 0x000003ff66124a00 in qemuDomainObjStart.constprop.52 () from /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so > #13 0x000003ff66125030 in qemuDomainCreateWithFlags () from /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so > #14 0x000003ff8c87bfca in virDomainCreate () from /lib64/libvirt.so.0 > #15 0x000002aa1f2d516c in remoteDispatchDomainCreate (server=<optimized out>, msg=0x2aa4c488b00, args=<optimized out>, rerr=0x3ff883fdae8, client=<optimized out>) at remote/remote_daemon_dispatch_stubs.h:4434 > #16 remoteDispatchDomainCreateHelper (server=<optimized out>, client=<optimized out>, msg=0x2aa4c488b00, rerr=0x3ff883fdae8, args=<optimized out>, ret=0x3ff78004c80) at remote/remote_daemon_dispatch_stubs.h:4410 > #17 0x000003ff8c78bc80 in virNetServerProgramDispatch () from /lib64/libvirt.so.0 > #18 0x000003ff8c792aba in virNetServerHandleJob () from /lib64/libvirt.so.0 > #19 0x000003ff8c6bbb16 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #20 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #21 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #22 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 2 (Thread 0x3ff88bff910 (LWP 193298)): > #0 0x000003ff8b70d7b8 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 > #1 0x000003ff8c6baf92 in virCondWait () from /lib64/libvirt.so.0 > #2 0x000003ff8c6bbbe0 in virThreadPoolWorker () from /lib64/libvirt.so.0 > #3 0x000003ff8c6bace6 in virThreadHelper () from /lib64/libvirt.so.0 > #4 0x000003ff8b7079a8 in start_thread () from /lib64/libpthread.so.0 > #5 0x000003ff8b5f9706 in thread_start () from /lib64/libc.so.6 > > Thread 1 (Thread 0x3ff8cbd7970 (LWP 193297)): > #0 0x000003ff8b5ee502 in poll () from /lib64/libc.so.6 > #1 0x000003ff8c65629c in virEventPollRunOnce () from /lib64/libvirt.so.0 > #2 0x000003ff8c654caa in virEventRunDefaultImpl () from /lib64/libvirt.so.0 > #3 0x000003ff8c7921de in virNetDaemonRun () from /lib64/libvirt.so.0 > #4 0x000002aa1f2a4c12 in main (argc=<optimized out>, argv=<optimized out>) at remote/remote_daemon.c:1461 > > > When I try to reproduce I always get one domain running and the other > > failing to start because qemu is unable to do its locking. When I put > > <shareable/> in both, they start successfully. > > Yes, I get the same (expected) behaviour, just with the timeout. > > > TBH, I don't run SELinux enabled host so I'm testing DAC only, but the > > changes to the code are merely the same. So I'm wondering if this is > > really an issue in my SELinux impl or somewhere else. > > > > BTW: The one minute timeout comes from patch 16/23: > > > > #define LOCK_ACQUIRE_TIMEOUT 60 > > > > Michal > > It's entirely possible that my setup is broken, this was just a quick > test. If you can't reproduce it or make sense out of the backtrace then > it's fine, I will try to put some time to debug it next week. I was just > interested if this would fix a long-standing bug where this scenario > would crash the first domain because the label changed. > > Bjoern > Still seeing the same timeout. Is this expected behaviour? -- IBM Systems Linux on Z & Virtualization Development ------------------------------------------------------------------------ IBM Deutschland Research & Development GmbH Schönaicher Str. 220, 71032 Böblingen Phone: +49 7031 16 1819 ------------------------------------------------------------------------ Vorsitzende des Aufsichtsrats: Martina Koederitz Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294
Attachment:
signature.asc
Description: PGP signature
-- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list
