----- "Matthias Bolte" <matthias.bolte@xxxxxxxxxxxxxx> wrote:
> 2010/1/10 <pspreadborough@xxxxxxxxxxx>:
> >
> > ----- "Matthias Bolte" <matthias.bolte@xxxxxxxxxxxxxx> wrote:
> >
> >> 2010/1/10 <pspreadborough@xxxxxxxxxxx>:
> >> >
> >> > Hello,
> >> >
> >> > I have been trying to use the domain event C code example but
> >> > unfortunately it segfaults (signal 11) every time I run it:
> >> >
> >> > [root@Spring events-c]# ./event-test
> >> > myEventAddHandleFunc:221: Add handle 5 1 0xf081a0 0x8f727f8
> >> > myEventAddHandleFunc:221: Add handle 7 1 0xf09990 0x8f727f8
> >> > myEventAddHandleFunc:221: Add handle 8 1 0xed7940 0x8f727f8
> >> > myEventAddTimeoutFunc:251: Adding Timeout -1 0xedefa0 0x8f727f8
> >> > myEventAddHandleFunc:221: Add handle 11 1 0xed7940 0x8f727f8
> >> > myEventAddTimeoutFunc:251: Adding Timeout -1 0xedefa0 0x8f727f8
> >> > main:322 :: Registering domain event cbs
> >> > Segmentation fault (core dumped)
> >> >
> >> > Core was generated by
> >> >
> >> `/root/libvirt-0.7.5/examples/domain-events/events-c/.libs/lt-event-test'.
> >> > Program terminated with signal 11, Segmentation fault.
> >> > [New process 21806]
> >> > [New process 21822]
> >> > #0 remoteDomainEventQueueFlush (timer=-1, opaque=0x8f727f8) at
> >> > remote/remote_driver.c:8720
> >> > 8720 tempQueue.count = priv->domainEvents->count;
> >> > (gdb) bt
> >> > #0 remoteDomainEventQueueFlush (timer=-1, opaque=0x8f727f8) at
> >> > remote/remote_driver.c:8720
> >> > #1 0x080490d3 in main (argc=Cannot access memory at address 0x1
> >> > ) at event-test.c:347
> >> >
> >> > The stack looks corrupted so I'm doubtful that this trace if of much
> >> value.
> >> > I have built
> >> > and installed libvirt-0.7.5 and it and it's tools seem to be
> >> operating
> >> > correctly.
> >>
> >> I tried the event-test with libvirt-0.7.5 and QEMU/Xen and both are
> >> working as expected. No segfaults.
> >>
> >> Could you inspect the values of priv and priv->domainEvents in GDB
> >> using 'p priv' to see if they are NULL and try to dereference them in
> >> GDB using 'p *priv' to see if they point to valid memory areas?
> >>
> >> Yes the backtrace looks corrupted. If there is stack/heap corruption
> >> involved valgrind may reveal it, so try to run the event-test in
> >> valgrind and see if that gives any hints.
> >>
> >> You can also try the GIT version of libvirt. There was a invalid free
> >> call (resulting in heap corruption) in the node device code fixed
> >> after the 0.7.5 release. But that should have no effect on the
> >> event-test.
> >>
> >> Matthias
> >
> > Matthias,
> >
> > priv->domainEvents is NULL, here's the gdb output:
>
> This explains the segfault. The next question is, why is it NULL?
>
> > (gdb) p *priv
> > $1 = {lock = {lock = {__data = {__lock = 1, __count = 0, __owner = 21806, __kind = 0, __nusers = 1, {__spins = 0, __list = {
> > __next = 0x0}}}, __size = "\001\000\000\000\000\000\000\000.U\000\000\000\000\000\000\001\000\000\000\000\000\000",
> > __align = 1}}, sock = 150469168, watch = 3, pid = 4, uses_tls = 1982791681, is_secure = 1815048801, session = 0x782f6269,
>
> Seeing uses_tls and is_secure being large numbers and knowing that
> both are used as boolean values in the code and should have values of
> 0 or 1 make me think that priv points to already freed memory here.
>
> > type = 0x2f646e65 <Address 0x2f646e65 out of bounds>, counter = 1684956536, localUses = 1668248365,
> > hostname = 0x74656b <Address 0x74656b out of bounds>, debugLog = 0x0, saslconn = 0x0, saslDecoded = 0x0, saslDecodedLength = 0,
>
> type and hostname are char pointers, but the seem to point into
> nowhere, confirms that this is either freed memory or priv itself got
> overwritten due to heap corruption.
>
> > saslDecodedOffset = 0, saslEncoded = 0x0, saslEncodedLength = 0, saslEncodedOffset = 0,
> > buffer = '\0' <repeats 68 times>, "n\000\000\000\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\001\000\000\000\001\000\000\000\000\000\000\000\001\000\000\000\bQ�\b����\030\034�\b\000\000\000\000�\033�\b\001\000\000\000\002\000\000\000�\033�\b\000\000\000\000\025|�\000\a", '\0' <repeats 11 times>, "X\000�\b", '\0' <repeats 12 times>, "\021\000\000\000\002\000\000\000P��\b\000\000\000\000\021", '\0' <repeats 15 times>, "\021\000\000\0008\036�\b\f\000\000\000\020\000\000\000\021\000\000\000\a\000\000\000\b\000\000\000\t\000\000\000\021\000\000\000\002\000\000\000\230\034�\b\000\000\000\000A\000\000\000\003\000\000\000\001\000\000\000\001\000"..., bufferLength = 0,
> > bufferOffset = 0, callbackList = 0x0, domainEvents = 0x0, eventFlushTimer = 0, domainEventDispatching = 1, wakeupSendFD = 0,
> > wakeupReadFD = 0, waitDispatch = 0x0, streams = 0x0}
> >
> > I'll try a run with valgrind and post the results.
> >
> > Pete
> >
>
> Could you test the Python version of this example found in
> examples/domain-events/events-python/event-test.py? Does this work?
>
> Otherwise lets see if valgrind gives any hints.
>
> Matthias
>
Matthias,
I have built the latest git image and run it through valgrind, here's the valgrind output:
[root@Spring .libs]# valgrind -v ./event-test xen:///
==19881== Memcheck, a memory error detector.
==19881== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al.
==19881== Using LibVEX rev 1658, a library for dynamic binary translation.
==19881== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP.
==19881== Using valgrind-3.2.1, a dynamic binary instrumentation framework.
==19881== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al.
==19881==
--19881-- Command line
--19881-- ./event-test
--19881-- xen:///
--19881-- Startup, with flags:
--19881-- -v
--19881-- Contents of /proc/version:
--19881-- Linux version 2.6.18-164.10.1.el5xen (mockbuild@xxxxxxxxxxxxxxxxxxxx) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Thu Jan 7 21:14:48 EST 2010
--19881-- Arch and hwcaps: X86, x86-sse1-sse2
--19881-- Valgrind library directory: /usr/lib/valgrind
--19881-- Reading syms from /lib/ld-2.5.so (0x163000)
--19881-- Reading syms from /root/libvirt/libvirt/examples/domain-events/events-c/.libs/event-test (0x8048000)
--19881-- Reading syms from /usr/lib/valgrind/x86-linux/memcheck (0x38000000)
--19881-- object doesn't have a dynamic symbol table
--19881-- Reading suppressions file: /usr/lib/valgrind/default.supp
--19881-- REDIR: 0x178790 (index) redirected to 0x38027D0F (vgPlain_x86_linux_REDIR_FOR_index)
--19881-- Reading syms from /usr/lib/valgrind/x86-linux/vgpreload_core.so (0x4001000)
--19881-- Reading syms from /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so (0x4003000)
==19881== WARNING: new redirection conflicts with existing -- ignoring it
--19881-- new: 0x00178790 (index ) R-> 0x04006080 index
--19881-- REDIR: 0x178930 (strlen) redirected to 0x4006250 (strlen)
--19881-- Reading syms from /usr/local/lib/libvirt.so.0.7.5 (0x4009000)
--19881-- Reading syms from /usr/lib/libxml2.so.2.6.26 (0x6254000)
--19881-- object doesn't have a symbol table
--19881-- Reading syms from /usr/lib/libz.so.1.2.3 (0xCD2000)
--19881-- object doesn't have a symbol table
--19881-- Reading syms from /lib/libm-2.5.so (0x413B000)
--19881-- Reading syms from /usr/lib/libgnutls.so.13.0.6 (0x670D000)
--19881-- object doesn't have a symbol table
--19881-- Reading syms from /usr/lib/libgcrypt.so.11.5.2 (0x6590000)
--19881-- object doesn't have a symbol table
--19881-- Reading syms from /usr/lib/libsasl2.so.2.0.22 (0x6536000)
--19881-- object doesn't have a symbol table
--19881-- Reading syms from /usr/lib/libxenstore.so.3.0.0 (0x4162000)
--19881-- object doesn't have a symbol table
--19881-- Reading syms from /lib/libpthread-2.5.so (0x416A000)
--19881-- Reading syms from /lib/libc-2.5.so (0x4183000)
--19881-- Reading syms from /lib/libdl-2.5.so (0xCA7000)
--19881-- Reading syms from /usr/lib/libgpg-error.so.0.3.0 (0xB28000)
--19881-- object doesn't have a symbol table
--19881-- Reading syms from /lib/libresolv-2.5.so (0x26E000)
--19881-- Reading syms from /lib/libcrypt-2.5.so (0x64CE000)
--19881-- REDIR: 0x41F42D0 (memset) redirected to 0x4006540 (memset)
--19881-- REDIR: 0x41F47C0 (memcpy) redirected to 0x4006C20 (memcpy)
--19881-- REDIR: 0x41F3430 (rindex) redirected to 0x4005F60 (rindex)
--19881-- REDIR: 0x41F3090 (strlen) redirected to 0x4006230 (strlen)
--19881-- REDIR: 0x41EED20 (malloc) redirected to 0x400533B (malloc)
--19881-- REDIR: 0x41F2B30 (strcmp) redirected to 0x4006300 (strcmp)
--19881-- REDIR: 0x41EE9E0 (calloc) redirected to 0x4004668 (calloc)
--19881-- REDIR: 0x41F3DD0 (memchr) redirected to 0x4006420 (memchr)
--19881-- REDIR: 0x41EC980 (free) redirected to 0x4004F55 (free)
--19881-- REDIR: 0x41EF190 (realloc) redirected to 0x40053EA (realloc)
==19881== Warning: noted but unhandled ioctl 0x305000 with no size/direction hints
==19881== This could cause spurious value errors to appear.
==19881== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==19881== Warning: noted but unhandled ioctl 0x305000 with no size/direction hints
==19881== This could cause spurious value errors to appear.
==19881== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==19881== Warning: noted but unhandled ioctl 0x305000 with no size/direction hints
==19881== This could cause spurious value errors to appear.
==19881== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
--19881-- REDIR: 0x41F29C0 (index) redirected to 0x4006050 (index)
--19881-- REDIR: 0x41F3280 (strncmp) redirected to 0x4006290 (strncmp)
--19881-- REDIR: 0x41F44C0 (stpcpy) redirected to 0x40068D0 (stpcpy)
--19881-- REDIR: 0x41F3140 (strnlen) redirected to 0x4006200 (strnlen)
--19881-- REDIR: 0x41F3380 (strncpy) redirected to 0x4006DA0 (strncpy)
--19881-- REDIR: 0x41F2BA0 (strcpy) redirected to 0x40069B0 (strcpy)
myEventAddHandleFunc:221: Add handle 5 1 0x40ad4a0 0x42d6188
myEventAddHandleFunc:221: Add handle 7 1 0x40aec90 0x42d6188
Allocating domainEvents:0x43bea68
myEventAddHandleFunc:221: Add handle 8 1 0x407c940 0x42d6188
myEventAddTimeoutFunc:251: Adding Timeout -1 0x4083fe0 0x42d6188
Allocating domainEvents:0x4e8c320
myEventAddHandleFunc:221: Add handle 11 1 0x407c940 0x42d6188
myEventAddTimeoutFunc:251: Adding Timeout -1 0x4083fe0 0x42d6188
main:322 :: Registering domain event cbs
poll
==19881== Invalid read of size 4
==19881== at 0x4084011: remoteDomainEventQueueFlush (remote_driver.c:8722)
==19881== by 0x80490C2: main (event-test.c:341)
==19881== Address 0x0 is not stack'd, malloc'd or (recently) free'd
==19881==
==19881== Process terminating with default action of signal 11 (SIGSEGV)
==19881== Access not within mapped region at address 0x0
==19881== at 0x4084011: remoteDomainEventQueueFlush (remote_driver.c:8722)
==19881== by 0x80490C2: main (event-test.c:341)
==19881==
==19881== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 30 from 1)
==19881==
==19881== 1 errors in context 1 of 1:
==19881== Invalid read of size 4
==19881== at 0x4084011: remoteDomainEventQueueFlush (remote_driver.c:8722)
==19881== by 0x80490C2: main (event-test.c:341)
==19881== Address 0x0 is not stack'd, malloc'd or (recently) free'd
--19881--
--19881-- supp: 30 Fedora-Core-6-hack3-ld25
==19881==
==19881== IN SUMMARY: 1 errors from 1 contexts (suppressed: 30 from 1)
==19881==
==19881== malloc/free: in use at exit: 590,340 bytes in 686 blocks.
==19881== malloc/free: 1,695 allocs, 1,009 frees, 1,700,423 bytes allocated.
==19881==
==19881== searching for pointers to 686 not-freed blocks.
==19881== checked 11,495,892 bytes.
==19881==
==19881== LEAK SUMMARY:
==19881== definitely lost: 0 bytes in 0 blocks.
==19881== possibly lost: 136 bytes in 1 blocks.
==19881== still reachable: 590,204 bytes in 685 blocks.
==19881== suppressed: 0 bytes in 0 blocks.
==19881== Reachable blocks (those to which a pointer was found) are not shown.
==19881== To see them, rerun with: --show-reachable=yes
--19881-- memcheck: sanity checks: 3 cheap, 1 expensive
--19881-- memcheck: auxmaps: 0 auxmap entries (0k, 0M) in use
--19881-- memcheck: auxmaps: 0 searches, 0 comparisons
--19881-- memcheck: SMs: n_issued = 36 (576k, 0M)
--19881-- memcheck: SMs: n_deissued = 0 (0k, 0M)
--19881-- memcheck: SMs: max_noaccess = 65535 (1048560k, 1023M)
--19881-- memcheck: SMs: max_undefined = 0 (0k, 0M)
--19881-- memcheck: SMs: max_defined = 244 (3904k, 3M)
--19881-- memcheck: SMs: max_non_DSM = 36 (576k, 0M)
--19881-- memcheck: max sec V bit nodes: 115 (5k, 0M)
--19881-- memcheck: set_sec_vbits8 calls: 679 (new: 115, updates: 564)
--19881-- memcheck: max shadow mem size: 885k, 0M
--19881-- translate: fast SP updates identified: 7,064 ( 89.2%)
--19881-- translate: generic_known SP updates identified: 543 ( 6.8%)
--19881-- translate: generic_unknown SP updates identified: 304 ( 3.8%)
--19881-- tt/tc: 13,590 tt lookups requiring 14,331 probes
--19881-- tt/tc: 13,590 fast-cache updates, 3 flushes
--19881-- transtab: new 6,511 (139,232 -> 2,275,366; ratio 163:10) [0 scs]
--19881-- transtab: dumped 0 (0 -> ??)
--19881-- transtab: discarded 8 (187 -> ??)
--19881-- scheduler: 374,982 jumps (bb entries).
--19881-- scheduler: 3/10,454 major/minor sched events.
--19881-- sanity: 4 cheap, 1 expensive checks.
--19881-- exectx: 30,011 lists, 566 contexts (avg 0 per list)
--19881-- exectx: 2,732 searches, 2,171 full compares (794 per 1000)
--19881-- exectx: 0 cmp2, 67 cmp4, 0 cmpAll
Killed
[root@Spring .libs]#
I tried the python version and did have some success, but not 100% since I didn't
see events for pausing the VM:
[root@Spring events-python]# python event-test.py xen:///
Using uri:xen:///
myDomainEventCallback2 EVENT: Domain vm-full-1(1) Started 0
myDomainEventCallback1 EVENT: Domain vm-full-1(1) Started 0
I noticed that the the C version looks in one place for the libvirt socket and the
python version in a different place, I don't know if this is significant or not?
I tweaked the /etc/libvirt/libvirtd.conf file to alter the location.
C: /usr/local/var/run/libvirt
python: /var/run/libvirt/
Regards,
Pete
-- Libvir-list mailing list Libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list