I included you on a thread on users, let us see if he can help you out.
On Mon, Aug 29, 2016 at 4:02 PM, Oleksandr Natalenko <oleksandr@xxxxxxxxxxxxxx> wrote:
More info here.
Massif puts the following warning on volume unmount:
===
valgrind: m_mallocfree.c:304 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 1, hi = 0.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata. If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away. Please try that before reporting this as a bug.
...
Thread 1: status = VgTs_Runnable
==30590== at 0x4C29037: free (in /usr/lib64/valgrind/vgpreload_massif-amd64-linux.so)
==30590== by 0x67CE63B: __libc_freeres (in /usr/lib64/libc-2.17.so)
==30590== by 0x4A246B4: _vgnU_freeres (in /usr/lib64/valgrind/vgpreload_core-amd64-linux.so)
==30590== by 0x66A2E2A: __run_exit_handlers (in /usr/lib64/libc-2.17.so)
==30590== by 0x66A2EB4: exit (in /usr/lib64/libc-2.17.so)
==30590== by 0x1117E9: cleanup_and_exit (glusterfsd.c:1308)
==30590== by 0x669F66F: ??? (in /usr/lib64/libc-2.17.so)
==30590== by 0x606EEF4: pthread_join (in /usr/lib64/libpthread-2.17.so)
==30590== by 0x4EC2687: event_dispatch_epoll (event-epoll.c:762)
==30590== by 0x10E876: main (glusterfsd.c:2370)
...
===
I rechecked mount/ls/unmount with memcheck tool as suggested and got the following:
===
...
==30315== Thread 8:
==30315== Syscall param writev(vector[...]) points to uninitialised byte(s)
==30315== at 0x675FEA0: writev (in /usr/lib64/libc-2.17.so)
==30315== by 0xE664795: send_fuse_iov (fuse-bridge.c:158)
==30315== by 0xE6649B9: send_fuse_data (fuse-bridge.c:197)
==30315== by 0xE666F7A: fuse_attr_cbk (fuse-bridge.c:753)
==30315== by 0xE6671A6: fuse_root_lookup_cbk (fuse-bridge.c:783)
==30315== by 0x14519937: io_stats_lookup_cbk (io-stats.c:1512)
==30315== by 0x14300B3E: mdc_lookup_cbk (md-cache.c:867)
==30315== by 0x13EE9226: qr_lookup_cbk (quick-read.c:446)
==30315== by 0x13CD8B66: ioc_lookup_cbk (io-cache.c:260)
==30315== by 0x1346405D: dht_revalidate_cbk (dht-common.c:985)
==30315== by 0x1320EC60: afr_discover_done (afr-common.c:2316)
==30315== by 0x1320EC60: afr_discover_cbk (afr-common.c:2361)
==30315== by 0x12F9EE91: client3_3_lookup_cbk (client-rpc-fops.c:2981)
==30315== Address 0x170b238c is on thread 8's stack
==30315== in frame #3, created by fuse_attr_cbk (fuse-bridge.c:723)
...
==30315== Warning: invalid file descriptor -1 in syscall close()
==30315== Thread 1:
==30315== Invalid free() / delete / delete[] / realloc()
==30315== at 0x4C2AD17: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30315== by 0x67D663B: __libc_freeres (in /usr/lib64/libc-2.17.so)
==30315== by 0x4A246B4: _vgnU_freeres (in /usr/lib64/valgrind/vgpreload_core-amd64-linux.so)
==30315== by 0x66AAE2A: __run_exit_handlers (in /usr/lib64/libc-2.17.so)
==30315== by 0x66AAEB4: exit (in /usr/lib64/libc-2.17.so)
==30315== by 0x1117E9: cleanup_and_exit (glusterfsd.c:1308)
==30315== by 0x66A766F: ??? (in /usr/lib64/libc-2.17.so)
==30315== by 0x6076EF4: pthread_join (in /usr/lib64/libpthread-2.17.so)
==30315== by 0x4ECA687: event_dispatch_epoll (event-epoll.c:762)
==30315== by 0x10E876: main (glusterfsd.c:2370)
==30315== Address 0x6a2d3d0 is 0 bytes inside data symbol "noai6ai_cached"
===
It seems Massif crashes (?) because of invalid memory access in glusterfs process cleanup stage.
Pranith? Nithya?
29.08.2016 13:14, Oleksandr Natalenko wrote:
===
valgrind --tool=massif --trace-children=yes /usr/sbin/glusterfs -N
--volfile-server=server.example.com --volfile-id=test
/mnt/net/glusterfs/test
===
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
--
Pranith
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
