2013/7/26 Anand Avati <anand.avati@xxxxxxxxx>: > Can you please post the backtrace and logs from the crash? # gdb ... (gdb) file /usr/local/bin/qemu-system-x86_64 Reading symbols from /usr/local/bin/qemu-system-x86_64...done. (gdb) run -uuid "e06dc280-d74a-0130-49e3-003018a4d17c" -name "deb-on-gluster" -m "2048" -vnc ":1" -k "it" -pidfile "/var/run/onboard/qemu-e06dc280.pid" -monitor unix:"/var/run/onboard/qemu-e06dc280.sock",server,nowait -smp 2 -device piix3-usb-uhci,id=piix3-uhci -device usb-ehci,id=ehci -device usb-tablet,bus=piix3-uhci.0 -drive serial="QME06DC28000",if="virtio",media="disk",cache="unsafe",file="gluster://localhost/gvtest/QEMU/deb-on-gluster/disk0.qcow2",index=0 -drive serial="QME06DC28001",if="ide",media="cdrom",bus=1,unit=0 -net nic,vlan=0,model=virtio,macaddr=DE:AD:BE:6C:CF:30 -net tap,vlan=0,ifname=vDebOnGluster_0,script=no,downscript=no -boot menu=on,order=dc -cpu host -enable-kvm -runas onboard ... On another terminal: # gluster volume add-brick gvtest replica 2 192.168.232.101:/var/export/gluster/gvtest volume add-brick: success And here the crash: [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7fffe5dad700 (LWP 8611)] [New Thread 0x7fffe55ac700 (LWP 8618)] [New Thread 0x7fffe4548700 (LWP 8621)] [New Thread 0x7fffe3b33700 (LWP 8622)] [New Thread 0x7fffdbdfd700 (LWP 8623)] [New Thread 0x7fffdb5fc700 (LWP 8624)] [New Thread 0x7fffd9dff700 (LWP 8626)] [New Thread 0x7fffc7fff700 (LWP 8627)] [New Thread 0x7fffc63fe700 (LWP 8628)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffe5dad700 (LWP 8611)] glfs_subvol_done (fs=0x5555566cfc20, subvol=subvol@entry=0x0) at glfs-resolve.c:802 802 glfs-resolve.c: No such file or directory. (gdb) (gdb) bt #0 glfs_subvol_done (fs=0x5555566cfc20, subvol=subvol@entry=0x0) at glfs-resolve.c:802 #1 0x00007ffff70aff50 in glfs_pwritev (glfd=0x5555566d79b0, iovec=<optimized out>, iovcnt=<optimized out>, offset=8368975872, flags=0) at glfs-fops.c:761 #2 0x00007ffff70b04e7 in glfs_io_async_task (data=<optimized out>) at glfs-fops.c:584 #3 0x00007ffff0080b22 in synctask_wrap (old_task=<optimized out>) at syncop.c:131 #4 0x00007ffff0541710 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #5 0x0000000000000000 in ?? () Log: /var/log/glusterfs/etc-glusterfs-glusterd.vol.log [2013-07-26 22:27:45.877325] I [glusterd-brick-ops.c:370:__glusterd_handle_add_brick] 0-management: Received add brick req [2013-07-26 22:27:45.877398] I [glusterd-brick-ops.c:417:__glusterd_handle_add_brick] 0-management: repl ica-count is 2 [2013-07-26 22:27:45.877415] I [glusterd-brick-ops.c:193:gd_addbr_validate_replica_count] 0-management: Changing the type of volume gvtest from 'distribute' to 'replica' [2013-07-26 22:27:45.879877] I [glusterd-brick-ops.c:894:glusterd_op_perform_add_bricks] 0-management: r eplica-count is set 2 [2013-07-26 22:27:45.879907] I [glusterd-brick-ops.c:898:glusterd_op_perform_add_bricks] 0-management: t ype is set 2, need to change it [2013-07-26 22:27:45.882314] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:45.882349] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:45.883952] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:45.883978] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:45.884704] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:46.617898] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:46.617945] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:47.618736] E [glusterd-utils.c:3627:glusterd_nodesvc_unlink_socket_file] 0-management: [2013-07-26 22:27:45.877325] I [glusterd-brick-ops.c:370:__glusterd_handle_add_brick] 0-management: Rece ived add brick req [2013-07-26 22:27:45.877398] I [glusterd-brick-ops.c:417:__glusterd_handle_add_brick] 0-management: repl ica-count is 2 [2013-07-26 22:27:45.877415] I [glusterd-brick-ops.c:193:gd_addbr_validate_replica_count] 0-management: Changing the type of volume gvtest from 'distribute' to 'replica' [2013-07-26 22:27:45.879877] I [glusterd-brick-ops.c:894:glusterd_op_perform_add_bricks] 0-management: r eplica-count is set 2 [2013-07-26 22:27:45.879907] I [glusterd-brick-ops.c:898:glusterd_op_perform_add_bricks] 0-management: t ype is set 2, need to change it [2013-07-26 22:27:45.882314] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:45.882349] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:45.883952] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:45.883978] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:45.884704] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:46.617898] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:46.617945] I [glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management: Found brick [2013-07-26 22:27:47.618736] E [glusterd-utils.c:3627:glusterd_nodesvc_unlink_socket_file] 0-management: [2013-07-26 22:27:47.619092] I [glusterd-utils.c:3661:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV3 successfully [2013-07-26 22:27:47.619386] I [glusterd-utils.c:3666:glusterd_nfs_pmap_deregister] 0-: De-registered MOUNTV1 successfully [2013-07-26 22:27:47.619682] I [glusterd-utils.c:3671:glusterd_nfs_pmap_deregister] 0-: De-registered NFSV3 successfully [2013-07-26 22:27:47.620004] I [glusterd-utils.c:3676:glusterd_nfs_pmap_deregister] 0-: De-registered NLM v4 successfully [2013-07-26 22:27:47.620343] I [glusterd-utils.c:3681:glusterd_nfs_pmap_deregister] 0-: De-registered NLM v1 successfully [2013-07-26 22:27:47.622905] I [rpc-clnt.c:962:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2013-07-26 22:27:47.622988] I [socket.c:3480:socket_init] 0-management: SSL support is NOT enabled [2013-07-26 22:27:47.623003] I [socket.c:3495:socket_init] 0-management: using system polling thread [2013-07-26 22:27:48.650339] E [glusterd-utils.c:3627:glusterd_nodesvc_unlink_socket_file] 0-management: Failed to remove /var/run/52b6ed075a07af2e9235e49dd9d214af.socke t error: No such file or directory [2013-07-26 22:27:48.652919] I [rpc-clnt.c:962:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600 [2013-07-26 22:27:48.653002] I [socket.c:3480:socket_init] 0-management: SSL support is NOT enabled [2013-07-26 22:27:48.653017] I [socket.c:3495:socket_init] 0-management: using system polling thread [2013-07-26 22:27:48.653215] W [socket.c:514:__socket_rwv] 0-management: readv failed (No data available) [2013-07-26 22:27:48.653333] I [mem-pool.c:541:mem_pool_destroy] 0-management: size=2236 max=0 total=0 [2013-07-26 22:27:48.653354] I [mem-pool.c:541:mem_pool_destroy] 0-management: size=124 max=0 total=0 [2013-07-26 22:27:48.653398] I [socket.c:2236:socket_event_handler] 0-transport: disconnecting now [2013-07-26 22:27:48.653453] W [socket.c:514:__socket_rwv] 0-management: readv failed (No data available) [2013-07-26 22:27:48.653478] I [mem-pool.c:541:mem_pool_destroy] 0-management: size=2236 max=0 total=0 [2013-07-26 22:27:48.653493] I [mem-pool.c:541:mem_pool_destroy] 0-management: size=124 max=0 total=0 [2013-07-26 22:27:48.653537] I [socket.c:2236:socket_event_handler] 0-transport: disconnecting now [2013-07-26 22:27:48.680240] E [rpcsvc.c:519:rpcsvc_handle_rpc_call] 0-glusterd: Request received from non-privileged port. Failing request [2013-07-26 22:27:52.093567] E [rpcsvc.c:519:rpcsvc_handle_rpc_call] 0-glusterd: Request received from non-privileged port. Failing request [2013-07-26 22:27:55.094469] E [rpcsvc.c:519:rpcsvc_handle_rpc_call] 0-glusterd: Request received from non-privileged port. Failing request -- G.