Hello everyone, It's a terror thing that meat data lose... The env like this : pveversion pve-manager/3.1-3/dc0e9b0e (running kernel: 2.6.32-23-pve) pvesm status file-storage-c2 glusterfs 1 787924096 35682688 716371200 5.24% vm-images-c2 glusterfs 1 364029184 17849088 331842688 5.60% vm-images-c9 glusterfs 1 1766111360 42094336 1634303360 3.01% gluster --version glusterfs 3.4.0 built on Jul 19 2013 03:56:40 gluster volume info Volume Name: vm-images-c9 Type: Striped-Replicate Volume ID: 6a30d329-1ea2-42de-8c7d-b13964941a64 Status: Started Number of Bricks: 1 x 14 x 2 = 28 Transport-type: tcp Bricks: Brick1: agent21.kisops.org:/d2/vm-images-c9 Brick2: agent22.kisops.org:/d2/vm-images-c9 Brick3: agent23.kisops.org:/d1/vm-images-c9 Brick4: agent24.kisops.org:/d1/vm-images-c9 Brick5: agent25.kisops.org:/d1/vm-images-c9 Brick6: agent26.kisops.org:/d1/vm-images-c9 Brick7: agent27.kisops.org:/d1/vm-images-c9 Brick8: agent28.kisops.org:/d1/vm-images-c9 Brick9: agent29.kisops.org:/d0/vm-images-c9 Brick10: agent30.kisops.org:/d1/vm-images-c9 Brick11: agent31.kisops.org:/d1/vm-images-c9 Brick12: agent32.kisops.org:/d1/vm-images-c9 Brick13: agent21.kisops.org:/d3/vm-images-c9 Brick14: agent22.kisops.org:/d3/vm-images-c9 Brick15: agent23.kisops.org:/d2/vm-images-c9 Brick16: agent24.kisops.org:/d2/vm-images-c9 Brick17: agent25.kisops.org:/d2/vm-images-c9 Brick18: agent26.kisops.org:/d2/vm-images-c9 Brick19: agent27.kisops.org:/d2/vm-images-c9 Brick20: agent30.kisops.org:/d2/vm-images-c9 Brick21: agent31.kisops.org:/d2/vm-images-c9 Brick22: agent23.kisops.org:/d3/vm-images-c9 Brick23: agent24.kisops.org:/d3/vm-images-c9 Brick24: agent25.kisops.org:/d3/vm-images-c9 Brick25: agent26.kisops.org:/d3/vm-images-c9 Brick26: agent27.kisops.org:/d3/vm-images-c9 Brick27: agent30.kisops.org:/d3/vm-images-c9 Brick28: agent31.kisops.org:/d3/vm-images-c9 /usr/bin/kvm -id 107 -chardev socket,id=qmp,path=/var/run/qemu-server/107.qmp,server,nowait -mon chardev=qmp,mode=control -vnc unix:/var/run/qemu-server/107.vnc,x509,password -pidfile /var/run/qemu-server/107.pid -daemonize -name ngrinder-Console -smp sockets=1,cores=3 -nodefaults -boot menu=on -vga cirrus -cpu kvm64,+x2apic,+sep -k en-us -m 6144 -device piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2 -device usb-tablet,id=tablet,bus=uhci.0,port=1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -drive if=none,id=drive-ide2,media=cdrom,aio=native -device ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200 -drive file=gluster:// agent21.kisops.org/vm-images-c9/images/107/vm-107-disk-1.qcow2,if=none,id=drive-virtio0,format=qcow2,cache=writeback,aio=native-device virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100 -netdev type=tap,id=net0,ifname=tap107i0,script=/var/lib/qemu-server/pve-bridge,vhost=on -device virtio-net-pci,mac=0E:17:15:37:D4:83,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300 cat /etc/pve/nodes/agent21/qemu-server/107.conf #Ngrinder-Controller balloon: 6 bootdisk: virtio0 cores: 3 ide2: none,media=cdrom memory: 6144 name: ngrinder-Console net0: virtio=0E:17:15:37:D4:83,bridge=vmbr0 ostype: l26 sockets: 1 virtio0: vm-images-c9:107/vm-107-disk-1.qcow2,format=qcow2,cache=writeback,backup=no,size=88G When on node 'agent24.kisops.org' shutdown by errors ,then the VMs errors : [image: ???? 1] reboot after errors : [image: ???? 2] some VMs could be OK after exec `fsck /vda1`,others couldn't up. kvm: -drive file=gluster:// agent21.kisops.org/vm-images-c9/images/109/vm-109-disk-2.qcow2,if=none,id=drive-virtio0,format=qcow2,cache=writeback,aio=native: could not open disk image gluster:// agent21.kisops.org/vm-images-c9/images/109/vm-109-disk-2.qcow2: Input/output error TASK ERROR: start failed: command '/usr/bin/kvm -id 109 -chardev 'socket,id=qmp,path=/var/run/qemu-server/109.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/109.vnc,x509,password -pidfile /var/run/qemu-server/109.pid -daemonize -name Ngrinder-Cient-26 -smp 'sockets=1,cores=2' -nodefaults -boot 'menu=on' -vga cirrus -cpu kvm64,+x2apic,+sep -k en-us -m 4096 -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -drive 'if=none,id=drive-ide2,media=cdrom,aio=native' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive 'file=gluster:// 10.20.197.21/vm-images-c9/images/109/vm-109-disk-2.qcow2,if=none,id=drive-virtio0,format=qcow2,cache=writeback,aio=native' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap109i0,script=/var/lib/qemu-server/pve-bridge,vhost=on' -device 'virtio-net-pci,mac=0A:0E:47:39:76:23,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300'' failed: exit code 1 I see there are io error on GlusterFS even 'agent24.kisops.org' is up : du -sh /mnt/pve/vm-images-c9/images/109/* 1.9G /mnt/pve/vm-images-c9/images/109/vm-109-disk-1.raw du: cannot access `/mnt/pve/vm-images-c9/images/109/vm-109-disk-2.qcow2': Input/output error If I shut down 'agent24.kisops.org' , the GlusterFS dir will be Empty. What's wrong with me ? Help ! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131008/90b8833a/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 57511 bytes Desc: not available URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131008/90b8833a/attachment.png> -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 64720 bytes Desc: not available URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20131008/90b8833a/attachment-0001.png>
