Hi List, i set up a 4 node cluster using corosync, pacemaker and ceph, created a 160gb rbd image file with qemu-img and started up one virtual machine using qemu-kvm. The virtual machine runs fine until i isse crm node standby on the running host. I can see the VM to migrate to another host and start up without issues, but the vm seems to be unable to access the disk so all processes inside the VM are hanging waiting on disk-io. If i migrate the VM back to the host where i started the VM, it runs normally without any problems. It seems that the other host is unable to use the rbd image - is there anything i am missing here? I also tried to use a file mounted on ceph as image, but then the node to which the VM wants to migrate gets kernel-oops when trying to access the image file and locks up :( Any help would be highly appreciated! ceph --version ceph version 0.39-54-g745be30 (commit:745be30f517216474d83b9ada2f355217a984258) virsh --version 0.9.8 qemu-system-x86_64 --version QEMU emulator version 1.0.50, Copyright (c) 2003-2008 Fabrice Bellard rbd --version ceph version 0.39-54-g745be30 (commit:745be30f517216474d83b9ada2f355217a984258) Executing rbd showmapped on node01 id pool image snap device 0 rbd ns1 - /dev/rbd0 Executing rbd showmapped on node02 id pool image snap device 0 rbd ns1 - /dev/rbd0 Executing rbd showmapped on node03 id pool image snap device 0 rbd ns1 - /dev/rbd0 Executing rbd showmapped on node04 id pool image snap device 0 rbd ns1 - /dev/rbd0 cat /etc/ceph/ceph.conf [global] pid file = /var/run/ceph/$name.pid debug ms = 1 auth supported = cephx osd journal = /data/ceph.journal osd_journal_size = 512 # filestore journal writeahead = true # filestore journal parallel = true mds max = 4 [mon] mon data = /data/ceph/mon [mon.0] host = node01 mon addr = xxx.xxx.xxx.4:6789 [mon.1] host = node02 mon addr = xxx.xxx.xxx.5:6789 [mon.2] host = node03 mon addr = xxx.xxx.xxx.6:6789 [mon.3] host = node04 mon addr = xxx.xxx.xxx.7:6789 [mds] keyring = /etc/ceph/keyring.$name # mds dir max commit size 32 [mds.0] host = node01 [mds.1] host = node02 [mds.2] host = node03 [mds.3] host = node04 [osd] sudo = true osd data = /data/ceph/osd keyring = /etc/ceph/keyring.$name [osd.0] host = node01 [osd.1] host = node02 [osd.2] host = node03 [osd.3] host = node04 ns1.xml: <domain type='kvm'> <name>ns1</name> <uuid>350e51e8-2fe5-274f-76c4-58b237bc0fba</uuid> <memory>1048576</memory> <currentMemory>524288</currentMemory> <vcpu>2</vcpu> <os> <type arch='x86_64' machine='pc-0.12'>hvm</type> <boot dev='hd'/> </os> <features> <acpi/> <apic/> <pae/> </features> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/bin/qemu-system-x86_64</emulator> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/ceph/debian-6.0.3-amd64-netinst.iso'/> <target dev='hdc' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='1' unit='0'/> </disk> <disk type='network' device='disk'> <driver name='qemu' type='raw'/> <source protocol='rbd' name='rbd/ns1'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <interface type='bridge'> <mac address='52:54:00:d2:ec:15'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <graphics type='vnc' port='5900' autoport='no'/> <video> <model type='cirrus' vram='9216' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </memballoon> </devices> </domain> -- Mit freundlichen Grüßen, Florian Wiessner Smart Weblications GmbH Martinsberger Str. 1 D-95119 Naila fon.: +49 9282 9638 200 fax.: +49 9282 9638 205 24/7: +49 900 144 000 00 - 0,99 EUR/Min* http://www.smart-weblications.de -- Sitz der Gesellschaft: Naila Geschäftsführer: Florian Wiessner HRB-Nr.: HRB 3840 Amtsgericht Hof *aus dem dt. Festnetz, ggf. abweichende Preise aus dem Mobilfunknetz -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html