Thank you it works now. The issue was the lack of the auth element. qemu has access to /etc/ceph/ceph.conf. Specifying the host elements in the source element caused a crash: error: Failed to start domain test0 error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/3 server name not found: thinkmate3:6789;thinkmate4:6789 (No such file or directory) unable to parse addrs in 'thinkmate3:6789;thinkmate4:6789' mon/MonClient.cc: In function 'void MonClient::_pick_new_mon()' thread 7fd1de5d67c0 time 2012-08-06 15:40:38.191920 mon/MonClient.cc: 424: FAILED assert(monmap.size() > 0) ceph version 0.48argonaut (commit:c2b20ca74249892c8e5e40c12aa14446a2bf2030) 1: (MonClient::_pick_new_mon()+0x40a) [0x7fd1dd6f4e8a] 2: (MonClient::_reopen_session()+0x187) [0x7fd1dd6f8c47] 3: (MonClient::authenticate(double)+0x1aa) [0x7fd1dd6f9fea] 4: (librados::RadosClient::connect()+0xb6c) [0x7fd1dd63d6ec] 5: (()+0x94393) [0x7fd1de698393] 6: (()+0x801e5) [0x7fd1de6841e5] 7: (()+0x80366) [0x7fd1de684366] 8: (()+0x802d9) [0x7fd1de6842d9] 9: (()+0x80ee7) [0x7fd1de684ee7] 10: (()+0xa3b03) [0x7fd1de6a7b03] 11: (()+0xfec4b) [0x7fd1de702c4b] 12: (()+0x11fbc2 On Monday, August 06, 2012 01:33:34 PM Josh Durgin wrote: > On 08/06/2012 12:12 PM, Jeff Strunk wrote: > > I'm having some trouble creating KVM domains with RBD block devices using > > virsh. I've managed to get virsh to define the domain, but it gives an > > error when trying to start the domain: > > error: Failed to start domain test0 > > error: internal error process exited while connecting to monitor: char > > device redirected to /dev/pts/3 kvm: -drive > > file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw: error > > connecting > This is the relevant error. If you don't have an /etc/ceph/ceph.conf > file, or it (or the keyring file possibly) is unreadable by qemu due to > apparmor/selinux etc, you'll need to specify all the information needed > to connect in the drive xml. This includes the monitor host and the > libvirt secret uuid, i.e.: > > <disk type='network' device='disk'> > <driver name='qemu' type='raw'/> > <source protocol='rbd' name='rbd/test0'> > <host name='thinkmate3' port='6789'/> > <host name='thinkmate4' port='6789'/> > </source> > <auth username='admin'> > <secret type='ceph' uuid='f0836fcb-9c05-5ccf-653a-eb1d0308e247'/> > </auth> > <target dev='vda' bus='virtio'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' > function='0x0'/> > </disk> > > Josh > > > kvm: -drive file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw: > > could not open disk image rbd:rbd/test0: No such file or directory> > >>From libvirt's QEMU log: > > 2012-08-06 18:36:57.788+0000: starting up > > LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin > > QEMU_AUDIO_DRV=none /usr/bin/kvm -name test0 -S -M pc-1.0 -enable-kvm -m > > 1024 -smp 1,sockets=1,cores=1,threads=1 -uuid > > 3e727624-f336-d674-4e0c-234641fc9987 -nodefconfig -nodefaults -chardev > > socket,id=charmonitor,path=/var/lib/libvirt/qemu/test0.monitor,server,now > > ait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc > > -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive > > file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw -device > > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=vi > > rtio-disk0,bootindex=1 -drive > > if=none,id=drive-ide0-1-0,readonly=on,format=raw -device > > ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev > > tap,fd=19,id=hostnet0 -device > > virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:8c:e4:92,bus=pci.0,ad > > dr=0x3 -chardev pty,id=charserial0 -device > > isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -vga cirrus -!> > > device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 > > > > Domain id=5 is tainted: high-privileges > > char device redirected to /dev/pts/2 > > kvm: -drive file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw: > > error connecting kvm: -drive > > file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw: could not > > open disk image rbd:rbd/test0: No such file or directory 2012-08-06 > > 18:36:58.353+0000: shutting down > > > > I can, however, successfully run a KVM virtual machine with an RBD block > > device with the following command: > > kvm -drive file=rbd:rbd/test1,id=drive-virtio-disk0,format=raw -cdrom > > /var/lib/libvirt/images/ubuntu-12.04-server-amd64.iso > > > > kvm will boot with the "-drive > > file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw" options, but > > the disk is not detected unless "if=none" is removed. > > > > Has anyone else figured this out? > > > > Thank you, > > Jeff > > > > Here is my rbd pool definition: > > ######### BEGIN > > <pool type="rbd"> > > <name>rbd</name> > > <source> > > <name>rbd</name> > > <host name='thinkmate3' port='6789'/> > > <host name='thinkmate4' port='6789'/> > > <auth username='admin' type='ceph'> > > <secret uuid='f0836fcb-9c05-5ccf-653a-eb1d0308e247'/> > > </auth> > > </source> > > </pool> > > ######### END > > > > My rbd volumes: > > # virsh vol-list rbd > > Name Path > > ----------------------------------------- > > test0 rbd/test0 > > test1 rbd/test1 > > test2 rbd/test2 > > > > Here is my domain definition: > > ######### BEGIN > > <domain type='kvm'> > > > > <name>test0</name> > > <uuid>3e727624-f336-d674-4e0c-234641fc9987</uuid> > > <memory unit='KiB'>1048576</memory> > > <currentMemory unit='KiB'>1048576</currentMemory> > > <vcpu placement='static'>1</vcpu> > > <os> > > > > <type arch='x86_64' machine='pc-1.0'>hvm</type> > > <boot dev='hd'/> > > > > </os> > > <features> > > > > <acpi/> > > <apic/> > > <pae/> > > > > </features> > > <clock offset='utc'/> > > <on_poweroff>destroy</on_poweroff> > > <on_reboot>restart</on_reboot> > > <on_crash>restart</on_crash> > > <devices> > > > > <emulator>/usr/bin/kvm</emulator> > > <disk type='network' device='disk'> > > > > <driver name='qemu' type='raw'/> > > <source protocol='rbd' name='rbd/test0'/> > > <target dev='vda' bus='virtio'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' > > function='0x0'/> > > > > </disk> > > <disk type='block' device='cdrom'> > > > > <driver name='qemu' type='raw'/> > > <target dev='hdc' bus='ide'/> > > <readonly/> > > <address type='drive' controller='0' bus='1' target='0' unit='0'/> > > > > </disk> > > <controller type='ide' index='0'> > > > > <address type='pci' domain='0x0000' bus='0x00' slot='0x01' > > function='0x1'/> > > > > </controller> > > <controller type='usb' index='0'> > > > > <address type='pci' domain='0x0000' bus='0x00' slot='0x01' > > function='0x2'/> > > > > </controller> > > <interface type='network'> > > > > <mac address='52:54:00:8c:e4:92'/> > > <source network='default'/> > > <model type='virtio'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' > > function='0x0'/> > > > > </interface> > > <serial type='pty'> > > > > <target port='0'/> > > > > </serial> > > <console type='pty'> > > > > <target type='serial' port='0'/> > > > > </console> > > <input type='mouse' bus='ps2'/> > > <graphics type='vnc' port='-1' autoport='yes'/> > > <video> > > > > <model type='cirrus' vram='9216' heads='1'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > > function='0x0'/> > > > > </video> > > <memballoon model='virtio'> > > > > <address type='pci' domain='0x0000' bus='0x00' slot='0x05' > > function='0x0'/> > > > > </memballoon> > > > > </devices> > > > > </domain> > > ######## END _______________________________________________ libvirt-users mailing list libvirt-users@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvirt-users
