Using image type raw actually got kvm to create the VM but then the virt-viewer console shows Booting from Hard Disk Geom Error ----------- We do not even get as far as GRUB. Below is the network stanza from XML. <disk type='network' device='disk'> <source protocol='rbd' name='one/one-21-61-0'> <host name='stkendca01a' port='6789'/> <host name='stkendca04a' port='6789'/> <host name='stkendca02a' port='6789'/> </source> <auth username='libvirt2'> <secret type='ceph' uuid='3bd5a6a5-6b2a-44b2-a07 5-ec9da47ae3f4'/> </auth> <target dev='vda'/> <driver name='qemu' type='raw' cache='none'/> </disk> Any other tweak I might be missing? Thanks Steve Timm By the way--the following is what the raw file in question looked like before I loaded it into CEPH [root at one4dev timm]# file gcso_sl6_giwms.raw gcso_sl6_giwms.raw: x86 boot sector; GRand Unified Bootloader, stage1 version 0x3, boot drive 0x80, 1st sector stage2 0x1307f70, GRUB version 0.94; partition 1: ID=0x83, active, starthead 32, startsector 2048, 6291456 sectors, code offset 0x4 On Thu, 18 Sep 2014, Steven Timm wrote: > thanks Luke, I will try that. > > Steve > > > On Wed, 17 Sep 2014, Luke Jing Yuan wrote: > >> Hi, >> >> From the ones we managed to configure in our lab here. I noticed that >> using image format "raw" instead of "qcow2" worked for us. >> >> Regards, >> Luke >> >> -----Original Message----- >> From: ceph-users [mailto:ceph-users-bounces at lists.ceph.com] On Behalf Of >> Steven Timm >> Sent: Thursday, 18 September, 2014 5:01 AM >> To: ceph-users at lists.ceph.com >> Subject: ceph issue: rbd vs. qemu-kvm >> >> >> I am trying to use Ceph as a data store with OpenNebula 4.6 and have >> followed the instructions in OpenNebula's documentation at >> http://docs.opennebula.org/4.8/administration/storage/ceph_ds.html >> >> and compared them against the "using libvirt with ceph" >> >> http://ceph.com/docs/master/rbd/libvirt/ >> >> We are using the ceph-recompiled qemu-kvm and qemu-img as found at >> >> http://ceph.com/packages/qemu-kvm/ >> >> under Scientific Linux 6.5 which is a Redhat clone. Also a kernel-lt-3.10 >> kernel. >> >> [root at fgtest15 qemu]# kvm -version >> QEMU PC emulator version 0.12.1 (qemu-kvm-0.12.1.2), Copyright (c) >> 2003-2008 Fabrice Bellard >> >> >> From qemu-img >> >> Supported formats: raw cow qcow vdi vmdk cloop dmg bochs vpc vvfat qcow2 >> qed parallels nbd blkdebug host_cdrom host_floppy host_device file rbd >> >> >> ------------------ >> Libvirt is trying to execute the following KVM command: >> >> 2014-09-17 19:50:12.774+0000: starting up >> LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none >> /usr/libexec/qemu-kvm -name one-60 -S -M rhel6.3.0 -enable-kvm -m 4096 >> -smp 2,sockets=2,cores=1,threads=1 -uuid >> 572499bf-07f3-3014-8d6a-dfa1ebb99aa4 -nodefconfig -nodefaults -chardev >> socket,id=charmonitor,path=/var/lib/libvirt/qemu/one-60.monitor,server,nowait >> -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc >> -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive >> file=rbd:one/one-19-60-0:id=libvirt2:key=AQAV5BlU2OV7NBAApurqxG0K8UkZlQVy6hKmkA==:auth_supported=cephx\;none:mon_host=stkendca01a\:6789\;stkendca04a\:6789\;stkendca02a\:6789,if=none,id=drive-virtio-disk0,format=qcow2,cache=none >> -device >> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 >> -drive >> file=/var/lib/one//datastores/102/60/disk.1,if=none,id=drive-virtio-disk1,format=raw,cache=none >> -device >> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 >> -drive >> file=/var/lib/one//datastores/102/60/disk.2,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw >> -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 >> -netdev tap,fd=22,id=hostnet0,vhost=on,vhostfd=23 -device >> virtio-net-pci,netdev=hostnet0,id=net0,mac=54:52:00:02:0b:04,bus=pci.0,addr=0x3 >> -chardev pty,id=charserial0 -device >> isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:60 -k en-us -vga >> cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 >> char device redirected to /dev/pts/3 >> qemu-kvm: -drive >> file=rbd:one/one-19-60-0:id=libvirt2:key=AQAV5BlU2OV7NBAApurqxG0K8UkZlQVy6hKmkA==:auth_supported=cephx\;none:mon_host=stkendca01a\:6789\;stkendca04a\:6789\;stkendca02a\:6789,if=none,id=drive-virtio-disk0,format=qcow2,cache=none: >> could not open disk image >> rbd:one/one-19-60-0:id=libvirt2:key=AQAV5BlU2OV7NBAApurqxG0K8UkZlQVy6hKmkA==:auth_supported=cephx\;none:mon_host=stkendca01a\:6789\;stkendca04a\:6789\;stkendca02a\:6789: >> Invalid argument >> 2014-09-17 19:50:12.980+0000: shutting down >> >> ----------- >> >> just to show that from the command line I can see the rbd pool fine >> >> [root at fgtest15 qemu]# rbd list one >> foo >> one-19 >> one-19-58-0 >> one-19-60-0 >> [root at fgtest15 qemu]# rbd info one/one-19-60-0 >> rbd image 'one-19-60-0': >> size 40960 MB in 10240 objects >> order 22 (4096 kB objects) >> block_name_prefix: rb.0.3c39.238e1f29 >> format: 1 >> >> >> and even mount stuff with rbd map, etc. >> >> It's only inside libvirt that we had the problem. >> >> At first we were getting "permission denied" but then I upped the >> permissions allowed to the libvirt user (client.libvirt2) and then >> we are just getting "invalid argument" >> >> >> client.libvirt2 >> key: AQAV5BlU2OV7NBAApurqxG0K8UkZlQVy6hKmkA== >> caps: [mon] allow r >> caps: [osd] allow *, allow rwx pool=one >> >> ------ >> >> Any idea why kvm doesn't like the argument I am delivering in the file= >> argument? Better--does anyone have a working kvm command out >> of either opennebula or openstack against which I can compare? >> >> Thanks >> >> Steve Timm >> >> >> >> >> >> ------------------------------------------------------------------ >> Steven C. Timm, Ph.D (630) 840-8525 >> timm at fnal.gov http://home.fnal.gov/~timm/ >> Fermilab Scientific Computing Division, Scientific Computing Services >> Quad. >> Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing >> _______________________________________________ >> ceph-users mailing list >> ceph-users at lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> >> ________________________________ >> DISCLAIMER: >> >> >> This e-mail (including any attachments) is for the addressee(s) only and >> may be confidential, especially as regards personal data. If you are not >> the intended recipient, please note that any dealing, review, >> distribution, printing, copying or use of this e-mail is strictly >> prohibited. If you have received this email in error, please notify the >> sender immediately and delete the original message (including any >> attachments). >> >> >> MIMOS Berhad is a research and development institution under the purview >> of the Malaysian Ministry of Science, Technology and Innovation. Opinions, >> conclusions and other information in this e-mail that do not relate to the >> official business of MIMOS Berhad and/or its subsidiaries shall be >> understood as neither given nor endorsed by MIMOS Berhad and/or its >> subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts >> responsibility for the same. All liability arising from or in connection >> with computer viruses and/or corrupted e-mails is excluded to the fullest >> extent permitted by law. >> > > ------------------------------------------------------------------ > Steven C. Timm, Ph.D (630) 840-8525 > timm at fnal.gov http://home.fnal.gov/~timm/ > Fermilab Scientific Computing Division, Scientific Computing Services Quad. > Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > ------------------------------------------------------------------ Steven C. Timm, Ph.D (630) 840-8525 timm at fnal.gov http://home.fnal.gov/~timm/ Fermilab Scientific Computing Division, Scientific Computing Services Quad. Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing
 |