Mykola, Jason --
I notice on both clusters the qemu process running is /usr/libexec/qemu-kvm (this is RHEL 7.2) and the euser,fgroup,ruser,uname printed by ps are all user qemu.
(note: I am initiating the VM thru virsh define xyz.xml followed by virsh start, and the user initiating both those virsh commands is root).
On both clusters /etc/ceph/ceph.conf is 644 access.
On both clusters, I get
ldd /usr/libexec/qemu-kvm | grep rbd
librbd.so.1 => /lib64/librbd.so.1 (0x00007fbdf1071000)
When I try
rbd --id admin export <image-spec> - | md5sum
I get the same md5sum values on both clusters.
If I use --id qemu instead, I get
rbd: couldn't connect to the cluster
on both clusters.
If I leave off the --id option entirely, it works on both clusters and I get the same md5sum.
Have not tried setting debug options on the working cluster, will try that.
-- Tom
> -----Original Message-----
> From: Mykola Golub [mailto:mgolub@xxxxxxxxxxxx]
> Sent: Monday, August 29, 2016 3:03 AM
> To: Deneau, Tom <tom.deneau@xxxxxxx>
> Cc: ceph-devel@xxxxxxxxxxxxxxx
> Subject: Re: debugging librbd to a VM
>
> May be qemu does not see you ceph.conf? Or some other environmental
> issue (e.g. with qemu rbd driver). Try
>
> ldd /usr/bin/qemu-system-x86_64 |grep rbd
>
> Have you tried starting it from command line?
>
> You could check if config debug options work as expected setting them
> on your working cluster.
>
> On Fri, Aug 26, 2016 at 09:08:30PM +0000, Deneau, Tom wrote:
> > I have two small clusters configured with the same version of Ceph and
> the same OS version.
> > (different processor hardware)
> >
> > On these clusters I have bootable RBD images that I expect a VM to boot
> from.
> > On one cluster, the VM does not boot from the image. On the other
> cluster, it boots fine.
> > On the failing cluster, a VM can boot fine from the base qcow2 image
> from which the rbd image was created. It's only rbd images that are
> causing problems for the VM.
> >
> > I would like to enable some debug logging that could tell where the two
> clusters are diverging.
> >
> > I tried adding
> > debug rbd = 20
> > log file = /tmp/cbt/ceph/log/$name.log
> >
> > I do see some rbd debug messages to stderr during operations like rbd
> clone, etc. but I don't see any log files created in /tmp/cbt/ceph or
> /var/log/ceph or any messages when librbd is being used by say qemu-img
> convert, or by qemu-kvm.
> >
> > Is there a way to see detailed debug messages for librbd?
> >
> > Or is there a better way to see where things are diverging between these
> two clusters?
> > (note: I tried lttng but could not get it to record anything)
> >
> > -- Tom
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> > the body of a message to majordomo@xxxxxxxxxxxxxxx
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> --
> Mykola Golub
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
