Re: Ceph + Xen - RBD io hang

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi John,

On 10/28/2013 08:55 PM, John Wilkins wrote:
> John,
> 
> I've never installed anything on Scientific Linux.

SL6 is binary compatible with other EL6 varieties.  This host uses the
same repos as CentOS for Dave Scott's xenserver-core technology preview:
 EPEL6, Dave's XenServer + Ceph technology preview, Dave's
xenserver-core snapshot repo, and the ceph.com Dumpling repo.

> Are you sure that QEMU has RBD support?
>
> I have some wip-doc text, which I'm going to move around shortly. You
> can see the yum install requirements here:
> 
> http://ceph.com/docs/wip-doc-install/install/yum-priorities/
> http://ceph.com/docs/wip-doc-install/install/install-qemu/#rpm-packages

The SL6 qemu-img package does not have RBD support, but I did find those
repos yesterday, and installed the RBD-capable qemu-img package from there.

There are no other obvious qemu-related packages (as in 'rpm -qa | grep
qemu') originating from SL6, and just one from Dave's repo,
'libvirt-daemon-driver-qemu'.

> Make sure you can run QEMU commands on RBD first, since that's a dependency.
> 
> http://ceph.com/docs/wip-doc-install/rbd/qemu-rbd/

Since installing the ceph.com package, the 'qemu-img -f rbd' commands
work.  Before, the SL6 version 'qemu-img -f rbd' commands complained
about rbd being an unknown format.

However, xapi was able to launch VMs both before and after the qemu-img
package update, with no effect on the intermittent hangs.

The 'xe vm-start' hanging problem is not shared by the 'xe vdi-*'
commands (or the updated 'qemu-img -f rbd *' commands), which have no
trouble manipulating rbd-backed vdis at all.

Anything I might've missed in there?  Thanks for the tips!

	John


> On Mon, Oct 28, 2013 at 12:01 PM, John Morris <john@xxxxxxxxxxx> wrote:
>> I'm encountering a problem with RBD-backed Xen.  During a VM boot,
>> pygrub attaches the VM's root VDI to dom0.  This hangs with these
>> messages in the debug log:
>>
>> Oct 27 21:19:59 xen27 kernel:
>>   vbd vbd-51728: 16 Device in use; refusing to close
>> Oct 27 21:19:59 xen27 xenopsd-xenlight:
>>   [xenops] waiting for backend to close
>> Oct 27 21:19:59 xen27 kernel:
>>   qemu-system-i38[2899]: segfault at 7fac042e4000 ip 00007fac0447b129
>>   sp 00007fffe7028630 error 4 in qemu-system-i386[7fac042ed000+309000]
>>
>> More details here:
>>
>> http://pastebin.ca/2472234
>>
>>   - Scientific Linux 6
>>   - 64-bit, Phenom CPU
>>   - Ceph from RPM ceph-0.67.4-0.el6.x86_64
>>   - XenAPI from Dave Scott's technology preview
>>   - two btrfs-backed OSDs with journals on separate drives
>>   - various kernels, incl. 3.4.6 from Dave Scott's repo and 3.11.6
>>     from elrepo.org.
>>
>> This thread (whose Subject: I borrowed) describes what I'm seeing quite
>> well, but no resolution was posted:
>>
>> http://comments.gmane.org/gmane.comp.file-systems.ceph.user/3636
>>
>> In my case, udevd starts a 'blkid' process that holds /dev/xvdb open.
>> Like in James's case, any interaction with the device will hang, and
>> usually can't be killed.  This same problem prevents the machine from
>> completing shutdown.
>>
>> In that thread, Sylvain Munaut says the OSD and kernel driver shouldn't
>> be run in the same host.  I believe my setup does not violate that,
>> since the rbd kernel module is not loaded, and instead the device is
>> attached through the xen_blkfront module instead.
>>
>> Thanks-
>>
>>         John
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> 
> 
> 
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux