Re: iSCSI ceph rbd

[Nick Fisk <nick@xxxxxxxxxx>]

To answer the 1^st question, yes you can mount the RBD’s on the existing nodes, however there have been reported problems with RBD clients on the same server as the OSD’s. From memory these have been mainly crashes and hangs. Whether or not you will come across these problems is something you will have to test.

 

However potentially more of a concern would be if you are using pacemaker. If you are configuring pacemaker correctly you will need to set up stonith, which brings the possibility that it might start forcibly restarting your OSD’s or monitors as part of the process.

 

Regarding iSCSI multipath, currently you can configure ALUA active/passive but not active/active. Mike Christie from Redhat is currently working towards getting active/active working with Ceph but it is not currently ready for use. If you try and just use active/active in its current state you will likely end up with corruption.

 

If you want to use ALUA active/passive you will still need something like pacemaker to manage the ALUA states for each node. There is also outstanding problems with LIO+Ceph causing kernel panics and hangs. If you are intending to use this LIO+Ceph with ESXi there is also another problem which seems to frequently happen, where ESXI+LIO get stuck in a loop and the LUNs go offline. I’m currently using tgt with the RBD backend as the best solution for exporting RBD’s as iSCSI.

 

Generally, if you are happy with NFS sync performance and don’t require iSCSI, I would stick with it for the mean time.

 

Nick

 

From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Vasiliy Angapov
Sent: 22 May 2015 13:06
To: Gerson Ariel
Cc: ceph-users@xxxxxxxxxxxxxx
Subject: Re: iSCSI ceph rbd

 

Hi, Ariel, gentlemen,

 

I have the same question but with regard to multipath. Is it possible to just export iSCSI target on each Ceph node and use a multipath on client side?

Can it possibly lead to data inconsistency?

 

Regards, Vasily.

 

 

On Fri, May 22, 2015 at 12:59 PM, Gerson Ariel <ariel@xxxxxxxxxxxxxx> wrote:

I apologize beforehand for not using more descriptive subject for my question.

 

 

On Fri, May 22, 2015 at 4:55 PM, Gerson Ariel <ariel@xxxxxxxxxxxxxx> wrote:

Our hardware is like this, three identical servers with 8 osd disks, 1 ssd disk
as journal, 1 for os, 32GB of ECC RAM, 4 GiB copper ethernet. We deploy this
cluster since February 2015 and most of the the system load is not too great,
lots of idle time.

Right now we have a node that mounts rbd blocks and export them as nfs. It
works quite well but at a cost of one extra node as bridge between storage
client (vms) and storage provider  cluster (ceph osd and mon).

What I want to know is, is there any reason why I shouldn't mount rbd disks on
one of the server, the ones that also runs OSD and MON daemons, and export them
as nfs or iSCSI?  Assuming that I already done my homework to make my setup
highly available using pacemaker (eg. floating IP, iSCSI/NFS resource), isn't
something like this would be better as it is more reliable? ie. I remove the
middle-man node(s) so I only have to make sure about those ceph nodes and vm-hosts.

Thank you

 

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://xo4t.mj.am/link/xo4t/iju6il9/1/0HY-NXM3uNzVm4wXdIwWuQ/aHR0cDovL2xpc3RzLmNlcGguY29tL2xpc3RpbmZvLmNnaS9jZXBoLXVzZXJzLWNlcGguY29t

 

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com