On Sat, Mar 10, 2018 at 10:11 AM, shadow_lin <shadow_lin@xxxxxxx> wrote: > Hi Jason, > >>As discussed in this thread, for active/passive, upon initiator >>failover, we used the RBD exclusive-lock feature to blacklist the old >>"active" iSCSI target gateway so that it cannot talk w/ the Ceph >>cluster before new writes are accepted on the new target gateway. > > I can get during the new active target gateway was talking to rbd the old > active target gateway cannot write because of the RBD exclusive-lock > But after the new target gateway done the writes,if the old target gateway > had some blocked io during the failover,cant it then get the lock and > overwrite the new writes? Negative -- it's blacklisted so it cannot talk to the cluster. > PS: > Petasan say they can do active/active iscsi with patched suse kernel. I'll let them comment on these corner cases. > 2018-03-10 > ________________________________ > shadowlin > > ________________________________ > > 发件人：Jason Dillaman <jdillama@xxxxxxxxxx> > 发送时间：2018-03-10 21:40 > 主题：Re: iSCSI Multipath (Load Balancing) vs RBD Exclusive Lock > 收件人："shadow_lin"<shadow_lin@xxxxxxx> > 抄送："Mike Christie"<mchristi@xxxxxxxxxx>,"Lazuardi > Nasution"<mrxlazuardin@xxxxxxxxx>,"Ceph Users"<ceph-users@xxxxxxxxxxxxxx> > > On Sat, Mar 10, 2018 at 7:42 AM, shadow_lin <shadow_lin@xxxxxxx> wrote: >> Hi Mike, >> So for now only suse kernel with target_rbd_core and tcmu-runner can run >> active/passive multipath safely? > > Negative, the LIO / tcmu-runner implementation documented here [1] is > safe for active/passive. > >> I am a newbie to iscsi. I think the stuck io get excuted cause overwrite >> problem can happen with both active/active and active/passive. >> What makes the active/passive safer than active/active? > > As discussed in this thread, for active/passive, upon initiator > failover, we used the RBD exclusive-lock feature to blacklist the old > "active" iSCSI target gateway so that it cannot talk w/ the Ceph > cluster before new writes are accepted on the new target gateway. > >> What mechanism should be implement to avoid the problem with >> active/passive >> and active/active multipath? > > Active/passive it solved as discussed above. For active/active, we > don't have a solution that is known safe under all failure conditions. > If LIO supported MCS (multiple connections per session) instead of > just MPIO (multipath IO), the initiator would provide enough context > to the target to detect IOs from a failover situation. > >> 2018-03-10 >> ________________________________ >> shadowlin >> >> ________________________________ >> >> 发件人：Mike Christie <mchristi@xxxxxxxxxx> >> 发送时间：2018-03-09 00:54 >> 主题：Re: iSCSI Multipath (Load Balancing) vs RBD Exclusive Lock >> 收件人："shadow_lin"<shadow_lin@xxxxxxx>,"Lazuardi >> Nasution"<mrxlazuardin@xxxxxxxxx>,"Ceph Users"<ceph-users@xxxxxxxxxxxxxx> >> 抄送： >> >> On 03/07/2018 09:24 AM, shadow_lin wrote: >>> Hi Christie, >>> Is it safe to use active/passive multipath with krbd with exclusive lock >>> for lio/tgt/scst/tcmu? >> >> No. We tried to use lio and krbd initially, but there is a issue where >> IO might get stuck in the target/block layer and get executed after new >> IO. So for lio, tgt and tcmu it is not safe as is right now. We could >> add some code tcmu's file_example handler which can be used with krbd so >> it works like the rbd one. >> >> I do know enough about SCST right now. >> >> >>> Is it safe to use active/active multipath If use suse kernel with >>> target_core_rbd? >>> Thanks. >>> >>> 2018-03-07 >>> ------------------------------------------------------------------------ >>> shadowlin >>> >>> ------------------------------------------------------------------------ >>> >>> *发件人：*Mike Christie <mchristi@xxxxxxxxxx> >>> *发送时间：*2018-03-07 03:51 >>> *主题：*Re: iSCSI Multipath (Load Balancing) vs RBD >>> Exclusive Lock >>> *收件人：*"Lazuardi Nasution"<mrxlazuardin@xxxxxxxxx>,"Ceph >>> Users"<ceph-users@xxxxxxxxxxxxxx> >>> *抄送：* >>> >>> On 03/06/2018 01:17 PM, Lazuardi Nasution wrote: >>> > Hi, >>> > >>> > I want to do load balanced multipathing (multiple iSCSI >>> gateway/exporter >>> > nodes) of iSCSI backed with RBD images. Should I disable exclusive >>> lock >>> > feature? What if I don't disable that feature? I'm using TGT >>> (manual >>> > way) since I get so many CPU stuck error messages when I was using >>> LIO. >>> > >>> >>> You are using LIO/TGT with krbd right? >>> >>> You cannot or shouldn't do active/active multipathing. If you have >>> the >>> lock enabled then it bounces between paths for each IO and will be >>> slow. >>> If you do not have it enabled then you can end up with stale IO >>> overwriting current data. >>> >>> >>> >>> >> >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > > [1] http://docs.ceph.com/docs/master/rbd/iscsi-overview/ > > -- > Jason -- Jason _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |