On Sun, Mar 11, 2018 at 9:54 AM, shadow_lin <shadow_lin@xxxxxxx> wrote: > Hi Jason, > How the old target gateway is blacklisted? When the newly active target gateway breaks the lock of the old target gateway, that process will blacklist the old client [1]. > Is it a feature of the target > gateway(which can support active/passive multipath) should provide or is it > only by rbd excusive lock? > I think excusive lock only let one client can write to rbd at the same > time,but another client can obtain the lock later when the lock is released. In general, yes -- but blacklist on lock break has been part of exclusive-lock since the start. I am honestly not just making this up, this is how it works. > 2018-03-11 > ________________________________ > shadowlin > > ________________________________ > > 发件人：Jason Dillaman <jdillama@xxxxxxxxxx> > 发送时间：2018-03-11 07:46 > 主题：Re: Re: iSCSI Multipath (Load Balancing) vs RBD Exclusive > Lock > 收件人："shadow_lin"<shadow_lin@xxxxxxx> > 抄送："Mike Christie"<mchristi@xxxxxxxxxx>,"Lazuardi > Nasution"<mrxlazuardin@xxxxxxxxx>,"Ceph Users"<ceph-users@xxxxxxxxxxxxxx> > > On Sat, Mar 10, 2018 at 10:11 AM, shadow_lin <shadow_lin@xxxxxxx> wrote: >> Hi Jason, >> >>>As discussed in this thread, for active/passive, upon initiator >>>failover, we used the RBD exclusive-lock feature to blacklist the old >>>"active" iSCSI target gateway so that it cannot talk w/ the Ceph >>>cluster before new writes are accepted on the new target gateway. >> >> I can get during the new active target gateway was talking to rbd the old >> active target gateway cannot write because of the RBD exclusive-lock >> But after the new target gateway done the writes,if the old target gateway >> had some blocked io during the failover,cant it then get the lock and >> overwrite the new writes? > > Negative -- it's blacklisted so it cannot talk to the cluster. > >> PS: >> Petasan say they can do active/active iscsi with patched suse kernel. > > I'll let them comment on these corner cases. > >> 2018-03-10 >> ________________________________ >> shadowlin >> >> ________________________________ >> >> 发件人：Jason Dillaman <jdillama@xxxxxxxxxx> >> 发送时间：2018-03-10 21:40 >> 主题：Re: iSCSI Multipath (Load Balancing) vs RBD Exclusive Lock >> 收件人："shadow_lin"<shadow_lin@xxxxxxx> >> 抄送："Mike Christie"<mchristi@xxxxxxxxxx>,"Lazuardi >> Nasution"<mrxlazuardin@xxxxxxxxx>,"Ceph Users"<ceph-users@xxxxxxxxxxxxxx> >> >> On Sat, Mar 10, 2018 at 7:42 AM, shadow_lin <shadow_lin@xxxxxxx> wrote: >>> Hi Mike, >>> So for now only suse kernel with target_rbd_core and tcmu-runner can run >>> active/passive multipath safely? >> >> Negative, the LIO / tcmu-runner implementation documented here [1] is >> safe for active/passive. >> >>> I am a newbie to iscsi. I think the stuck io get excuted cause overwrite >>> problem can happen with both active/active and active/passive. >>> What makes the active/passive safer than active/active? >> >> As discussed in this thread, for active/passive, upon initiator >> failover, we used the RBD exclusive-lock feature to blacklist the old >> "active" iSCSI target gateway so that it cannot talk w/ the Ceph >> cluster before new writes are accepted on the new target gateway. >> >>> What mechanism should be implement to avoid the problem with >>> active/passive >>> and active/active multipath? >> >> Active/passive it solved as discussed above. For active/active, we >> don't have a solution that is known safe under all failure conditions. >> If LIO supported MCS (multiple connections per session) instead of >> just MPIO (multipath IO), the initiator would provide enough context >> to the target to detect IOs from a failover situation. >> >>> 2018-03-10 >>> ________________________________ >>> shadowlin >>> >>> ________________________________ >>> >>> 发件人：Mike Christie <mchristi@xxxxxxxxxx> >>> 发送时间：2018-03-09 00:54 >>> 主题：Re: iSCSI Multipath (Load Balancing) vs RBD Exclusive >>> Lock >>> 收件人："shadow_lin"<shadow_lin@xxxxxxx>,"Lazuardi >>> Nasution"<mrxlazuardin@xxxxxxxxx>,"Ceph Users"<ceph-users@xxxxxxxxxxxxxx> >>> 抄送： >>> >>> On 03/07/2018 09:24 AM, shadow_lin wrote: >>>> Hi Christie, >>>> Is it safe to use active/passive multipath with krbd with exclusive lock >>>> for lio/tgt/scst/tcmu? >>> >>> No. We tried to use lio and krbd initially, but there is a issue where >>> IO might get stuck in the target/block layer and get executed after new >>> IO. So for lio, tgt and tcmu it is not safe as is right now. We could >>> add some code tcmu's file_example handler which can be used with krbd so >>> it works like the rbd one. >>> >>> I do know enough about SCST right now. >>> >>> >>>> Is it safe to use active/active multipath If use suse kernel with >>>> target_core_rbd? >>>> Thanks. >>>> >>>> 2018-03-07 >>>> ------------------------------------------------------------------------ >>>> shadowlin >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> *发件人：*Mike Christie <mchristi@xxxxxxxxxx> >>>> *发送时间：*2018-03-07 03:51 >>>> *主题：*Re: iSCSI Multipath (Load Balancing) vs RBD >>>> Exclusive Lock >>>> *收件人：*"Lazuardi Nasution"<mrxlazuardin@xxxxxxxxx>,"Ceph >>>> Users"<ceph-users@xxxxxxxxxxxxxx> >>>> *抄送：* >>>> >>>> On 03/06/2018 01:17 PM, Lazuardi Nasution wrote: >>>> > Hi, >>>> > >>>> > I want to do load balanced multipathing (multiple iSCSI >>>> gateway/exporter >>>> > nodes) of iSCSI backed with RBD images. Should I disable exclusive >>>> lock >>>> > feature? What if I don't disable that feature? I'm using TGT >>>> (manual >>>> > way) since I get so many CPU stuck error messages when I was using >>>> LIO. >>>> > >>>> >>>> You are using LIO/TGT with krbd right? >>>> >>>> You cannot or shouldn't do active/active multipathing. If you have >>>> the >>>> lock enabled then it bounces between paths for each IO and will be >>>> slow. >>>> If you do not have it enabled then you can end up with stale IO >>>> overwriting current data. >>>> >>>> >>>> >>>> >>> >>> >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users@xxxxxxxxxxxxxx >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >> >> [1] http://docs.ceph.com/docs/master/rbd/iscsi-overview/ >> >> -- >> Jason > > > > -- > Jason [1] https://github.com/open-iscsi/tcmu-runner/blob/master/rbd.c#L542 -- Jason _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |