On 03/14/2018 01:26 PM, Michael Christie wrote: > On 03/14/2018 01:06 PM, Maxim Patlasov wrote: >> On Sun, Mar 11, 2018 at 5:10 PM, Mike Christie <mchristi@xxxxxxxxxx >> <mailto:mchristi@xxxxxxxxxx>> wrote: >> >> On 03/11/2018 08:54 AM, shadow_lin wrote: >> > Hi Jason, >> > How the old target gateway is blacklisted? Is it a feature of the target >> > gateway(which can support active/passive multipath) should provide or is >> > it only by rbd excusive lock? >> > I think excusive lock only let one client can write to rbd at the same >> > time,but another client can obtain the lock later when the lock is released. >> >> For the case where we had the lock and it got taken: >> >> If IO was blocked, then unjammed and it has already passed the target >> level checks then the IO will be failed by the OSD due to the >> blacklisting. When we get IO errors from ceph indicating we are >> blacklisted the tcmu rbd layer will fail the IO indicating the state >> change and that the IO can be retried. We will also tell the target >> layer rbd does not have the lock anymore and to just stop the iscsi >> connection while we clean up the blacklisting, running commands and >> update our state. >> >> >> Mike, can you please give more details on how you tell the target layer >> rbd does not have the lock and to stop iscsi connection. Which >> tcmu-runner/kernel-target functions are used for that? > > For this case it would be tcmu_rbd_handle_blacklisted_cmd. Note for > failback type of test, we might not hit that error if the initiator does > a RTPG before it sends IO. In that case we would see > tcmu_update_dev_lock_state get run first and the iscsi connection would > not be dropped. > >> >> In fact, I performed an experiment with three stale write requests stuck >> on blacklisted gateway, and one of them managed to overwrite newer data. > > What is the test exactly? What OS for the initiator? > > What kernel were you using and are you using the upstream tools/libs or > the RHCS ones? > > Can you run your tests and send the initiator side kernel logs and on > the iscsi targets send the /var/log/tcmu-runner.log with debugging in > enabled. To do that open > > /etc/tcmu/tcmu.conf > > on the iscsi target nodes and set > > log_level = 5 > > If that is too much output drop it to level 4. > > >> I followed all instructions from >> http://docs.ceph.com/docs/master/rbd/iscsi-target-cli-manual-install/ >> and http://docs.ceph.com/docs/master/rbd/iscsi-target-cli/, so I'm >> interested what I'm missing... > > You used the initiator settings in > http://docs.ceph.com/docs/master/rbd/iscsi-initiators/ > too right? > Ignore all these questions. I'm pretty sure I know the issue. _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |