----- Original Message ----- > From: "Robert LeBlanc" <robert@xxxxxxxxxxxxx> > To: "Laurence Oberman" <loberman@xxxxxxxxxx> > Cc: "Doug Ledford" <dledford@xxxxxxxxxx>, "Nicholas A. Bellinger" <nab@xxxxxxxxxxxxxxx>, "Zhu Lingshan" > <lszhu@xxxxxxxx>, "linux-rdma" <linux-rdma@xxxxxxxxxxxxxxx>, linux-scsi@xxxxxxxxxxxxxxx, "Sagi Grimberg" > <sagi@xxxxxxxxxxx>, "Christoph Hellwig" <hch@xxxxxx> > Sent: Friday, January 13, 2017 6:38:33 PM > Subject: Re: iscsi_trx going into D state > > Laurance, > > I'm really starting to think that the stars aligned with the phase of > the moon or something when I reproduced this in my lab before because > I've been unable to reproduce it on Infiniband the last two days. The > problem with this issue is that it is so hard to trigger, but causes a > lot of problems when it does happen. I really hate wasting people's > time when I can't reproduce it myself reliably. Please don't waste too > much time if you can't get it reproduced on Infiniband, I'll have to > wait until someone with the ConnectX-4-LX cards can replicate it. > > Hmmm.... you do have ConnectX-4 cards which may have the same bug it > Ethernet mode. I don't see the RoCE bug on my ConnectX-3 cards, but > your ConnectX-4 cards may work. Try putting the cards into Ethernet > mode, set the speed and advertised speed to something lower than the > max speed and verify that the link speed is that (ethtool). On the > ConnectX-4-LX cards, I just had to set both interfaces down and then > back up at the same time, on the ConnectX-3 I had to pull the cable > (shutting down the client might have worked). Then set up target and > client with iSER, format and run the test and it should trigger > automatically. > > Looking at release notes on the ConnectX-4-LX cards, the latest > firmware may fix the bug that so easily exposes the problem with that > card. My cards are SuperMicro branded cards and don't have the new > firmware available yet. > > Good luck. > ---------------- > Robert LeBlanc > PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 > > > On Fri, Jan 13, 2017 at 8:10 AM, Laurence Oberman <loberman@xxxxxxxxxx> > wrote: > > > > > > ----- Original Message ----- > >> From: "Robert LeBlanc" <robert@xxxxxxxxxxxxx> > >> To: "Laurence Oberman" <loberman@xxxxxxxxxx> > >> Cc: "Doug Ledford" <dledford@xxxxxxxxxx>, "Nicholas A. Bellinger" > >> <nab@xxxxxxxxxxxxxxx>, "Zhu Lingshan" > >> <lszhu@xxxxxxxx>, "linux-rdma" <linux-rdma@xxxxxxxxxxxxxxx>, > >> linux-scsi@xxxxxxxxxxxxxxx, "Sagi Grimberg" > >> <sagi@xxxxxxxxxxx>, "Christoph Hellwig" <hch@xxxxxx> > >> Sent: Thursday, January 12, 2017 4:26:05 PM > >> Subject: Re: iscsi_trx going into D state > >> > >> Sorry sent prematurely... > >> > >> On Thu, Jan 12, 2017 at 2:22 PM, Robert LeBlanc <robert@xxxxxxxxxxxxx> > >> wrote: > >> > I'm having trouble replicating the D state issue on Infiniband (I was > >> > able to trigger it reliably a couple weeks back, I don't know if OFED > >> > to verify the same results happen there as well. > >> > >> I'm having trouble replicating the D state issue on Infiniband (I was > >> able to trigger it reliably a couple weeks back, I don't know if OFED > >> being installed is altering things but it only installed for 3.10. The > >> ConnectX-4-LX exposes the issue easily if you have those cards.) to > >> verify the same results happen there as well. > >> > >> ---------------- > >> Robert LeBlanc > >> PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 > >> -- > >> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in > >> the body of a message to majordomo@xxxxxxxxxxxxxxx > >> More majordomo info at http://vger.kernel.org/majordomo-info.html > >> > > > > I am only back in the office next Wednesday. > > I have this all setup using ConnectX-4 with IB/ISER but have no way of > > remotely creating the disconnect as I currently have it back-to-back. > > Have run multiple tests with IB and ISER hard resting the client to break > > the IB connection but have not been able to reproduce as yet. > > So it will have to wait until I can pull cables next week as that seemed to > > be the way you have been reproducing this. > > > > This is in a code area I also don't have a lot of knowledge of the flow but > > have started trying to understand it better. > > > > Thanks > > Laurence > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-scsi" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > Hello Robert I will try this sometime tomorrow by running in ethernet mode. Its been days of resets with no reproduction so I agree, very hard ro trproduce with Infiniband. Thanks Laurence -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
