Hi Nicholas, I found a backdoor on the initiator to limit outstanding commands per session to one. This did not make the data digest error mis-detection problem go away. However, I was able to rule out the multiple outstanding "writes" as a possible contributor to the problem. It also simplified the trace. I a not sure where to proceed further on this. Do you want me to send you the latest Wireshark trace and /ver/log/messages? Thanks, -Joe ----- Original Message ----- From: "jrepac@xxxxxxxxx" <jrepac@xxxxxxxxx> To: Nicholas A. Bellinger <nab@xxxxxxxxxxxxxxx> Cc: target-devel <target-devel@xxxxxxxxxxxxxxx> Sent: Thursday, February 9, 2012 4:18 PM Subject: Re: Data Digest Errors Hi Nicholas, No dice on the MaxOutStandingR2T=1. It is already negotiated to be 1. However, the restriction doe not apply to initial R2Ts. Thanks for the pointer to iSCSI cmdSN session depth control on the target. My plan is to limit commands "inflight" to 1. There appears to be three writes in process on the session just before the failure. Limiting outstanding commands will at least give us a means to rule this out as a possible contributor to the failure. Thanks, -Joe ----- Original Message ----- From: Nicholas A. Bellinger <nab@xxxxxxxxxxxxxxx> To: jrepac@xxxxxxxxx Cc: target-devel <target-devel@xxxxxxxxxxxxxxx> Sent: Wednesday, February 8, 2012 7:51 PM Subject: Re: Data Digest Errors On Mon, 2012-02-06 at 07:25 -0800, jrepac@xxxxxxxxx wrote: > I noted something interesting with the digest error problem noted > over the past two months with the Emulex initiator and the target. > It seems that problem occurs just after the target sends out a sequence > of multiple tightly spaced R2Ts for outstanding write commands. The > problem occurs while simply trying to initialize or format the drive > with the MS iSCSI initiator (using the Emulex adapter instance). Errors > are reported in /var/log/messages as data digest failures but the wireshark > trace shows the digest is actually good. The MS drive initialization > completes fine when both data and header digest are disabled. > Hi Joe, Thanks for continuing to look into this issue. > This problem only seems to occur when a group of R2T's are sent > out with <20us between PDUs. So thinking about this a bit more, you may want to verify that the MaxOutstandingR2T value is being negotiated to '1' during iSCSI session login. Note that mainline iscsi_target_mod code does support >1 operation, but I'm pretty sure the MSFT login logic should be negotiating MaxOutstandingR2T=1.. Definitely worth checking at this point. > I did four trace captures on this last night and the behavior seems to be consistent. > The magic number so far is three R2Ts. Nicholas fixed a list corruption issue (noted below) > that was occurring when the first mis-detected digest error report. However, the mis-detected > digest error remains. It seems like data for another outstanding write command (released by > one of the R2Ts) is interfering with a digest calculation in progress. > > Can someone familiar with this area of the take a look at this? I plan to cut the > queue depth on the target to 1 today to see if the problem goes away. > I assume you mean iSCSI CmdSN session depth here, yes..? By default this value is currently set to 16, and can be changed via default_queue_depth (for TPG demo-mode operation) or via queue_depth attribute for explict iSCSI initiator NodeACLs. --nab -- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
