Hi Michael, Adding CC' to some RHAT folks. On Tue, 2017-04-25 at 09:05 -0400, Michael Di Domenico wrote: > under rhel6 i was using scsi-target-utils to share out a read only > iscsi target (with 4 luns) to ~200 machines or so. this setup worked > just fine Btw, RHEL6 is using tgtd btw. > > now using identical hardware, we've reinstalled to rhel7 and have > switched over to targetcli, for the most part things seemed to > configure up just fine. was able to create the luns, set them > read-only, etc and connect an initiator into the target > > but when i start up multiple initiators as few as 50, i start to get > errors in syslog > > Exiting Time2Retain handler because session_reinstatement=1 > tcp_sendpage() failure: .... > > the tcp_sendpage() error has various numbers after it, the top two > highest count of numbers are "-512" and "3720" > Btw, this means that TCP connections are being reset. Without more information it's hard to know which side (eg: initiator or target) is forcing the TCP reset. Can we see some logs for both target + initiator side when this occurs..? > i found some tuning parameters around on the web for adjusting > targetcli parameters and attributes, but none seemed to make a > difference. > > so here's the rub. if i uninstall targetcli and install > scsi-target-utils from epel, the whole system works just fine > > > is there a performance limitation on targetcli using multiple > initiators to a single target? do i have something totally mistuned > (this is flat rhel7 no extra tuning anywhere)? At the kernel level, no. The scalability / longevity test I focus on for iscsi-target are in the 800-1000 volumes + target endpoints per node range, with small block random I/O traffic. One thing that comes to mind is the per target endpoint (eg: TPG attribute in targetcli) is 'default_cmdsn_depth', which by default is set to 64. This controls how many I/Os can be in flight for a single iscsi session. I"m not sure what the default is for tgtd in rhel6, but with enough initiators connected you might want to consider setting this to something like 16, 8, or lower. -- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
