On Wed, 11 Mar 2009 14:05:42 +0100 Tomasz Chmielewski <mangoo@xxxxxxxx> wrote: > >> I understand it's a major design problem - do you think it could be > >> addressed? I.e., use prefork model used by Apache and just launch more > >> tgtd processes? > > > > I'm not sure Apache uses such model, 'client per process with > > preforking) because it sucks (I guess it used the model): > > > > http://www.kegel.com/c10k.html > > Apache binary can be build to run in different modes: prefork, worker etc. That's not the point, Apache support several models. The point is that the model that you talked about is inferior. > > Anyway, the target/lun limit problem is bad. So here's a patch to fix > > it. I've not tested it much. So let me know how it works. > > Great! I applied the patch to 0.9.4. > > On a 32 bit x86 system with 256 MB RAM I created 20000 targets, > each with one lun (without connecting any initiators). > > Later, I added another one target with 4200 luns. With the patch, tgtd creates some threads and all the targets and luns share them; a target or lun doesn't create the own threads. So you can have tons of targets and luns. Thanks for the reports. > There are the issues I found: > > 1) "tgtadm --op show --mode target", which: > - doesn't list all targets > - output for the last target is sometimes not full, i.e.: > > Target 19836: iqn.2007-02.com.example:test.break19836 > System information: > Driver: iscsi > State: ready > I_T nexus information: > LUN information: > LUN: 0 > Type: controller > SCSI ID: deadbeaf19836:0 > SCSI SN: beaf198360 > Size: 0 MB > Online: Yes > Removable media: No > Backing store: No backing store > LUN: 1 > Type: disk > SCSI ID: deadbeaf19836:1 > SCSI SN: beaf198361 > Size: 1024 MB > Online: Yes > Removable media: No > Backing store: /dev/sda > Account information: > Asuperthecus:/mnt/1# > > Note that the last piece of information outputted by tgtadm is: > > Account information: > A > > And it missed some targets up to 20000. > > Or, a similar output when there are lots of luns (4200): > > LUN: 3431 > Type: disk > SCSI ID: deadbeaf20001:3431 > SCSI SN: beaf200013431 > Size: 1024 MB > Online: Yes > Removable media: No > Backing store: /dev/sda > LUN: 3432 > Type: disk > SCSI ID: deadbeaf20001:3432 > SCSI SN: beaf200013432 > Size: 1024 MB > superthecus:/mnt/1# I think that there is some limit about the amount of data that tgtd can transfer to tgtadm. > 2) running "iscsiadm -m discovery -t sendtargets -p <TARGET_IP> while we keep > adding new targets breaks the target > (i.e. you can't add any new targets: showing target is not possible). (snip) > To reproduce: > > tgtd > i=1 > while [ $i -ne 20000 ] ; do > echo tgtadm --lld iscsi --op new --mode target --tid $i -T iqn.2007-02.com.example:test.break$i > tgtadm --lld iscsi --op new --mode target --tid $i -T iqn.2007-02.com.example:test.break$i > echo tgtadm --lld iscsi --op new --mode logicalunit --tid $i --lun 1 -b /dev/sda > tgtadm --lld iscsi --op new --mode logicalunit --tid $i --lun 1 -b /dev/sda > [ $? -ne 0 ] && echo Oooops && sleep 10m > tgtadm --lld iscsi --op bind --mode target --tid $i -I ALL > i=$((i+1)) > done > > > On the initiator, run - you will see problems like below: > > # iscsiadm -m discovery -t sendtargets -p 192.168.111.177 > iscsiadm: Connection to Discovery Address 192.168.111.177 closed > > # iscsiadm -m discovery -t sendtargets -p 192.168.111.177 > iscsiadm: buffer size 32768 too small for data length 68010 > > # iscsiadm -m discovery -t sendtargets -p 192.168.111.177 > iscsiadm: socket 3 header read timed out > iscsiadm: Login I/O error, failed to receive a PDU > iscsiadm: retrying discovery login to 192.168.111.177 > iscsiadm: socket 3 header read timed out > iscsiadm: Login I/O error, failed to receive a PDU > iscsiadm: retrying discovery login to 192.168.111.177 > > > 3) even when we use a much smaller number of targets (i.e. 350), running: > > # iscsiadm -m discovery -t sendtargets -p 192.168.111.177 > > succeeds for the first time, but the second run fails (i.e., tgtd does not answer). > > Maximum number of targets with one lun which worked for me was 96 (on 32 bit). > With 97, it was failing. We inherits this problem from IET (IET inherits it from Ardis iSCSI code). 'sendtargets' can't handle large data. open-iscsi might have the similar problem (I can't remember). The patch is not ready yet. I'll work on it after 0.9.5. -- To unsubscribe from this list: send the line "unsubscribe stgt" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
