BTW, in my program, i am doing some stress Testing, the program just repeatly do the things: create some volume, attached them, and then detach them, delete them then, just go on and on. when i am googleing the problem i met, i find this: https://eucalyptus.atlassian.net/browse/EUCA-3624 but i am not sure if i have the same problem, i will update once i confirm. what i want know is, for one target i can logout it, but for a lun, how can i make sure it 'logged out'. in my program, i just do this： "echo 1 >/sys/block/%s/device/delete", does it means 'logout'? At 2014-09-11 11:35:57, "席智勇" <xizhiyong18@xxxxxxx> wrote: >first of all, thanks for your reply. >in my program, it will use 128 LUNS at the most each target, so LUNS id will >not outnumber 255. > >here is some syslog when the problem occur. you can search [sdf] to find some >info, like "[sdf] READ CAPACITY(16) failed" or "[sdf] Unit Not Ready", and "sdf: detected capacity change from 9664266240 to 0", >"Sense Key : Illegal Request [current]", "Add. Sense: Logical unit not supported" >------------------------------------------------------------------------------------------ > >#:~/tgt_debug$ cat syslog |grep -C 1 -e "sdf\|sd 7:0:0:4" > >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.426520] sd 7:0:0:4: [sdf] 18875520 512-byte logical blocks: (9.66 GB/9.00 GiB) >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.426629] sd 7:0:0:3: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA >-- >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.427236] sd 7:0:0:5: Attached scsi generic sg12 type 0 >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.427409] sd 7:0:0:4: [sdf] Write Protect is off >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.427414] sd 7:0:0:4: [sdf] Mode Sense: 69 00 00 08 >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.427545] sd 7:0:0:5: [sdg] 2097280 512-byte logical blocks: (1.07 GB/1.00 GiB) >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.427875] sd 7:0:0:4: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.427926] scsi 7:0:0:7: Direct-Access IET VIRTUAL-DISK 0001 PQ: 0 ANSI: 5 >-- >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.443681] sd 7:0:0:5: [sdg] Attached SCSI disk >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.458714] sdf: unknown partition table >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.460786] sd 7:0:0:4: [sdf] Attached SCSI disk >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.476316] sdh: unknown partition table >-- >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.701790] Add. Sense: Reported luns data has changed >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.703810] sd 7:0:0:4: [sdf] Unit Not Ready >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.703814] sd 7:0:0:4: [sdf] >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.703815] Sense Key : Unit Attention [current] >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.703818] sd 7:0:0:4: [sdf] >Aug 26 16:02:25 10-120-120-35 kernel: [1144399.703821] Add. Sense: Reported luns data has changed >-- >Aug 26 16:02:26 10-120-120-35 kernel: [1144400.336478] Add. Sense: Reported luns data has changed >Aug 26 16:02:26 10-120-120-35 kernel: [1144400.338547] sd 7:0:0:4: [sdf] Unit Not Ready >Aug 26 16:02:26 10-120-120-35 kernel: [1144400.338552] sd 7:0:0:4: [sdf] >Aug 26 16:02:26 10-120-120-35 kernel: [1144400.338554] Sense Key : Unit Attention [current] >Aug 26 16:02:26 10-120-120-35 kernel: [1144400.338558] sd 7:0:0:4: [sdf] >Aug 26 16:02:26 10-120-120-35 kernel: [1144400.338561] Add. Sense: Reported luns data has changed >-- >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.752462] sdd: detected capacity change from 1073807360 to 0 >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.752987] sd 7:0:0:4: [sdf] Unit Not Ready >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.752990] sd 7:0:0:4: [sdf] >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.752991] Sense Key : Unit Attention [current] >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.752994] sd 7:0:0:4: [sdf] >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.752997] Add. Sense: Reported luns data has changed >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.753254] sd 7:0:0:4: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments. >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.753472] sd 7:0:0:4: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments. >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.753655] sd 7:0:0:4: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments. >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.753666] sd 7:0:0:4: [sdf] READ CAPACITY(16) failed >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.753672] sd 7:0:0:4: [sdf] >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.753674] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.753676] sd 7:0:0:4: [sdf] >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.753677] Sense Key : Unit Attention [current] >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.753680] sd 7:0:0:4: [sdf] >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.753682] Add. Sense: Reported luns data has changed >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.753883] sd 7:0:0:4: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments. >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.754075] sd 7:0:0:4: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments. >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.754275] sd 7:0:0:4: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments. >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.754286] sd 7:0:0:4: [sdf] READ CAPACITY failed >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.754291] sd 7:0:0:4: [sdf] >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.754292] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.754295] sd 7:0:0:4: [sdf] >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.754296] Sense Key : Unit Attention [current] >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.754299] sd 7:0:0:4: [sdf] >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.754301] Add. Sense: Reported luns data has changed >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.756150] sdf: detected capacity change from 9664266240 to 0 >Aug 26 16:02:28 10-120-120-35 kernel: [1144402.756639] sd 7:0:0:5: [sdg] Unit Not Ready >-- >Aug 26 16:02:29 10-120-120-35 kernel: [1144402.925588] scsi 3:0:0:21: Direct-Access IET VIRTUAL-DISK 0001 PQ: 0 ANSI: 5 >Aug 26 16:02:29 10-120-120-35 kernel: [1144402.925959] sd 7:0:0:4: [sdf] 18875520 512-byte logical blocks: (9.66 GB/9.00 GiB) >Aug 26 16:02:29 10-120-120-35 kernel: [1144402.926000] sd 3:0:0:21: Attached scsi generic sg59 type 0 >-- >Aug 26 16:02:29 10-120-120-35 kernel: [1144402.927749] sd 3:0:0:22: [sdbc] 18875520 512-byte logical blocks: (9.66 GB/9.00 GiB) >Aug 26 16:02:29 10-120-120-35 kernel: [1144402.927762] sdf: detected capacity change from 0 to 9664266240 >Aug 26 16:02:29 10-120-120-35 kernel: [1144402.927843] scsi 3:0:0:23: Direct-Access IET VIRTUAL-DISK 0001 PQ: 0 ANSI: 5 >-- >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.191743] Add. Sense: Reported luns data has changed >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.195776] sd 7:0:0:4: [sdf] Unit Not Ready >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.195779] sd 7:0:0:4: [sdf] >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.195780] Sense Key : Unit Attention [current] >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.195784] sd 7:0:0:4: [sdf] >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.195786] Add. Sense: Reported luns data has changed >-- >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.592427] sd 7:0:0:39: [sdbx] Mode Sense: 69 00 00 08 >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.592809] sd 7:0:0:40: Attached scsi generic sg82 type 0 >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.594486] sd 7:0:0:39: [sdbx] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.594696] sd 7:0:0:40: [sdby] 18875520 512-byte logical blocks: (9.66 GB/9.00 GiB) >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.595825] sd 7:0:0:40: [sdby] Write Protect is off >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.595831] sd 7:0:0:40: [sdby] Mode Sense: 69 00 00 08 >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.596261] sd 7:0:0:40: [sdby] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.605687] sdbx: unknown partition table >-- >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.610455] sd 7:0:0:39: [sdbx] Attached SCSI disk >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.613057] sd 7:0:0:40: [sdby] Attached SCSI disk >Aug 26 16:02:29 10-120-120-35 kernel: [1144403.617631] sd 3:0:0:13: [sdaf] Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments. >-- >Aug 26 16:02:33 10-120-120-35 kernel: [1144407.432518] md113: unknown partition table >Aug 26 16:02:33 10-120-120-35 kernel: [1144407.662356] sd 7:0:0:4: [sdf] Synchronizing SCSI cache >Aug 26 16:02:33 10-120-120-35 kernel: [1144407.663163] sd 7:0:0:4: [sdf] >Aug 26 16:02:33 10-120-120-35 kernel: [1144407.663167] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE >Aug 26 16:02:33 10-120-120-35 kernel: [1144407.663170] sd 7:0:0:4: [sdf] >Aug 26 16:02:33 10-120-120-35 kernel: [1144407.663172] Sense Key : Illegal Request [current] >Aug 26 16:02:33 10-120-120-35 kernel: [1144407.663175] sd 7:0:0:4: [sdf] >Aug 26 16:02:33 10-120-120-35 kernel: [1144407.663178] Add. Sense: Logical unit not supported > > >At 2014-09-11 02:21:59, "ronnie sahlberg" <ronniesahlberg@xxxxxxxxx> wrote: >>When you say "many devices" exactly how many are you referring to? >>Can you check if this problem you have only occur when you go from 255 >>LUNs to something higher? >> >> >>The default LUN addressing scheme only allows LUNs numbered 0 to 255. >>To go to higher LUN numbers, which TGTD supports, doe require that you >>switch to a different addressing format >>and it may be that your initiator can not handle those other modes. >> >>Often it might look like you have LUNs 0-255 and then there is a jump >>to the next LUN which has the number 16640 if the initiator can not >>handle addressing modes properly. >> >> >>I do not know how IET does the LUN numbering. >> >>If you can collect a wireshark trace of iscsiadm --login from both >>STGT and IET then I can take a quick look and see if that is the issue >>you are facing. >> >> >> >>regards >>ronnie sahlberg >> >> >>On Mon, Sep 1, 2014 at 11:00 PM, 席智勇 <xizhiyong18@xxxxxxx> wrote: >>> >>> when i export many block device by tgt , in one single target , this means many exported lun in one target, to another >>> machine by iscsi。 >>> >>> in another machine , attach the devices by open-iscsi。when i wait the >>> size of the device attached change from 0 to correct size， i create raid1 with two devices by mdadm。at this time, mdadm report like >>> "/dev/sdem is not suitable for this array.", i read the code of mdadm ,that because it cannot open the device。sometime it even report >>> "failed to open /dev/** after earlier success - aborting "。 >>> >>> i do not know whether it's because i used the device when it's not ready yet, or the problem of tgt。 >>> >>> all the above said happened when many target lun(tgt) created at the same >>> time concurrently。it do not happen in nomal time. when i change tgt to iet, it >>> do not happen either。 >>> >>> >>> >>> tgt version: 1.0.48 >>> >>> open-iscsi version:iscsiadm version 2.0-873 >>> >>> linux system: Linux 3.10.40-amd64 GNU/Linux >>> >>> >>> >>> >>> >>> ------------- >>> >>> Zhiyong Xi ?韬{.n?壏煯壄?%娝?檩?w?{.n?壏 ?{ay?蕠跈?jf＂穐殢飦?戧鐉_璁(殠娸"濟?m??G珴?⒏?櫒璀?ｘ忈
