Greetings folks, This patch series adds complete fabric level support for Asymmetric Logical Unit Assignment (ALUA) explict (w/ remote SET_TARGET_PORT_GROUPS in-band method) and implict (w/ local configfs method) with all SPC-4 defined ALUA primary and secondary access states, as well as the full ALUA logical unit group abstraction to the lio-core-2.6.git tree. This patch series has been tested on v2.6.31-rc5 x86 KVM using sg_stpg for the explict transition case from Open-iSCSI Initiator, and using lio-utils.git CLI ops for the implict case. The userspace commits to make it go can be found here: http://git.kernel.org/?p=linux/storage/lio/lio-utils.git;a=commitdiff;h=481916be1c568fbab505ade534527d60b33bbb8a http://git.kernel.org/?p=linux/storage/lio/lio-utils.git;a=commitdiff;h=7e3e2134397145ce75aba2cbd1725abe761bb8bd Here are the patches: [PATCH 1/4] [Target_Core_Mod/ALUA]: Add support for explict/implict ALUA primary/secondary state transition [PATCH 2/4] [Target_Core_Mod/ConfigFS]: Add implict ALUA attributes and move tg_pt_gp list [PATCH 3/4] [LIO-Target]: Add ALUA Active/NonOptimized state check [PATCH 4/4] [LIO-Target/ConfigFS]: Add attribute for ALUA secondary port access state Any comments from the SCSI and multipath folks would be apperciated, Many thanks for your most valuable of time, --nab Here is what it looks like in action with Open-iSCSI Initiators using scsi_dh_alua + multipath: *) A single TCM storage object (iblock/lvm_test0) has two non default ALUA target port groups created for itself, lio_alua_west and lio_alua_east. Each of these two ALUA target port groups has a diSCSI target port/LUN active from different iSCSI target endpoints: target:~# tcm_node --listaluatpgs iblock_0/lvm_test0 \------> lio_alua_west Target Port Group ID: 1 Active ALUA Access Type(s): Implict and Explict Primary Access State: Active/Optimized Primary Access Status: None Preferred Bit: 0 Active/NonOptimized Delay in milliseconds: 100 \------> TG Port Group Members iSCSI/iqn.2003-01.org.linux-iscsi.target.i686:sn.e475ed6fcdd0/tpgt_1/lun_0 \------> lio_alua_east Target Port Group ID: 2 Active ALUA Access Type(s): Implict and Explict Primary Access State: Active/Optimized Primary Access Status: None Preferred Bit: 0 Active/NonOptimized Delay in milliseconds: 100 \------> TG Port Group Members iSCSI/iqn.2003-01.org.linux-iscsi.target.i686:sn.cff3eedbd2fd/tpgt_1/lun_0 <SNIP> *) On the Initiator side, two sessions are created to each iSCSI target endpoint: [ 877.235778] sd 4:0:0:0: Attached scsi generic sg8 type 0 [ 877.669556] sd 4:0:0:0: alua: supports implicit and explicit TPGS [ 877.676442] sd 4:0:0:0: alua: port group 01 rel port 02 [ 877.681561] sd 4:0:0:0: alua: port group 01 state A supports TOUSNA [ 877.726035] sd 4:0:0:0: alua: port group 01 state A supports TOUSNA [ 877.746897] sd 3:0:0:0: alua: supports implicit and explicit TPGS [ 877.755445] sd 3:0:0:0: alua: port group 01 rel port 01 [ 877.760968] sd 3:0:0:0: alua: port group 01 state A supports TOUSNA The two iSCSI Target port/LUNs appear as /dev/sde and /dev/sdi on the initiator under mpath4: mpath4 (36001405a97e4ce21c0711de829b000c2) dm-2 LIO-ORG,IBLOCK [size=500M][features=0][hwhandler=1 alua] \_ round-robin 0 [prio=100][enabled] \_ 3:0:0:0 sde 8:64 [active][ready] \_ 4:0:0:0 sdi 8:128 [active][ready] *) Next using explict ALUA from the initiator, change TG_PT_GP_ID=1 (lio_alua_west) to Active/NonOptimized: initiator# sg_stpg -vvv -a /dev/sdi --tp 1 open /dev/sdi with flags=0x802 set target port groups cdb: a4 0a 00 00 00 00 00 00 00 08 00 00 set target port groups parameter list: 00 00 00 00 01 00 00 01 duration=8 ms *) Send a REPORT_TARGET_PORT_GROUPS from the initiator to one of the iSCSI LUNs: initiator# sg_rtpg --decode /dev/sde Report target port groups: <SNIP> target port group id : 0x2 , Pref=0 target port group asymmetric access state : 0x00 (active/optimized) T_SUP : 1, O_SUP : 1, U_SUP : 1, S_SUP : 1, AN_SUP : 1, AO_SUP : 1 status code : 0x00 (no status available) vendor unique status : 0x00 target port count : 01 Relative target port ids: 0x02 target port group id : 0x1 , Pref=0 target port group asymmetric access state : 0x01 (active/non optimized) T_SUP : 1, O_SUP : 1, U_SUP : 1, S_SUP : 1, AN_SUP : 1, AO_SUP : 1 status code : 0x01 (target port asym. state changed by SET TARGET PORT GROUPS command) vendor unique status : 0x00 target port count : 01 Relative target port ids: 0x01 *) Then, using implict ALUA via configfs, change lio_alua_east (TG_PT_GP_ID=2) to Standby: target:~# tcm_node --setaluastate iblock_0/lvm_test0 lio_alua_east s Successfully set primary ALUA access state for TG PT Group: lio_alua_east to Standby and then check the status again to see the updated Primary Access State and Status for lio_alua_east: target:~# tcm_node --listaluatpgs iblock_0/lvm_test0 \------> lio_alua_west Target Port Group ID: 1 Active ALUA Access Type(s): Implict and Explict Primary Access State: Active/NonOptimized Primary Access Status: Altered by Explict STPG Preferred Bit: 0 Active/NonOptimized Delay in milliseconds: 100 \------> TG Port Group Members iSCSI/iqn.2003-01.org.linux-iscsi.target.i686:sn.e475ed6fcdd0/tpgt_1/lun_0 \------> lio_alua_east Target Port Group ID: 2 Active ALUA Access Type(s): Implict and Explict Primary Access State: Standby Primary Access Status: Altered by Implict ALUA Preferred Bit: 0 Active/NonOptimized Delay in milliseconds: 100 \------> TG Port Group Members iSCSI/iqn.2003-01.org.linux-iscsi.target.i686:sn.cff3eedbd2fd/tpgt_1/lun_0 <SNIP> *) On the initiator side, check multipath status for lio_alua_east (TG_PT_GP_ID=2) going into Standby: mpath4 (36001405a97e4ce21c0711de829b000c2) dm-2 LIO-ORG,IBLOCK [size=500M][features=0][hwhandler=1 alua] \_ round-robin 0 [prio=10][enabled] \_ 3:0:0:0 sde 8:64 [active][ready] \_ 4:0:0:0 sdi 8:128 [active][faulty] *) Using implict ALUA via configfs, set lio_alua_east (TG_PT_GP_ID=2) back to Active/Optimized: target:~# tcm_node --setaluastate iblock_0/lvm_test0 lio_alua_east o Successfully set primary ALUA access state for TG PT Group: lio_alua_east to Active/Optimized *) Set the preferred bit for lio_alua_east (TG_PT_GP_ID=2) target:~# tcm_node --setaluapref iblock_0/lvm_test0 lio_alua_east Successfully enabled PREFERRED bit for TG Pt Group: lio_alua_east *) Issue REPORT_TARGET_PORT_GROUPS again from the initiator side to see the updated status: initiator# sg_rtpg --decode /dev/sde Report target port groups: <SNIP> target port group id : 0x2 , Pref=1 target port group asymmetric access state : 0x00 (active/optimized) T_SUP : 1, O_SUP : 1, U_SUP : 1, S_SUP : 1, AN_SUP : 1, AO_SUP : 1 status code : 0x02 (target port asym. state changed by implicit lu behaviour) vendor unique status : 0x00 target port count : 01 Relative target port ids: 0x02 target port group id : 0x1 , Pref=0 target port group asymmetric access state : 0x01 (active/non optimized) T_SUP : 1, O_SUP : 1, U_SUP : 1, S_SUP : 1, AN_SUP : 1, AO_SUP : 1 status code : 0x01 (target port asym. state changed by SET TARGET PORT GROUPS command) vendor unique status : 0x00 target port count : 01 Relative target port ids: 0x01 *) And check the multipath status again from the initiator to see the updates status: mpath4 (36001405a97e4ce21c0711de829b000c2) dm-2 LIO-ORG,IBLOCK [size=500M][features=0][hwhandler=1 alua] \_ round-robin 0 [prio=60][enabled] \_ 3:0:0:0 sde 8:64 [active][ready] \_ 4:0:0:0 sdi 8:128 [active][ready] *) And finally, the output for the above sequence from dmesg on the target side showing the explict/implict ALUA state transitions and the allocated UNIT ATTENTIONs to notify SCSI Initiator Ports of the transitions: [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06 [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06 [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06 [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06 [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06 [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06 [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06 Successful explict ALUA transition TG PT Group: lio_alua_west ID: 1 from primary access state: Active/Optimized to Active/NonOptimized [iSCSI]: Releasing UNIT ATTENTION condition with INTLCK_CTRL: 0, mapped LUN: 0, got CDB: 0x00 reported ASC: 0x2a, ASCQ: 0x06 [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06 Successful implict ALUA transition TG PT Group: lio_alua_east ID: 2 from primary access state: Active/Optimized to Standby [iSCSI]: Releasing UNIT ATTENTION condition with INTLCK_CTRL: 0, mapped LUN: 0, got CDB: 0x00 reported ASC: 0x2a, ASCQ: 0x06 [iSCSI]: Allocated UNIT ATTENTION, mapped LUN: 0, ASC: 0x2a, ASCQ: 0x06 Successful implict ALUA transition TG PT Group: lio_alua_east ID: 2 from primary access state: Standby to Active/Optimized [iSCSI]: Releasing UNIT ATTENTION condition with INTLCK_CTRL: 0, mapped LUN: 0, got CDB: 0x00 reported ASC: 0x2a, ASCQ: 0x06 -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
