Hi list,
I'm working on Fibre Channel LIO setup featuring MPIO and ALUA.
My idea is to export the same backstore object as different LUNs of
two different FC targets, then create two ALUA groups and make those
two LUNs be members of my custom ALUA groups.
So I created my backstore object (fileio_0/file0), and exported it as
LUNs of two FC targets:
/> ls qla2xxx/21:00:00:0e:1e:15:e3:40/luns
o- luns ...................................................................... [2 LUNs]
o- lun0 .................................................. [iblock/disk0 (/dev/ram0)]
o- lun1 ................................... [fileio/file0 (/root/target-backstore-0)]
/> ls qla2xxx/21:00:00:0e:1e:15:e3:41/luns
o- luns ...................................................................... [2 LUNs]
o- lun0 ................................... [fileio/file0 (/root/target-backstore-0)]
o- lun1 .................................................. [iblock/disk1 (/dev/ram1)]
Then I created two ALUA groups "north" and "south":
tcm_node --addaluatpgwithmd fileio_0/file0 north 1
tcm_node --addaluatpgwithmd fileio_0/file0 south 2
(See complete setup "tcm_dump.txt" attached to this mail.)
Now when I check for group members I don't see any in my custom ALUA groups:
tcm_node --listtgptgps fileio_0/file0
\------> south Target Port Group ID: 2
Active ALUA Access Type(s): Implict and Explict
Primary Access State: Active/Optimized
Primary Access Status: Altered by Implict ALUA
Preferred Bit: 0
Active/NonOptimized Delay in milliseconds: 100
Transition Delay in milliseconds: 0
\------> TG Port Group Members
No Target Port Group Members
\------> north Target Port Group ID: 1
Active ALUA Access Type(s): Implict and Explict
Primary Access State: Active/Optimized
Primary Access Status: Altered by Implict ALUA
Preferred Bit: 0
Active/NonOptimized Delay in milliseconds: 100
Transition Delay in milliseconds: 0
\------> TG Port Group Members
No Target Port Group Members
\------> default_tg_pt_gp Target Port Group ID: 0
Active ALUA Access Type(s): Implict and Explict
Primary Access State: Active/Optimized
Primary Access Status: Altered by Implict ALUA
Preferred Bit: 0
Active/NonOptimized Delay in milliseconds: 100
Transition Delay in milliseconds: 0
\------> TG Port Group Members
qla2xxx/naa.2100000e1e15e340/tpgt_1/lun_1
qla2xxx/naa.2100000e1e15e341/tpgt_1/lun_0
As I checked all the "members" files in /sys/kernel/config/target/core/
directory are read-only, so I wasn't able to modify them by hand.
So my question is how can I add members to my own custom ALUA groups?
My guess is that I'm missing something in my setup (for example here I can
see iSCSI setup with custom ALUA groups sharing the same LUN:
https://groups.google.com/forum/#!topic/linux-iscsi-target-dev/QrZiOnLEzeo),
but I just can't figure out what.
Will appreciate your help very much,
Yuri
--
Yuri Arapov <yarapov@xxxxxxxxxxxxxxx>
modprobe target_core_mod
#### ALUA Logical Unit Groups
mkdir -p /sys/kernel/config/target/core/alua/lu_gps/south
mkdir -p /sys/kernel/config/target/core/alua/lu_gps/north
#### Parameters for TCM subsystem plugin storage object reference
tcm_node --establishdev iblock_1/disk1 /dev/ram1
tcm_node --setunitserialwithmd iblock_1/disk1 86fd57e7-d33a-4849-93ab-9c4ae5cbc699
#### ALUA Target Port Groups
tcm_node --addaluatpgwithmd iblock_1/disk1 default_tg_pt_gp 0
echo 3 > /sys/kernel/config/target/core/iblock_1/disk1/alua/default_tg_pt_gp/alua_access_type
echo 0 > /sys/kernel/config/target/core/iblock_1/disk1/alua/default_tg_pt_gp/preferred
echo 100 > /sys/kernel/config/target/core/iblock_1/disk1/alua/default_tg_pt_gp/nonop_delay_msecs
echo 0 > /sys/kernel/config/target/core/iblock_1/disk1/alua/default_tg_pt_gp/trans_delay_msecs
#### Attributes for /sys/kernel/config/target/core/iblock_1/disk1
echo 65535 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/max_write_same_len
echo 0 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/unmap_granularity_alignment
echo 8 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/unmap_granularity
echo 1 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/max_unmap_block_desc_count
echo 42949672 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/max_unmap_lba_count
echo 128 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/queue_depth
echo 8192 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/optimal_sectors
echo 8192 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/fabric_max_sectors
echo 512 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/block_size
echo 0 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/emulate_rest_reord
echo 0 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/is_nonrot
echo 1 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/enforce_pr_isids
echo 1 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/emulate_3pc
echo 1 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/emulate_caw
echo 0 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/emulate_tpws
echo 0 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/emulate_tpu
echo 1 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/emulate_tas
echo 0 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/emulate_ua_intlck_ctrl
echo 0 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/emulate_write_cache
echo 0 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/emulate_fua_read
echo 1 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/emulate_fua_write
echo 0 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/emulate_dpo
echo 0 > /sys/kernel/config/target/core/iblock_1/disk1/attrib/emulate_model_alias
#### Parameters for TCM subsystem plugin storage object reference
tcm_node --establishdev iblock_0/disk0 /dev/ram0
tcm_node --setunitserialwithmd iblock_0/disk0 19dcab6f-a3a3-484e-bee2-68e8e4d4ad8e
#### ALUA Target Port Groups
tcm_node --addaluatpgwithmd iblock_0/disk0 default_tg_pt_gp 0
echo 3 > /sys/kernel/config/target/core/iblock_0/disk0/alua/default_tg_pt_gp/alua_access_type
echo 0 > /sys/kernel/config/target/core/iblock_0/disk0/alua/default_tg_pt_gp/preferred
echo 100 > /sys/kernel/config/target/core/iblock_0/disk0/alua/default_tg_pt_gp/nonop_delay_msecs
echo 0 > /sys/kernel/config/target/core/iblock_0/disk0/alua/default_tg_pt_gp/trans_delay_msecs
#### Attributes for /sys/kernel/config/target/core/iblock_0/disk0
echo 65535 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/max_write_same_len
echo 0 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/unmap_granularity_alignment
echo 8 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/unmap_granularity
echo 1 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/max_unmap_block_desc_count
echo 42949672 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/max_unmap_lba_count
echo 128 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/queue_depth
echo 8192 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/optimal_sectors
echo 8192 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/fabric_max_sectors
echo 512 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/block_size
echo 0 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/emulate_rest_reord
echo 0 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/is_nonrot
echo 1 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/enforce_pr_isids
echo 1 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/emulate_3pc
echo 1 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/emulate_caw
echo 0 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/emulate_tpws
echo 0 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/emulate_tpu
echo 1 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/emulate_tas
echo 0 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/emulate_ua_intlck_ctrl
echo 0 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/emulate_write_cache
echo 0 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/emulate_fua_read
echo 1 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/emulate_fua_write
echo 0 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/emulate_dpo
echo 0 > /sys/kernel/config/target/core/iblock_0/disk0/attrib/emulate_model_alias
#### Parameters for TCM subsystem plugin storage object reference
tcm_node --establishdev fileio_0/file0 fd_dev_name=/root/target-backstore-0,fd_dev_size=512000000
tcm_node --setunitserialwithmd fileio_0/file0 d25dcb60-3d40-4c3a-af58-4008ee4143e2
#### ALUA Target Port Groups
tcm_node --addaluatpgwithmd fileio_0/file0 south 2
echo 3 > /sys/kernel/config/target/core/fileio_0/file0/alua/south/alua_access_type
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/alua/south/preferred
echo 100 > /sys/kernel/config/target/core/fileio_0/file0/alua/south/nonop_delay_msecs
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/alua/south/trans_delay_msecs
tcm_node --addaluatpgwithmd fileio_0/file0 north 1
echo 3 > /sys/kernel/config/target/core/fileio_0/file0/alua/north/alua_access_type
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/alua/north/preferred
echo 100 > /sys/kernel/config/target/core/fileio_0/file0/alua/north/nonop_delay_msecs
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/alua/north/trans_delay_msecs
tcm_node --addaluatpgwithmd fileio_0/file0 default_tg_pt_gp 0
echo 3 > /sys/kernel/config/target/core/fileio_0/file0/alua/default_tg_pt_gp/alua_access_type
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/alua/default_tg_pt_gp/preferred
echo 100 > /sys/kernel/config/target/core/fileio_0/file0/alua/default_tg_pt_gp/nonop_delay_msecs
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/alua/default_tg_pt_gp/trans_delay_msecs
#### Attributes for /sys/kernel/config/target/core/fileio_0/file0
echo 4096 > /sys/kernel/config/target/core/fileio_0/file0/attrib/max_write_same_len
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/attrib/unmap_granularity_alignment
echo 1 > /sys/kernel/config/target/core/fileio_0/file0/attrib/unmap_granularity
echo 1 > /sys/kernel/config/target/core/fileio_0/file0/attrib/max_unmap_block_desc_count
echo 8192 > /sys/kernel/config/target/core/fileio_0/file0/attrib/max_unmap_lba_count
echo 128 > /sys/kernel/config/target/core/fileio_0/file0/attrib/queue_depth
echo 8192 > /sys/kernel/config/target/core/fileio_0/file0/attrib/optimal_sectors
echo 8192 > /sys/kernel/config/target/core/fileio_0/file0/attrib/fabric_max_sectors
echo 512 > /sys/kernel/config/target/core/fileio_0/file0/attrib/block_size
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/attrib/emulate_rest_reord
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/attrib/is_nonrot
echo 1 > /sys/kernel/config/target/core/fileio_0/file0/attrib/enforce_pr_isids
echo 1 > /sys/kernel/config/target/core/fileio_0/file0/attrib/emulate_3pc
echo 1 > /sys/kernel/config/target/core/fileio_0/file0/attrib/emulate_caw
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/attrib/emulate_tpws
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/attrib/emulate_tpu
echo 1 > /sys/kernel/config/target/core/fileio_0/file0/attrib/emulate_tas
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/attrib/emulate_ua_intlck_ctrl
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/attrib/emulate_write_cache
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/attrib/emulate_fua_read
echo 1 > /sys/kernel/config/target/core/fileio_0/file0/attrib/emulate_fua_write
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/attrib/emulate_dpo
echo 0 > /sys/kernel/config/target/core/fileio_0/file0/attrib/emulate_model_alias
