Hello.
I'm trying to get multipathing working between LIO target and VMWare ESXi 5.1 U1. However, I'm not having a lot of luck as I'm not entirely sure which side is the one not playing nice. From VMWare side, I can see that it's using each interface, but it's only managing to connect to single interface on the target.
############
Runtime name Target LUN
vmhba37:C4:T0:L0 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 0
vmhba37:C4:T0:L1 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 1
vmhba37:C4:T0:L2 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 2
vmhba37:C5:T0:L0 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 0
vmhba37:C5:T0:L1 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 1
vmhba37:C5:T0:L2 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 2
vmhba37:C6:T0:L0 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 0
vmhba37:C6:T0:L1 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 1
vmhba37:C6:T0:L2 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 2
vmhba37:C7:T0:L0 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 0
vmhba37:C7:T0:L1 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 1
vmhba37:C7:T0:L2 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 2
############
As far as I'm aware, every interface is responding as they should (only respond to ARP requests on their own IP & packet arriving on interface X will be sent back by same interface).
############
~ # esxcli network ip neighbor list #ESXi
Neighbor Mac Address Vmknic Expiry State
------------------------- ----------------- ------ -------- ---------
10.255.10.5 bc:ae:c5:1b:94:d7 vmk1 1020 sec
10.255.10.6 00:1b:21:a5:1e:29 vmk1 1013 sec
10.255.10.7 00:1b:21:a5:1e:28 vmk1 906 sec
10.255.10.8 00:1b:21:a5:1e:2b vmk1 909 sec
~ # esxcli network ip interface ipv4 get
Name IPv4 Address IPv4 Netmask IPv4 Broadcast Address Type DHCP DNS
---- ------------ ------------- -------------- ------------ --------
vmk1 10.255.10.9 255.255.255.0 10.255.10.255 STATIC false
vmk2 10.255.10.10 255.255.255.0 10.255.10.255 STATIC false
vmk3 10.255.10.11 255.255.255.0 10.255.10.255 STATIC false
vmk4 10.255.10.12 255.255.255.0 10.255.10.255 STATIC false
root@tibbers:~# ip addr | grep 10.255.10. #Target
inet 10.255.10.6/24 brd 10.255.10.255 scope global eth2
inet 10.255.10.7/24 brd 10.255.10.255 scope global eth3
inet 10.255.10.5/24 brd 10.255.10.255 scope global eth1
inet 10.255.10.8/24 brd 10.255.10.255 scope global eth4
root@tibbers:~# tcpdump -i eth2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes
10:32:39.705869 IP 10.255.10.9 > 10.255.10.6: ICMP echo request, id 32557, seq 0, length 64
10:32:39.706018 IP 10.255.10.6 > 10.255.10.9: ICMP echo reply, id 32557, seq 0, length 64
10:32:40.707893 IP 10.255.10.9 > 10.255.10.6: ICMP echo request, id 32557, seq 1, length 64
10:32:40.707905 IP 10.255.10.6 > 10.255.10.9: ICMP echo reply, id 32557, seq 1, length 64
root@tibbers:~# tcpdump -i eth3
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth3, link-type EN10MB (Ethernet), capture size 65535 bytes
10:37:09.475447 IP 10.255.10.9 > 10.255.10.7: ICMP echo request, id 9006, seq 0, length 64
10:37:09.475651 IP 10.255.10.7 > 10.255.10.9: ICMP echo reply, id 9006, seq 0, length 64
10:37:10.477539 IP 10.255.10.9 > 10.255.10.7: ICMP echo request, id 9006, seq 1, length 64
10:37:10.477558 IP 10.255.10.7 > 10.255.10.9: ICMP echo reply, id 9006, seq 1, length 64
############
When VMWare tries to connect to target, these show up in the target's dmesg (seemingly randomly, sometimes it's just one type, sometimes it's both. And Min/Max keep changing):
############
[95195.714641] First opcode is not login request, failing login request.
[95195.714694] iSCSI Login negotiation failed.
[95195.714991] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x5f/0x6c, rejecting login.
[95195.715111] iSCSI Login negotiation failed.
[93950.152667] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x6e/0x75, rejecting login.
[94053.807933] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0xc0/0x01, rejecting login.
[95195.714991] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x5f/0x6c, rejecting login.
[95225.681112] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x84/0xed, rejecting login.
[95225.681288] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0xc7/0xb1, rejecting login.
[95225.681761] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x88/0xec, rejecting login.
[95225.682141] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x01/0x48, rejecting login.
[95373.532874] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x72/0x6f, rejecting login.
############
Jumboframes are enabled on target's 4 NICs, VMWare's vSwitch & VMWare's portgroups.
Supporting information:
/> ls
o- / ......................................................................................................................... [...]
o- backstores .............................................................................................................. [...]
| o- fileio ................................................................................................... [1 Storage Object]
| | o- null ............................................................................................ [/tmp/nullfile activated]
| o- iblock .................................................................................................. [2 Storage Objects]
| | o- vmware1 ................................................................ [/dev/disk/by-id/scsi-2001b4d2058524594 activated]
| | o- vmware2 ................................................................ [/dev/disk/by-id/scsi-2001b4d2065331987 activated]
| o- pscsi .................................................................................................... [0 Storage Object]
| o- rd_dr .................................................................................................... [0 Storage Object]
| o- rd_mcp ................................................................................................... [0 Storage Object]
o- iscsi .............................................................................................................. [1 Target]
| o- iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware ................................................................... [1 TPG]
| o- tpgt1 ........................................................................................................... [enabled]
| o- acls ............................................................................................................ [1 ACL]
| | o- iqn.2013-02.fi.splab.storage.iscsi.vmware:annie ....................................................... [3 Mapped LUNs]
| | o- mapped_lun0 ............................................................................................. [lun0 (rw)]
| | o- mapped_lun1 ............................................................................................. [lun1 (rw)]
| | o- mapped_lun2 ............................................................................................. [lun2 (rw)]
| o- luns ........................................................................................................... [3 LUNs]
| | o- lun0 .................................................................................... [fileio/null (/tmp/nullfile)]
| | o- lun1 ........................................................ [iblock/vmware1 (/dev/disk/by-id/scsi-2001b4d2058524594)]
| | o- lun2 ........................................................ [iblock/vmware2 (/dev/disk/by-id/scsi-2001b4d2065331987)]
| o- portals ..................................................................................................... [4 Portals]
| o- 10.255.10.5:3260 ................................................................................................. [OK]
| o- 10.255.10.6:3260 ................................................................................................. [OK]
| o- 10.255.10.7:3260 ................................................................................................. [OK]
| o- 10.255.10.8:3260 ................................................................................................. [OK]
o- loopback ........................................................................................................... [0 Target]
o- tcm_fc ............................................................................................................. [0 Target]
root@tibbers:~# dpkg -l |grep targetcli
ii targetcli 2.0rc1-2 all administration tool for managing LIO core target
root@tibbers:~# cat /etc/issue
Debian GNU/Linux 7.0 \n \l
Regards,
-Eljas Alakulppi
--
To unsubscribe from this list: send the line "unsubscribe target-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
