Hi Eljas, On Wed, 2013-05-22 at 12:17 +0000, Eljas Alakulppi wrote: > Hello. > > I'm trying to get multipathing working between LIO target and VMWare > ESXi 5.1 U1. However, I'm not having a lot of luck as I'm not entirely > sure which side is the one not playing nice. From VMWare side, I can > see that it's using each interface, but it's only managing to connect > to single interface on the target. > > ############ > Runtime name Target LUN > vmhba37:C4:T0:L0 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 0 > vmhba37:C4:T0:L1 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 1 > vmhba37:C4:T0:L2 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 2 > vmhba37:C5:T0:L0 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 0 > vmhba37:C5:T0:L1 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 1 > vmhba37:C5:T0:L2 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 2 > vmhba37:C6:T0:L0 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 0 > vmhba37:C6:T0:L1 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 1 > vmhba37:C6:T0:L2 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 2 > vmhba37:C7:T0:L0 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 0 > vmhba37:C7:T0:L1 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 1 > vmhba37:C7:T0:L2 iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware:10.255.10.5:3260 2 > ############ > > As far as I'm aware, every interface is responding as they should > (only respond to ARP requests on their own IP & packet arriving on > interface X will be sent back by same interface). > ############ > ~ # esxcli network ip neighbor list #ESXi > Neighbor Mac Address Vmknic Expiry State > ------------------------- ----------------- ------ -------- --------- > 10.255.10.5 bc:ae:c5:1b:94:d7 vmk1 1020 sec > 10.255.10.6 00:1b:21:a5:1e:29 vmk1 1013 sec > 10.255.10.7 00:1b:21:a5:1e:28 vmk1 906 sec > 10.255.10.8 00:1b:21:a5:1e:2b vmk1 909 sec > ~ # esxcli network ip interface ipv4 get > Name IPv4 Address IPv4 Netmask IPv4 Broadcast Address Type DHCP DNS > ---- ------------ ------------- -------------- ------------ -------- > vmk1 10.255.10.9 255.255.255.0 10.255.10.255 STATIC false > vmk2 10.255.10.10 255.255.255.0 10.255.10.255 STATIC false > vmk3 10.255.10.11 255.255.255.0 10.255.10.255 STATIC false > vmk4 10.255.10.12 255.255.255.0 10.255.10.255 STATIC false > > root@tibbers:~# ip addr | grep 10.255.10. #Target > inet 10.255.10.6/24 brd 10.255.10.255 scope global eth2 > inet 10.255.10.7/24 brd 10.255.10.255 scope global eth3 > inet 10.255.10.5/24 brd 10.255.10.255 scope global eth1 > inet 10.255.10.8/24 brd 10.255.10.255 scope global eth4 > root@tibbers:~# tcpdump -i eth2 > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes > 10:32:39.705869 IP 10.255.10.9 > 10.255.10.6: ICMP echo request, id 32557, seq 0, length 64 > 10:32:39.706018 IP 10.255.10.6 > 10.255.10.9: ICMP echo reply, id 32557, seq 0, length 64 > 10:32:40.707893 IP 10.255.10.9 > 10.255.10.6: ICMP echo request, id 32557, seq 1, length 64 > 10:32:40.707905 IP 10.255.10.6 > 10.255.10.9: ICMP echo reply, id 32557, seq 1, length 64 > root@tibbers:~# tcpdump -i eth3 > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on eth3, link-type EN10MB (Ethernet), capture size 65535 bytes > 10:37:09.475447 IP 10.255.10.9 > 10.255.10.7: ICMP echo request, id 9006, seq 0, length 64 > 10:37:09.475651 IP 10.255.10.7 > 10.255.10.9: ICMP echo reply, id 9006, seq 0, length 64 > 10:37:10.477539 IP 10.255.10.9 > 10.255.10.7: ICMP echo request, id 9006, seq 1, length 64 > 10:37:10.477558 IP 10.255.10.7 > 10.255.10.9: ICMP echo reply, id 9006, seq 1, length 64 Btw, if your going to run multiple network portals on the same subnet, you'll need separate VLANs to isolate traffic amongst the ports, otherwise you'll end up seeing strange scenarios where outgoing traffic is getting sent on the wrong Ethernet port. > ############ > > When VMWare tries to connect to target, these show up in the target's > dmesg (seemingly randomly, sometimes it's just one type, sometimes > it's both. And Min/Max keep changing): > ############ > [95195.714641] First opcode is not login request, failing login request. > [95195.714694] iSCSI Login negotiation failed. > [95195.714991] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x5f/0x6c, rejecting login. > [95195.715111] iSCSI Login negotiation failed. > > [93950.152667] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x6e/0x75, rejecting login. > [94053.807933] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0xc0/0x01, rejecting login. > [95195.714991] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x5f/0x6c, rejecting login. > [95225.681112] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x84/0xed, rejecting login. > [95225.681288] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0xc7/0xb1, rejecting login. > [95225.681761] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x88/0xec, rejecting login. > [95225.682141] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x01/0x48, rejecting login. > [95373.532874] Unsupported iSCSI IETF Pre-RFC Revision, version Min/Max 0x72/0x6f, rejecting login. Mmm, this would seem to indicate that something is afoul below the application layer.. > ############ > > Jumboframes are enabled on target's 4 NICs, VMWare's vSwitch & VMWare's portgroups. > Can you verify if your able to reproduce without jumbo frames enabled..? > Supporting information: > > /> ls > o- / ......................................................................................................................... [...] > o- backstores .............................................................................................................. [...] > | o- fileio ................................................................................................... [1 Storage Object] > | | o- null ............................................................................................ [/tmp/nullfile activated] > | o- iblock .................................................................................................. [2 Storage Objects] > | | o- vmware1 ................................................................ [/dev/disk/by-id/scsi-2001b4d2058524594 activated] > | | o- vmware2 ................................................................ [/dev/disk/by-id/scsi-2001b4d2065331987 activated] > | o- pscsi .................................................................................................... [0 Storage Object] > | o- rd_dr .................................................................................................... [0 Storage Object] > | o- rd_mcp ................................................................................................... [0 Storage Object] > o- iscsi .............................................................................................................. [1 Target] > | o- iqn.2013-02.fi.splab.storage.iscsi.tibbers:vmware ................................................................... [1 TPG] > | o- tpgt1 ........................................................................................................... [enabled] > | o- acls ............................................................................................................ [1 ACL] > | | o- iqn.2013-02.fi.splab.storage.iscsi.vmware:annie ....................................................... [3 Mapped LUNs] > | | o- mapped_lun0 ............................................................................................. [lun0 (rw)] > | | o- mapped_lun1 ............................................................................................. [lun1 (rw)] > | | o- mapped_lun2 ............................................................................................. [lun2 (rw)] > | o- luns ........................................................................................................... [3 LUNs] > | | o- lun0 .................................................................................... [fileio/null (/tmp/nullfile)] > | | o- lun1 ........................................................ [iblock/vmware1 (/dev/disk/by-id/scsi-2001b4d2058524594)] > | | o- lun2 ........................................................ [iblock/vmware2 (/dev/disk/by-id/scsi-2001b4d2065331987)] > | o- portals ..................................................................................................... [4 Portals] > | o- 10.255.10.5:3260 ................................................................................................. [OK] > | o- 10.255.10.6:3260 ................................................................................................. [OK] > | o- 10.255.10.7:3260 ................................................................................................. [OK] > | o- 10.255.10.8:3260 ................................................................................................. [OK] > o- loopback ........................................................................................................... [0 Target] > o- tcm_fc ............................................................................................................. [0 Target] > This looks sane.. --nab -- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
