After more digging, I found out this out of no where hex string is
referred to as "initiator_ext", and is being sent since srp_daemon.sh
give srp_daemon the "-n" flag. Since there's no user-set
initiator_ext, srp_daemon grabs it from somewhere.
If I want to continue using the "-n" flag, how can I find out what my
initiator is going to send as initiator_ext, so the target can be set
up to work with it?
Or, can I set my own initiator_ext, it looks by appending it to
/etc/srp_daemon.conf ?
Why doesn't ibsrpdm give the initiator_ext value?
On Wed, Jan 6, 2016 at 3:21 AM, james harvey <jamespharvey20@xxxxxxxxx> wrote:
> TL;DR. I can find my initiator's guid in
> /sys/class/infiniband/mlx4_0/ports/1/gids/0, but using that on my
> target machine in its acls makes the target kernel reject the
> connection. Instead, giving the acls the i_port_id given in the
> kernel rejection message works. Why is the first half of i_port_id
> different than the guid, and where can I find it? (Other than
> attempting a connection, getting it rejected, and looking at the
> rejection message?)
>
> I'm at a loss on how to get the hex address to go underneath acls when
> setting up srp. If I use the fe80 prefix I see everywhere including
> /sys, it gets rejected. If I use the prefix shown in the kernel
> rejection method, which I can't find anywhere else, it works fine.
>
> I am using targetcli, but I don't think this is a targetcli issue,
> because it's the kernel showing in dmesg the bizzare prefix that I
> can't figure out where it's coming from.
>
> Was linux 4.2.5 (-1 Arch) on both machines. Target upgraded to 4.3.2
> (-2 Arch) with no change. Been acting like this for months, since I
> started with InfiniBand.
>
> Just upgraded from ConnectX to ConnectX-2 cards, and seeing the same behavior.
>
> targetcli configuration:
>
> /> ls
> o- / .........................................................................................................................
> [...]
> o- backstores
> ..............................................................................................................
> [...]
> | o- block ..................................................................................................
> [Storage Objects: 3]
> | | o- kvm1 ....................................................................
> [/dev/disk1/kvm1 (100.0GiB) write-thru activated]
> | | o- kvm2 ....................................................................
> [/dev/disk2/kvm2 (100.0GiB) write-thru activated]
> | | o- kvm3 ....................................................................
> [/dev/disk3/kvm3 (100.0GiB) write-thru activated]
> | o- fileio .................................................................................................
> [Storage Objects: 0]
> | o- pscsi ..................................................................................................
> [Storage Objects: 0]
> | o- ramdisk ................................................................................................
> [Storage Objects: 0]
> o- iscsi ............................................................................................................
> [Targets: 0]
> o- loopback .........................................................................................................
> [Targets: 0]
> o- sbp ..............................................................................................................
> [Targets: 0]
> o- srpt .............................................................................................................
> [Targets: 1]
> | o- ib.fe800000000000000002c90300001679
> ...........................................................................
> [no-gen-acls]
> | o- acls ............................................................................................................
> [ACLs: 1]
> | | o- ib.fe800000000000000002c90300001e79
> ....................................................................
> [Mapped LUNs: 3]
> | | o- mapped_lun0
> ....................................................................................
> [lun0 block/kvm1 (rw)]
> | | o- mapped_lun1
> ....................................................................................
> [lun1 block/kvm2 (rw)]
> | | o- mapped_lun2
> ....................................................................................
> [lun2 block/kvm3 (rw)]
> | o- luns ............................................................................................................
> [LUNs: 3]
> | o- lun0
> .....................................................................................
> [block/kvm1 (/dev/disk1/kvm1)]
> | o- lun1
> .....................................................................................
> [block/kvm2 (/dev/disk2/kvm2)]
> | o- lun2
> .....................................................................................
> [block/kvm3 (/dev/disk3/kvm3)]
> o- vhost ............................................................................................................
> [Targets: 0]
> o- xen_pvscsi
> .......................................................................................................
> [Targets: 0]
>
> NOTE my InfiniBand cards have similar IDs, and only differ by one
> character -- compare the end, 1679 vs 1e79.
>
> On the target (host):
> $ cat /sys/class/infiniband/mlx4_0/ports/1/gids/0
> fe80:0000:0000:0000:0002:c903:0000:1679
> {It's my understanding this should be used underneath srpt, as the wwn}
>
> On the initiator (client):
> $ cat /sys/class/infiniband/mlx4_0/ports/1/gids/0
> fe80:0000:0000:0000:0002:c903:0000:1e79
> {It's my understanding this should be used underneath acls, and mapped
> with the luns}
> # ibsrpdm -c
> id_ext=0002c90300001678,ioc_guid=0002c90300001678,dgid=fe800000000000000002c90300001679,pkey=ffff,service_id=0002c90300001678
> {so, its /etc/srp_daemon.conf is just comments and the line}
> a id_ext=0002c90300001678,ioc_guid=0002c90300001678,dgid=fe800000000000000002c90300001679,pkey=ffff,service_id=0002c90300001678
>
> But, after starting service target on the target, and service
> srpdaemon on the initiator, dmesg on target shows:
> [ 849.710278] ib_srpt Received SRP_LOGIN_REQ with i_port_id
> 0x7916000003c90200:0x2c90300001e79, t_port_id
> 0x2c90300001678:0x2c90300001678 and it_iu_len 260 on port 1
> (guid=0xfe80000000000000:0x2c90300001679)
> [ 849.714528] ib_srpt Session : kernel thread ib_srpt_compl (PID 24481) started
> [ 849.714601] ib_srpt Rejected login because no ACL has been
> configured yet for initiator 0x7916000003c902000002c90300001e79.
> [ 849.714613] ib_srpt Session 0x7916000003c902000002c90300001e79:
> kernel thread ib_srpt_compl (PID 24481) stopped
>
> And, dmesg on initiator shows:
> [ 976.616221] scsi host11: ib_srp: REJ received
> [ 976.616229] scsi host11: ib_srp: SRP LOGIN from
> fe80:0000:0000:0000:0002:c903:0000:1e79 to
> fe80:0000:0000:0000:0002:c903:0000:1679 REJECTED, reason 0x00010001
> [ 976.616269] scsi host11: ib_srp: Connection 0/4 failed
> [ 976.616276] scsi host11: ib_srp: Sending CM DREQ failed
>
> Why is the target machine, in the srp context only, seeing the ports
> as starting with 0x7916000003c90200 and 0x2c90300001678, and the
> client machine seeing the ports as starting with fe80:0000:0000:0000?
> What are these 0x7916... and 0x2c90... prefixes, and how can I find
> them besides getting a rejection and looking at the kernel logs? I
> haven't seen these prefixes anywhere else on either system.
>
> In targetcli, if I use acls with 0x7916000003c902000002c90300001e79,
> then everything works great.
--
To unsubscribe from this list: send the line "unsubscribe target-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
