cm_process_routed_req() does not resonate well with RoCE systems

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



With the introduction of RoCE systems, a CM REQ message will contain (pasted from Wireshark):

Primary Hop Limit: 0x40
.... 0... = Primary Subnet Local: 0x0

This because cma_resolve_iboe_route() has:

        if (((struct sockaddr *)&id_priv->id.route.addr.dst_addr)->sa_family != AF_IB)
                /* TODO: get the hoplimit from the inet/inet6 device */
                route->path_rec->hop_limit = addr->dev_addr.hoplimit;
        else
                route->path_rec->hop_limit = 1;

The addr->dev_addr.hoplimit is coming from addr4_resolve(), which has:

	addr->hoplimit = ip4_dst_hoplimit(&rt->dst);

ip4_dst_hoplimit() returns the value of the sysctl net.ipv4.ip_default_ttl in this case (64).

For the purpose of this case, consider the CM REQ to have the Primary SL != 0.

When this REQ gets processed by cm_req_handler(), the cm_process_routed_req() function is called.

Since the Primary Subnet Local value is zero in the request, and since this is RoCE (Primary Local LID is permissive), the following statement will be executed:

	IBA_SET(CM_REQ_PRIMARY_SL, req_msg, wc->sl);

At least on the system I ran on, which was equipped with a Mellanox CX-5 HCA, the wc->sl is zero. Hence, the request to setup a connection using an SL != zero, will not be honoured, and a connection using SL zero will be created instead.

As a side note, in cm_process_routed_req(), we have:

	IBA_SET(CM_REQ_PRIMARY_REMOTE_PORT_LID, req_msg, wc->dlid_path_bits);

which is strange, since a LID is 16 bits, whereas dlid_path_bits is only eight.

I am uncertain about the correct fix here. Any input appreciated.



Thxs, Håkon
















[Index of Archives]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Photo]     [Yosemite News]     [Yosemite Photos]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux