On Sun, Jun 24, 2018 at 09:44:12PM +0300, Sagi Grimberg wrote:
>
>
> On 06/22/2018 04:45 AM, Vijay Immanuel wrote:
> >Select the source udp port number for a QP based on a hash
> >of the source and destination QPNs. This provides a better
> >spread of traffic across NIC RX queues.
> >
> >Signed-off-by: Vijay Immanuel <vijayi@xxxxxxxxxxxxxxxxx>
> >---
> >Changes in v2:
> > - Moved setting the src_port to where the QP state is set to RTR.
> >
> > drivers/infiniband/sw/rxe/rxe_net.c | 6 ++----
> > drivers/infiniband/sw/rxe/rxe_qp.c | 4 ++++
> > drivers/infiniband/sw/rxe/rxe_verbs.h | 1 +
> > 3 files changed, 7 insertions(+), 4 deletions(-)
> >
> >diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c
> >index 9da6e37..04e5097 100644
> >--- a/drivers/infiniband/sw/rxe/rxe_net.c
> >+++ b/drivers/infiniband/sw/rxe/rxe_net.c
> >@@ -409,8 +409,7 @@ static int prepare4(struct rxe_dev *rxe, struct rxe_pkt_info *pkt,
> > if (!memcmp(saddr, daddr, sizeof(*daddr)))
> > pkt->mask |= RXE_LOOPBACK_MASK;
> >- prepare_udp_hdr(skb, htons(RXE_ROCE_V2_SPORT),
> >- htons(ROCE_V2_UDP_DPORT));
> >+ prepare_udp_hdr(skb, htons(qp->src_port), htons(ROCE_V2_UDP_DPORT));
>
> Better to use cpu_to_be16 instead htons.
Ok.
>
> Hi Vijay,
>
> > case IB_QPS_RTR:
> > pr_debug("qp#%d state -> RTR\n", qp_num(qp));
> > + qp->src_port = RXE_ROCE_V2_SPORT +
> > + (hash_64_generic
> > + (((u64)qp->attr.dest_qp_num << 24) +
> > + (u64)qp_num(qp), 14) & 0x3fff);
> maybe make it a little helper with some documentation for the motivation?
>
> Also,
>
> > prepare_ipv4_hdr(dst, skb, saddr->s_addr, daddr->s_addr, IPPROTO_UDP,
> > av->grh.traffic_class, av->grh.hop_limit, df, xnet);
> >@@ -440,8 +439,7 @@ static int prepare6(struct rxe_dev *rxe, struct rxe_pkt_info *pkt,
> > if (!memcmp(saddr, daddr, sizeof(*daddr)))
> > pkt->mask |= RXE_LOOPBACK_MASK;
> >- prepare_udp_hdr(skb, htons(RXE_ROCE_V2_SPORT),
> >- htons(ROCE_V2_UDP_DPORT));
> >+ prepare_udp_hdr(skb, htons(qp->src_port), htons(ROCE_V2_UDP_DPORT));
> > prepare_ipv6_hdr(dst, skb, saddr, daddr, IPPROTO_UDP,
> > av->grh.traffic_class,
> >diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c
> >index b9f7aa1..174c811 100644
> >--- a/drivers/infiniband/sw/rxe/rxe_qp.c
> >+++ b/drivers/infiniband/sw/rxe/rxe_qp.c
> >@@ -726,6 +726,10 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask,
> > case IB_QPS_RTR:
> > pr_debug("qp#%d state -> RTR\n", qp_num(qp));
> >+ qp->src_port = RXE_ROCE_V2_SPORT +
> >+ (hash_64_generic
> >+ (((u64)qp->attr.dest_qp_num << 24) +
> >+ (u64)qp_num(qp), 14) & 0x3fff);
>
> Is 64bit hash really needed if you are taking only 14 bits? (why not
> 16 btw?)
Hi Sagi,
The hash is over the 24-bit source and dest QPNs. Checking with Moni on this.
We only take 14-bits because the top 2-bits are always 1. RXE_ROCE_V2_SPORT is 0xc000
which is the start of the ephemeral port range.
>
> maybe make it a little helper with some documentation for the motivation?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
