On Tue, Jul 28, 2020 at 11:46:36PM +0800, Zhu Yanjun wrote: > On 7/28/2020 9:44 PM, Kamal Heib wrote: > > On Tue, Jul 28, 2020 at 09:21:06PM +0800, Zhu Yanjun wrote: > > > On 7/28/2020 4:35 PM, Kamal Heib wrote: > > > > On Thu, Jul 23, 2020 at 11:15:00PM +0800, Zhu Yanjun wrote: > > > > > On 7/23/2020 9:15 PM, Jason Gunthorpe wrote: > > > > > > On Thu, Jul 23, 2020 at 09:08:39PM +0800, Zhu Yanjun wrote: > > > > > > > On 7/23/2020 3:25 PM, Kamal Heib wrote: > > > > > > > > On Thu, Jul 23, 2020 at 02:58:41PM +0800, Zhu Yanjun wrote: > > > > > > > > > On 7/23/2020 1:57 PM, Kamal Heib wrote: > > > > > > > > > > On Wed, Jul 22, 2020 at 10:09:04AM +0800, Zhu Yanjun wrote: > > > > > > > > > > > On Tue, Jul 21, 2020 at 7:28 PM Yanjun Zhu <yanjunz@xxxxxxxxxxxx> wrote: > > > > > > > > > > > > From: Kamal Heib <kamalheib1@xxxxxxxxx> > > > > > > > > > > > > Sent: Tuesday, July 21, 2020 6:16 PM > > > > > > > > > > > > To: linux-rdma@xxxxxxxxxxxxxxx > > > > > > > > > > > > Cc: Yanjun Zhu <yanjunz@xxxxxxxxxxxx>; Doug Ledford <dledford@xxxxxxxxxx>; Jason Gunthorpe <jgg@xxxxxxxx>; Kamal Heib <kamalheib1@xxxxxxxxx> > > > > > > > > > > > > Subject: [PATCH for-next] RDMA/rxe: Remove pkey table > > > > > > > > > > > > > > > > > > > > > > > > The RoCE spec require from RoCE devices to support only the defualt pkey, While the rxe driver maintain a 64 enties pkey table and use only the first entry. With that said remove the maintaing of the pkey table and used the default pkey when needed. > > > > > > > > > > > > > > > > > > > > > > > Hi Kamal > > > > > > > > > > > > > > > > > > > > > > After this patch is applied, do you make tests with SoftRoCE and mlx hardware? > > > > > > > > > > > > > > > > > > > > > > The SoftRoCE should work well with the mlx hardware. > > > > > > > > > > > > > > > > > > > > > > Zhu Yanjun > > > > > > > > > > > > > > > > > > > > > Hi Zhu, > > > > > > > > > > > > > > > > > > > > Yes, please see below: > > > > > > > > > > > > > > > > > > > > $ ibv_rc_pingpong -d mlx5_0 -g 11 > > > > > > > > > > local address: LID 0x0000, QPN 0x0000e3, PSN 0x728a4f, GID ::ffff:172.31.40.121 > > > > > > > > > Can you make tests with GSI QP? > > > > > > > > > > > > > > > > > > Zhu Yanjun > > > > > > > > > > > > > > > > Is this the GSI ? > > > > > > > > > > > > > > Please check GSI in "InfiniBandTM Architecture Specification Volume 1 > > > > > > > Release 1.3" > > > > > > > > > > > > > > Then make tests with GSI again. > > > > > The followings are also removed by this commit. Not sure if it is good. > > > > > > > > > > " > > > > > > > > > > C9-42: If the destination QP is QP1, the BTH:P_Key shall be compared to the > > > > > set of P_Keys associated with the port on which the packet arrived. If the > > > > > P_Key matches any of the keys associated with the port, it shall be > > > > > considered valid. > > > > > > > > > > " > > > > > > > > > The above is correct for ports that configured to work in InfiniBand > > > > mode, while in RoCEv2 mode only the default P_Key should be associated > > > > with the port (Please see below from "ANNEX A17: ROCEV2 (IP ROUTABLE > > > > ROCE)): > > > > > > > > """ > > > > 17.7.1 LOADING THE P_KEY TABLE > > > > > > > > Compliance statement C17-7: on page 1193 describes requirements for > > > > setting the P_Key table based on an assumption that the P_Key table is > > > > set directly by a Subnet Manager. However, RoCEv2 ports do not support > > > > InfiniBand Subnet Management. Therefore, compliance statement C17-7: > > > > on page 1193 does not apply to RoCEv2 ports. > > > " > > > > > > C17-7: An HCA shall require no OS involvement to set the P_Key table; > > > > > > the P_Key table shall be set directly by Subnet Manager MADs. > > > > > > " > > > > > > In SoftRoCE, what set the P_Key table? > > > > > No one is setting the P_Key table in SoftRoCE, and no subnet manager in > > the RoCE fabric. > > > > Could you please tell me what is wrong with this patch? > > Please read the mail thread again. > > GSI QP number is 1. In your commits, the handle of qpn == 1 is removed. > > It seems that it conflicts with IB specification. > > Not sure if it is good. > Could you please read my patch again and point to what do you think is wrong? What I did in this patch is to verify that the pkey value in the received packet is the default P_Key regardless of the qpn, because RoCE devices should maintain only the default P_Key. Thanks, Kamal > > > > Thanks, > > Kamal > > > > > > Methods for setting the P_Key table associated with a RoCEv2 port are > > > > not defined in this specification, except for the requirements for a > > > > default P_Key described elsewhere in this annex. > > > > """ > > > > > > > > Thanks, > > > > Kamal > > > > > > > > > > > > > > rping uses RDMA CM which goes over the GSI > > > > > > > > > > > > Jason > >
