On Sat, Dec 17, 2022 7:10 PM Li, Zhijian wrote:
>
>
>
> On 15/12/2022 18:14, Daisuke Matsuda wrote:
> > If you create MRs more than 0x10000 times after loading the module,
> > responder starts to reply NAKs for RDMA/Atomic operations because of rkey
> > violation detected in check_rkey(). The root cause is that rkeys are
> > incremented each time a new MR is created and the value overflows into the
> > range reserved for MWs.
> >
> > Fixes: 0994a1bcd5f7 ("RDMA/rxe: Bump up default maximum values used via uverbs")
> > Signed-off-by: Daisuke Matsuda <matsuda-daisuke@xxxxxxxxxxx>
> > ---
> > drivers/infiniband/sw/rxe/rxe_param.h | 8 ++++----
> > 1 file changed, 4 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/infiniband/sw/rxe/rxe_param.h b/drivers/infiniband/sw/rxe/rxe_param.h
> > index a754fc902e3d..a3d31bd45895 100644
> > --- a/drivers/infiniband/sw/rxe/rxe_param.h
> > +++ b/drivers/infiniband/sw/rxe/rxe_param.h
> > @@ -98,10 +98,10 @@ enum rxe_device_param {
> > RXE_MAX_SRQ = DEFAULT_MAX_VALUE - RXE_MIN_SRQ_INDEX,
> >
> > RXE_MIN_MR_INDEX = 0x00000001,
> > - RXE_MAX_MR_INDEX = DEFAULT_MAX_VALUE,
> > - RXE_MAX_MR = DEFAULT_MAX_VALUE - RXE_MIN_MR_INDEX,
> > - RXE_MIN_MW_INDEX = 0x00010001,
> > - RXE_MAX_MW_INDEX = 0x00020000,
> > + RXE_MAX_MR_INDEX = DEFAULT_MAX_VALUE >> 1,
> > + RXE_MAX_MR = 0x00001000,
>
> May i know why the RXE_MAX_MR isn't (RXE_MAX_MR_INDEX - RXE_MIN_MR_INDEX)
> 0x00001000 is much less than previous value
I just thought nobody will use that many MRs at the same time, so I made it
take after RXE_MAX_MW, but I see there was a reason to make this large.
Cf. https://lore.kernel.org/all/20210927191907.GA1582097@xxxxxxxxxx/
I shall change this and submit v2. Perhaps, I should also change RXE_MAX_MW.
Daisuke
>
>
>
>
> > + RXE_MIN_MW_INDEX = (DEFAULT_MAX_VALUE >> 1) + 1,
> > + RXE_MAX_MW_INDEX = DEFAULT_MAX_VALUE,
> > RXE_MAX_MW = 0x00001000,
> >
> > RXE_MAX_PKT_PER_ACK = 64,
