Re: [PATCH] RDMA/rxe: Bump up default maximum values used via uverbs

[Shoaib Rao <rao.shoaib@xxxxxxxxxx>]

On 7/13/21 1:40 AM, Zhu Yanjun wrote:
> On Tue, Jul 13, 2021 at 4:37 PM Rao Shoaib <Rao.Shoaib@xxxxxxxxxx> wrote:
> > From: Rao Shoaib <rao.shoaib@xxxxxxxxxx>
> >
> > In our internal testing we have found that the
> > current maximum are too smalls. Ideally there should
> > be no limits but currently maximum values are reported
> > via ibv_query_device, so we have to keep maximum values
> > but they have been made suffiently large.
> >
> > Signed-off-by: Rao Shoaib <rao.shoaib@xxxxxxxxxx>
> > ---
> >   drivers/infiniband/sw/rxe/rxe_param.h | 26 ++++++++++++++------------
> >   1 file changed, 14 insertions(+), 12 deletions(-)
> >
> > diff --git a/drivers/infiniband/sw/rxe/rxe_param.h b/drivers/infiniband/sw/rxe/rxe_param.h
> > index 742e6ec93686..66a948adb1e1 100644
> > --- a/drivers/infiniband/sw/rxe/rxe_param.h
> > +++ b/drivers/infiniband/sw/rxe/rxe_param.h
> > @@ -9,6 +9,8 @@
> >
> >   #include <uapi/rdma/rdma_user_rxe.h>
> >
> > +#define DEFAULT_MAX_VALUE (1 << 20)
> How do you get this DEFAULT_MAX_VALUE? From spec? or others?
>
> Zhu Yanjun

The spec does not specify any limits. The value was proposed as part of 
discussion on my previous patch.

Shoaib


> > +
> >   static inline enum ib_mtu rxe_mtu_int_to_enum(int mtu)
> >   {
> >          if (mtu < 256)
> > @@ -37,7 +39,7 @@ static inline enum ib_mtu eth_mtu_int_to_enum(int mtu)
> >   enum rxe_device_param {
> >          RXE_MAX_MR_SIZE                 = -1ull,
> >          RXE_PAGE_SIZE_CAP               = 0xfffff000,
> > -       RXE_MAX_QP_WR                   = 0x4000,
> > +       RXE_MAX_QP_WR                   = DEFAULT_MAX_VALUE,
> >          RXE_DEVICE_CAP_FLAGS            = IB_DEVICE_BAD_PKEY_CNTR
> >                                          | IB_DEVICE_BAD_QKEY_CNTR
> >                                          | IB_DEVICE_AUTO_PATH_MIG
> > @@ -58,40 +60,40 @@ enum rxe_device_param {
> >          RXE_MAX_INLINE_DATA             = RXE_MAX_WQE_SIZE -
> >                                            sizeof(struct rxe_send_wqe),
> >          RXE_MAX_SGE_RD                  = 32,
> > -       RXE_MAX_CQ                      = 16384,
> > +       RXE_MAX_CQ                      = DEFAULT_MAX_VALUE,
> >          RXE_MAX_LOG_CQE                 = 15,
> > -       RXE_MAX_PD                      = 0x7ffc,
> > +       RXE_MAX_PD                      = DEFAULT_MAX_VALUE,
> >          RXE_MAX_QP_RD_ATOM              = 128,
> >          RXE_MAX_RES_RD_ATOM             = 0x3f000,
> >          RXE_MAX_QP_INIT_RD_ATOM         = 128,
> >          RXE_MAX_MCAST_GRP               = 8192,
> >          RXE_MAX_MCAST_QP_ATTACH         = 56,
> >          RXE_MAX_TOT_MCAST_QP_ATTACH     = 0x70000,
> > -       RXE_MAX_AH                      = 100,
> > -       RXE_MAX_SRQ_WR                  = 0x4000,
> > +       RXE_MAX_AH                      = DEFAULT_MAX_VALUE,
> > +       RXE_MAX_SRQ_WR                  = DEFAULT_MAX_VALUE,
> >          RXE_MIN_SRQ_WR                  = 1,
> >          RXE_MAX_SRQ_SGE                 = 27,
> >          RXE_MIN_SRQ_SGE                 = 1,
> >          RXE_MAX_FMR_PAGE_LIST_LEN       = 512,
> > -       RXE_MAX_PKEYS                   = 1,
> > +       RXE_MAX_PKEYS                   = DEFAULT_MAX_VALUE,
> >          RXE_LOCAL_CA_ACK_DELAY          = 15,
> >
> > -       RXE_MAX_UCONTEXT                = 512,
> > +       RXE_MAX_UCONTEXT                = DEFAULT_MAX_VALUE,
> >
> >          RXE_NUM_PORT                    = 1,
> >
> > -       RXE_MAX_QP                      = 0x10000,
> > +       RXE_MAX_QP                      = DEFAULT_MAX_VALUE,
> >          RXE_MIN_QP_INDEX                = 16,
> > -       RXE_MAX_QP_INDEX                = 0x00020000,
> > +       RXE_MAX_QP_INDEX                = 0x00040000,
> >
> > -       RXE_MAX_SRQ                     = 0x00001000,
> > +       RXE_MAX_SRQ                     = DEFAULT_MAX_VALUE,
> >          RXE_MIN_SRQ_INDEX               = 0x00020001,
> >          RXE_MAX_SRQ_INDEX               = 0x00040000,
> >
> > -       RXE_MAX_MR                      = 0x00001000,
> > +       RXE_MAX_MR                      = DEFAULT_MAX_VALUE,
> >          RXE_MAX_MW                      = 0x00001000,
> >          RXE_MIN_MR_INDEX                = 0x00000001,
> > -       RXE_MAX_MR_INDEX                = 0x00010000,
> > +       RXE_MAX_MR_INDEX                = 0x00040000,
> >          RXE_MIN_MW_INDEX                = 0x00010001,
> >          RXE_MAX_MW_INDEX                = 0x00020000,
> >
> > --
> > 2.27.0