> From: Saleem, Shiraz <shiraz.saleem@xxxxxxxxx> > Sent: Friday, March 26, 2021 1:40 AM > > Subject: RE: [PATCH v2 05/23] ice: Add devlink params support [..] > > > > Resources are better represented as devlink resource. > > Such as, > > > > $ devlink resource set pci/0000:06:00.0 /rdma/max_qps 16384 $ devlink > > resource set pci/0000:06:00.0 /rdma/max_cqs 8192 $ devlink resource > > set pci/0000:06:00.0 /rdma/max_mrs 16384 > > > > Hi Parav - Thank you for the feedback. > > Maybe I am missing something but I see that a devlink hot reload is required > to enforce the update? It isn't mandatory to reload, but yes either reload or driver unbind/bind is needed as you suggested below. > There isn't really a de-init required of PCI driver entities in this case for this > rdma param. > But only an unplug, plug of the auxdev with new value. Intuitively it feels > more runtime-ish. > Driver unbind/bind to reflect new limits is ok for cases where it is not time sensitive. For mlx5 use cases, user expects device to be provisioned in < few msecs. And driver unbind/bind are sub-optimal to achieve it from time and memory wise. So mlx5 driver prefers to stay away from driver unbind/bind steps. So I am working on series to not create class aux devices by default for SFs. Rather to create them on reload. Something like, $ devlink dev param set pci/0000:06:00.0 name pcisf_classes value false cmode driverinit $ devlink dev class set auxiliary/mlx5_core.sf.4 rdma true $ devlink resource set auxiliary/mlx5_core.sf.4 path rdma/max_qps size 200000 $ devlink dev reload auxiliary/mlx5_core.sf.4 This last command will create the mlx5_core.rdma.4 aux device and when its bound to driver, it will create IB device with right max_qp. So rdma device is created only once, instead of twice using unbind/bind sequence. This may not be possible for the PF/VF device due to backward compatibility and their different usage in system. > There is also a device powerof2 requirement on the maxqp which I don't see > enforceable as it stands. > Right, but similar to size_params.size_max, size_granularity, I believe size_params can be extended for alignment restriction. > This is not super-critical for the initial submission but a nice to have. But I do > want to brainstorm options.. > If max_qp is truly a dynamic value that doesn't require device recreation, extending existing rdma resource command seems more useful to end user than doing unbind/bind.
