On Thu, Apr 04, 2024 at 07:51:03PM +0300, Leon Romanovsky wrote: > On Thu, Apr 04, 2024 at 11:01:13AM -0300, Jason Gunthorpe wrote: > > On Tue, Apr 02, 2024 at 12:50:21PM +0300, Leon Romanovsky wrote: > > > From: Michael Guralnik <michaelgur@xxxxxxxxxx> > > > > > > ib_umad is keeping the received MAD packets in a list that is not > > > limited in size. As the extraction of packets from this list is done > > > from user-space application, there is no way to guarantee the extraction > > > rate to be faster than the rate of incoming packets. This can cause to > > > the list to grow uncontrollably. > > > > > > As a solution, let's add new ysfs control knob for the users to limit > > > the number of received MAD packets in the list. > > > > > > Packets received when the list is full would be dropped. Sent packets > > > that are queued on the receive list for whatever reason, like timed out > > > sends, are not dropped even when the list is full. > > > > > > Signed-off-by: Michael Guralnik <michaelgur@xxxxxxxxxx> > > > Signed-off-by: Leon Romanovsky <leonro@xxxxxxxxxx> > > > --- > > > .../ABI/stable/sysfs-class-infiniband | 12 ++++ > > > drivers/infiniband/core/user_mad.c | 63 ++++++++++++++++++- > > > 2 files changed, 72 insertions(+), 3 deletions(-) > > > > > > diff --git a/Documentation/ABI/stable/sysfs-class-infiniband b/Documentation/ABI/stable/sysfs-class-infiniband > > > index 694f23a03a28..0ea9d590ab0e 100644 > > > --- a/Documentation/ABI/stable/sysfs-class-infiniband > > > +++ b/Documentation/ABI/stable/sysfs-class-infiniband > > > @@ -275,6 +275,18 @@ Description: > > > =============== =========================================== > > > > > > > > > +What: /sys/class/infiniband_mad/umad<N>/max_recv_list_size > > > +Date: January, 2024 > > > +KernelVersion: v6.9 > > > +Contact: linux-rdma@xxxxxxxxxxxxxxx > > > +Description: > > > + (RW) Limit the size of the list of MAD packets waiting to be > > > + read by the user-space agent. > > > + The default value is 0, which means unlimited list size. > > > + Packets received when the list is full will be silently > > > + dropped. > > > > I'm really not keen on this as a tunable, when we get to future > > designs it may be hard to retain this specific behavior. > > > > Why do we need a tunable? Can we just set it to something large and be > > done with it? > > I don't know which value to set to be large enough from one side and > small enough to do not cause to OOM while host gets MAD packets. I'd say something like 32M worth of packets is probably good from both directions? Or 1% of system memory, or something like that. Jason
