On 10/7/21 3:40 PM, Shoaib Rao wrote: > > On 10/7/21 12:57 PM, Jason Gunthorpe wrote: >> On Thu, Oct 07, 2021 at 02:51:11PM -0500, Bob Pearson wrote: >>> On 10/7/21 2:05 PM, Jason Gunthorpe wrote: >>>> On Thu, Oct 07, 2021 at 01:53:27PM -0500, Bob Pearson wrote: >>>> >>>>> On looking, Rao's patch is not in for-next. Last one was >>>>> January. Which branch are you looking at? >>>> Oh, it is still in the wip branch, try now >>>> >>>> Jason >>>> >>> I see the issue. Rao is asking for 2^20 objects max by default which will >>> require 128KiB of memory in the index reservation bit mask for each of them. >>> There are 4 indexed objects QP by qpn, SRQ by srqn, MR by rkey and MW by rkey. >>> That's 512KiB of memory which seems excessive to me for many use cases where the >>> number of objects is fairly small. >>> >>> The bit mask is used to allocate and free the indices and there is also a red black >>> tree that is used to look up objects by their index (or key if they use keys instead.) >>> >>> If there is a usual way to address these kinds of issues in Linux maybe we should >>> consider that. >> Use an allocating xarray >> >> But for these AV patches just fix the merge conflict to something sane >> and go ahead >> >> Jason > > I did not want to increase the values too high but we discussed it so I did. Let me know if I need to modify the patch and reduce the values. > > Shoaib > If we convert the rxe_pools to use xarrays as Jason suggests it looks like this issue goes away. I'm looking at that. Bob
