> From: Parav Pandit [mailto:pandit.parav@xxxxxxxxx] > > On Fri, Nov 4, 2016 at 10:22 AM, Liran Liss <liranl@xxxxxxxxxxxx> wrote: > >> From: Parav Pandit [mailto:pandit.parav@xxxxxxxxx] > > > >> > > >> > A global HCA metric is indeed in the right direction. > >> > However, rethinking this, I think that we should specify the metric > >> > in terms of > >> RDMA objects rather than percentage. > >> > Basically, any resource that consumes an IDR is charged. > >> > > >> If metric definition is based on RDMA objects (count) and not based > >> on percentage, how would user specify the metric without really > >> specifying object type. > >> Current patch defines the metric as absolute numbers and objects as well. > >> > > > > That is the requested change. The absolute number would account for any > object allocation. We won't distinguish between types. > > Only a single counter (per device). > > > > In that case ucontext deserve a additional count. Because that is handful in > range of 256 to 1K. > If we give absolute consolidated number as 2000, one container will allocate all > the doorbell uctx and no other container can run. > Percentage works for this particular case. > Hmm.. I guess that you are right. So we can add another count for "HCA handles", or alternatively, each provider will restrict the number of handles per device to a reasonable small number (which won't be treated as one of the "HCA resources"). Typically, a process shouldn't need to open more than a single handle... > > >> Comment from Leon about his discussion with Matan, Tejun, Christoph > >> says opposite of this for user level configuration. > >> May be I am missing something. > >> > >> > The reasons are: > >> > - Some HCAs can have a huge amount of resources (millions of > >> > objects), of > >> which even a small percentage may consume a considerable amount of > >> kernel memory. > >> > - We follow the same notion as FD limits, which accounts for > >> > numerous resource types that consume file objects in the kernel > >> > (files, pipes, > >> > sockets) > >> > - The namespaces for RDMA resources are large (usually 24 bits). So > >> > even large resource counts won't come nowhere close in depleting > >> > the namespace. (Compare that to the mere 64K socket port space...) > >> > - The metric measures the actual application usage of resources, > >> > rather than > >> proportional to the resources of a given HCA adapter. > >> > - We can continue to use the cgroup mechanism for charging (just as > >> > in the original proposal) > >> > > >> > I have discussed this matter with Doug and Matan, and it seems like > >> > this is the > >> right direction. > >> > --Liran > >> > ��.n��������+%������w��{.n������.����n�r������&��z�ޗ�zf���h���~����������_��+v���)ߣ�
