Hello, Parav. On Wed, Sep 09, 2015 at 09:27:40AM +0530, Parav Pandit wrote: > This is one old white paper, but most of the reasoning still holds true on RDMA. > http://h10032.www1.hp.com/ctg/Manual/c00257031.pdf Just read it. Much appreciated. ... > These resources include are- QP (queue pair) to transfer data, CQ > (Completion queue) to indicate completion of data transfer operation, > MR (memory region) to represent user application memory as source or > destination for data transfer. > Common resources are QP, SRQ (shared received queue), CQ, MR, AH > (Address handle), FLOW, PD (protection domain), user context etc. It's kinda bothering that all these are disparate resources. I suppose that each restriction comes from the underlying hardware and there's no accepted higher level abstraction for these things? > >> This patch-set allows limiting rdma resources to set of processes. > >> It extend device cgroup controller for limiting rdma device limits. > > > > I don't think this belongs to devcg. If these make sense as a set of > > resources to be controlled via cgroup, the right way prolly would be a > > separate controller. > > > > In past there has been similar comment to have dedicated cgroup > controller for RDMA instead of merging with device cgroup. > I am ok with both the approach, however I prefer to utilize device > controller instead of spinning of new controller for new devices > category. > I anticipate more such need would arise and for new device category, > it might not be worth to have new cgroup controller. > RapidIO though very less popular and upcoming PCIe are on horizon to > offer similar benefits as that of RDMA and in future having one > controller for each of them again would not be right approach. > > I certainly seek your and others inputs in this email thread here whether > (a) to continue to extend device cgroup (which support character, > block devices white list) and now RDMA devices > or > (b) to spin of new controller, if so what are the compelling reasons > that it can provide compare to extension. I'm doubtful that these things are gonna be mainstream w/o building up higher level abstractions on top and if we ever get there we won't be talking about MR or CQ or whatever. Also, whatever next-gen is unlikely to have enough commonalities when the proposed resource knobs are this low level, so let's please keep it separate, so that if/when this goes out of fashion for one reason or another, the controller can silently wither away too. > Current scope of the patch is limited to RDMA resources as first > patch, but for fact I am sure that there are more functionality in > pipe to support via this cgroup by me and others. > So keeping atleast these two aspects in mind, I need input on > direction of dedicated controller or new one. > > In future, I anticipate that we might have sub directory to device > cgroup for individual device class to control. > such as, > <sys/fs/cgroup/devices/ > /char > /block > /rdma > /pcie > /child_cgroup..1..N > Each controllers cgroup access files would remain within their own > scope. We are not there yet from base infrastructure but something to > be done as it matures and users start using it. I don't think that jives with the rest of cgroup and what generic block or pcie attributes are directly exposed to applications and need to be hierarchically controlled via cgroup? Thanks. -- tejun -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
