On Wed, May 3, 2023 at 3:15 PM Chris Li <chrisl@xxxxxxxxxx> wrote: [...] > I am also interested in this topic. T.J. and I have some offline > discussion about this. We have some proposals to solve this > problem. > > I will share the write up here for the up coming LSF/MM discussion. > > > Shared Memory Cgroup Controllers > > = Introduction > > The current memory cgroup controller does not support shared memory objects. For the memory that is shared between different processes, it is not obvious which process should get charged. Google has some internal tmpfs “memcg=” mount option to charge tmpfs data to a specific memcg that’s often different from where charging processes run. However it faces some difficulties when the charged memcg exits and the charged memcg becomes a zombie memcg. What is the exact problem this proposal is solving? Is it the zombie memcgs? To me that is just a side effect of memory shared between different memcgs. > Other approaches include “re-parenting” the memcg charge to the parent memcg. Which has its own problem. If the charge is huge, iteration of the reparenting can be costly. What is the iteration of the reparenting? Are you referring to reparenting the LRUs or something else? > > = Proposed Solution > > The proposed solution is to add a new type of memory controller for shared memory usage. E.g. tmpfs, hugetlb, file system mmap and dma_buf. This shared memory cgroup controller object will have the same life cycle of the underlying shared memory. I am confused by the relationship between shared memory controller and the underlying shared memory. What does the same life cycle mean? Are the users expected to register the shared memory objects with the smemcg? What about unnamed shared memory objects like MAP_SHARED or memfds? How does the charging work for smemcg? Is this new controller hierarchical? > > Processes can not be added to the shared memory cgroup. Instead the shared memory cgroup can be added to the memcg using a “smemcg” API file, similar to adding a process into the “tasks” API file. Is the charge of the underlying shared memory live with smemcg or the memcg where smemcg is attached? Can a smemcg detach and reattach to a different memcg? > When a smemcg is added to the memcg, the amount of memory that has been shared in the memcg process will be accounted for as the part of the memcg “memory.current”.The memory.current of the memcg is make up of two parts, 1) the processes anonymous memory and 2) the memory shared from smemcg. The above is somewhat giving the impression that the charge of shared memory lives with smemcg. This can mess up or complicate the hierarchical property of the original memcg. > > When the memcg “memory.current” is raised to the limit. The kernel will active try to reclaim for the memcg to make “smemcg memory + process anonymous memory” within the limit. Further memory allocation within those memcg processes will fail if the limit can not be followed. If many reclaim attempts fail to bring the memcg “memory.current” within the limit, the process in this memcg will get OOM killed. The OOM killing for remote charging needs much more thought. Please see https://lwn.net/Articles/787626/ for previous discussion on related topic.
