On Tue, Jul 21, 2020 at 01:41:26PM -0400, Johannes Weiner wrote: > On Tue, Jul 21, 2020 at 11:19:52AM +0000, jingrui wrote: > > Cc: Johannes Weiner <hannes@xxxxxxxxxxx> ; Michal Hocko <mhocko@xxxxxxxxxx>; Vladimir Davydov <vdavydov.dev@xxxxxxxxx> > > > > Thanks. > > > > --- > > PROBLEM: cgroup cost too much memory when transfer small files to tmpfs. > > > > keywords: cgroup PERCPU/memory cost too much. > > > > description: > > > > We send small files from node-A to node-B tmpfs /tmp directory using sftp. On > > node-B the systemd configured with pam on like below. > > > > cat /etc/pam.d/password-auth | grep systemd > > -session optional pam_systemd.so > > > > So when transfer a file, a systemd session is created, that means a cgroup is > > created, then file saved at /tmp will associated with a cgroup object. After > > file transferred, session and cgroup-dir will be removed, but the file in /tmp > > still associated with the cgroup object. The PERCPU memory in cgroup/css object > > cost a lot(about 0.5MB/per-cgroup-object) on 200/cpus machine. > > CC Roman who had a patch series to free all this extended (percpu) > memory upon cgroup deletion: > > https://lore.kernel.org/patchwork/cover/1050508/ > > It looks like it never got merged for some reason. The mentioned patchset can make the problem less noticeable, but can't solve it completely. It has never been merged, because the dying cgroup problem was mostly solved by other methods: slab memory reparenting and various reclaim fixes. So there was no more reason to complicate the code to release the memcg memory early. The overhead of creating and destroying a new memory cgroup for a transfer of a small file will be noticeable anyway. So IMO the solution is to use a single cgroup for all transfers. I don't know if systemd supports such mode out of the box, but it shouldn't be hard to add it. But also I wonder if we need a special tmpfs mount option, something like "noaccount". Not only for this specific case, but also for the case when tmpfs is extensively shared between multiple cgroups or if it's used to pass some data from one cgroup to another, or if we care about the performance more than about the accounting; in other words for cases where the accounting makes more harm than good. Thanks!
