On Wed, Jul 18, 2018 at 10:58 AM Bruce Merry <bmerry@xxxxxxxxx> wrote: > > On 18 July 2018 at 19:48, Shakeel Butt <shakeelb@xxxxxxxxxx> wrote: > > On Wed, Jul 18, 2018 at 10:40 AM Bruce Merry <bmerry@xxxxxxxxx> wrote: > >> > Yes, very easy to produce zombies, though I don't think kernel > >> > provides any way to tell how many zombies exist on the system. > >> > > >> > To create a zombie, first create a memcg node, enter that memcg, > >> > create a tmpfs file of few KiBs, exit the memcg and rmdir the memcg. > >> > That memcg will be a zombie until you delete that tmpfs file. > >> > >> Thanks, that makes sense. I'll see if I can reproduce the issue. Do > >> you expect the same thing to happen with normal (non-tmpfs) files that > >> are sitting in the page cache, and/or dentries? > >> > > > > Normal files and their dentries can get reclaimed while tmpfs will > > stick and even if the data of tmpfs goes to swap, the kmem related to > > tmpfs files will remain in memory. > > Sure, page cache and dentries are reclaimable given memory pressure. > These machines all have more memory than they need though (64GB+) and > generally don't come under any memory pressure. I'm just wondering if > the behaviour we're seeing can be explained as a result of a lot of > dentries sticking around (because there is no memory pressure) and in > turn causing a lot of zombie cgroups to stay present until something > forces reclamation of dentries. > Yes, if there is no memory pressure such memory can stay around. On your production machine, before deleting memory containers, you can try force_empty to reclaim such memory from them. See if that helps. Shakeel
