On Tue, Oct 10, 2023 at 11:20:00AM -0700, Andrew Morton wrote: > > The patch titled > Subject: mm: kmem: add direct objcg pointer to task_struct > has been added to the -mm mm-unstable branch. Its filename is > mm-kmem-add-direct-objcg-pointer-to-task_struct.patch > > This patch will shortly appear at > https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-kmem-add-direct-objcg-pointer-to-task_struct.patch > > This patch will later appear in the mm-unstable branch at > git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm > > Before you just go and hit "reply", please: > a) Consider who else should be cc'ed > b) Prefer to cc a suitable mailing list as well > c) Ideally: find the original patch on the mailing list and do a > reply-to-all to that, adding suitable additional cc's > > *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** > > The -mm tree is included into linux-next via the mm-everything > branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm > and is updated there every 2-3 working days > > ------------------------------------------------------ > From: Roman Gushchin <roman.gushchin@xxxxxxxxx> > Subject: mm: kmem: add direct objcg pointer to task_struct > Date: Mon, 9 Oct 2023 17:09:26 -0700 > > To charge a freshly allocated kernel object to a memory cgroup, the kernel > needs to obtain an objcg pointer. Currently it does it indirectly by > obtaining the memcg pointer first and then calling to > __get_obj_cgroup_from_memcg(). > > Usually tasks spend their entire life belonging to the same object cgroup. > So it makes sense to save the objcg pointer on task_struct directly, so > it can be obtained faster. It requires some work on fork, exit and cgroup > migrate paths, but these paths are way colder. > > To avoid any costly synchronization the following rules are applied: > 1) A task sets it's objcg pointer itself. > > 2) If a task is being migrated to another cgroup, the least > significant bit of the objcg pointer is set atomically. > > 3) On the allocation path the objcg pointer is obtained locklessly > using the READ_ONCE() macro and the least significant bit is > checked. If it's set, the following procedure is used to update > it locklessly: > - task->objcg is zeroed using cmpxcg > - new objcg pointer is obtained > - task->objcg is updated using try_cmpxchg > - operation is repeated if try_cmpxcg fails > It guarantees that no updates will be lost if task migration > is racing against objcg pointer update. It also allows to keep > both read and write paths fully lockless. > > Because the task is keeping a reference to the objcg, it can't go away > while the task is alive. > > This commit doesn't change the way the remote memcg charging works. > > Link: https://lkml.kernel.org/r/20231010000929.450702-3-roman.gushchin@xxxxxxxxx > Signed-off-by: Roman Gushchin (Cruise) <roman.gushchin@xxxxxxxxx> > Acked-by: Johannes Weiner <hannes@xxxxxxxxxxx> > Cc: David Rientjes <rientjes@xxxxxxxxxx> > Cc: Dennis Zhou <dennis@xxxxxxxxxx> > Cc: Michal Hocko <mhocko@xxxxxxxxxx> > Cc: Muchun Song <muchun.song@xxxxxxxxx> > Cc: Naresh Kamboju <naresh.kamboju@xxxxxxxxxx> > Cc: Shakeel Butt <shakeelb@xxxxxxxxxx> > Cc: Vlastimil Babka <vbabka@xxxxxxx> > Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> Andrew, can you, please, add Tested-by: Naresh Kamboju <naresh.kamboju@xxxxxxxxxx> here? Thanks!
