On Mon, Oct 16, 2023 at 3:19 PM Roman Gushchin <roman.gushchin@xxxxxxxxx> wrote: > > To charge a freshly allocated kernel object to a memory cgroup, the > kernel needs to obtain an objcg pointer. Currently it does it > indirectly by obtaining the memcg pointer first and then calling to > __get_obj_cgroup_from_memcg(). > > Usually tasks spend their entire life belonging to the same object > cgroup. So it makes sense to save the objcg pointer on task_struct > directly, so it can be obtained faster. It requires some work on fork, > exit and cgroup migrate paths, but these paths are way colder. > > To avoid any costly synchronization the following rules are applied: > 1) A task sets it's objcg pointer itself. > > 2) If a task is being migrated to another cgroup, the least > significant bit of the objcg pointer is set atomically. > > 3) On the allocation path the objcg pointer is obtained locklessly > using the READ_ONCE() macro and the least significant bit is > checked. If it's set, the following procedure is used to update > it locklessly: > - task->objcg is zeroed using cmpxcg > - new objcg pointer is obtained > - task->objcg is updated using try_cmpxchg > - operation is repeated if try_cmpxcg fails > It guarantees that no updates will be lost if task migration > is racing against objcg pointer update. It also allows to keep > both read and write paths fully lockless. > > Because the task is keeping a reference to the objcg, it can't go away > while the task is alive. > > This commit doesn't change the way the remote memcg charging works. > > Signed-off-by: Roman Gushchin (Cruise) <roman.gushchin@xxxxxxxxx> > Tested-by: Naresh Kamboju <naresh.kamboju@xxxxxxxxxx> > Acked-by: Johannes Weiner <hannes@xxxxxxxxxxx> Acked-by: Shakeel Butt <shakeelb@xxxxxxxxxx>
