Following the move of the TREE_RCU batching kvfree_rcu() implementation
to slab, we still have the simple non-batching implementation in tiny
RCU, and RCU implementation specific ifdefs in slab code.
Finish the move and integration into slab. Allow using the simple
call_rcu() based implementation also with tree RCU when SLUB_TINY is
enabled, as its goal is also to limit memory footprint with less concern
for top performance.
In order to avoid RCU having to recognize the fake callback function
pointers (__is_kvfree_rcu_offset()) when handling call_rcu(), implement
a callback that can calculate the object's address from the embedded
rcu_head pointer without knowing the specific offset (previously SLOB
would not have made it possible, but it's gone now).
After this series, AFAIK only the following kvfree_rcu specific code
remains in RCU:
- a call to kfree_rcu_scheduler_running() from rcu_set_runtime_mode()
- probably necessary and a generic registration interface would be
unnecessary bloat?
- declarations of kfree_rcu() API in include/linux/rcupdate.h
- could be moved to slab.h after checking for/fixing up potential
missing includes
git tree:
https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=b4/slub-tiny-kfree_rcu
To: Christoph Lameter <cl@xxxxxxxxx>
To: David Rientjes <rientjes@xxxxxxxxxx>
Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
Cc: Roman Gushchin <roman.gushchin@xxxxxxxxx>
Cc: Hyeonggon Yoo <42.hyeyoo@xxxxxxxxx>
Cc: linux-mm@xxxxxxxxx
To: "Paul E. McKenney" <paulmck@xxxxxxxxxx>
To: Joel Fernandes <joel@xxxxxxxxxxxxxxxxx>
To: Josh Triplett <josh@xxxxxxxxxxxxxxxx>
To: Boqun Feng <boqun.feng@xxxxxxxxx>
To: Uladzislau Rezki <urezki@xxxxxxxxx>
Cc: Steven Rostedt <rostedt@xxxxxxxxxxx>
Cc: Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx>
Cc: Lai Jiangshan <jiangshanlai@xxxxxxxxx>
Cc: Zqiang <qiang.zhang1211@xxxxxxxxx>
Cc: rcu@xxxxxxxxxxxxxxx
Signed-off-by: Vlastimil Babka <vbabka@xxxxxxx>
---
Vlastimil Babka (4):
slab, rcu: move TINY_RCU variant of kvfree_rcu() to SLAB
rcu: remove trace_rcu_kvfree_callback
rcu, slab: use a regular callback function for kvfree_rcu
slab: don't batch kvfree_rcu() with SLUB_TINY
include/linux/rcupdate.h | 29 ++++++++++++++---------------
include/linux/rcutiny.h | 36 ------------------------------------
include/linux/rcutree.h | 3 ---
include/linux/slab.h | 14 ++++++++++++++
include/trace/events/rcu.h | 34 ----------------------------------
kernel/rcu/tiny.c | 24 ------------------------
kernel/rcu/tree.c | 9 ++-------
mm/Kconfig | 4 ++++
mm/slab.h | 2 ++
mm/slab_common.c | 32 +++++++++++++++++++++++++-------
mm/slub.c | 42 ++++++++++++++++++++++++++++++++++++++++++
11 files changed, 103 insertions(+), 126 deletions(-)
---
base-commit: e492fac3657b60b8dd78a6e8ca26d1d14706c7b3
change-id: 20250123-slub-tiny-kfree_rcu-bd65bfe222f2
Best regards,
--
Vlastimil Babka <vbabka@xxxxxxx>
