Hello! This series provides two variants of Tasks RCU, a rude variant inspired by Steven Rostedt's use of schedule_on_each_cpu(), and a tracing variant requested by the BPF folks and perhaps also of use for other tracing use cases. The tracing variant has explicit read-side markers to permit finite grace periods even given in-kernel loops in PREEMPT=n builds It also protects code in the idle loop, on exception entry/exit paths, and on the various CPU-hotplug online/offline code paths, thus having protection properties similar to SRCU. However, unlike SRCU, this variant avoids expensive instructions in the read-side primitives, thus having read-side overhead similar to that of preemptible RCU. There are of course downsides. The grace-period code can send IPIs to CPUs, even when those CPUs are in the idle loop or in nohz_full userspace. It is necessary to scan the full tasklist, much as for Tasks RCU. There is a single callback queue guarded by a single lock, again, much as for Tasks RCU. If needed, these downsides can be at least partially remedied. Perhaps most important, this variant of RCU does not affect the vanilla flavors, rcu_preempt and rcu_sched. The fact that RCU Tasks Trace readers can operate from idle, offline, and exception entry/exit in no way allows rcu_preempt and rcu_sched readers to also do so. This effort benefited greatly from off-list discussions of BPF requirements with Alexei Starovoitov and Andrii Nakryiko, as well as from numerous on-list discussions, at least some of which are captured in the "Link:" tags on the patches themselves. The patches in this series are as follows: 1. Add function to sample state of non-running function. I would guess that the API is still subject to change. ;-) 2. Use the above function to add per-task state to RCU CPU stall warnings. 3. Add rcutorture module parameter to produce non-busy-wait task stalls, thus allowing the above RCU CPU stall change to be exercised. 4. Move Tasks RCU to its own file. 5. Create struct to hold RCU-tasks state information. 6. Reinstate synchronize_rcu_mult(), as there will likely once again be a need to wait on multiple flavors of RCU. 7. Add an rcutorture test for synchronize_rcu_mult(). 8. Refactor RCU-tasks to allow variants to be added. 9. Add an RCU-tasks rude variant, based on Steven Rostedt's use of schedule_on_each_cpu(). 10. Add torture tests for RCU Tasks Rude. 11. Use unique names for RCU-Tasks kthreads and messages. 12. Further refactor RCU-tasks to allow adding even more variants. 13. Code movement to allow even more Tasks RCU variants. 14. Add an RCU Tasks Trace to simplify protection of tracing hooks, including BPF. 15. Add torture tests for RCU Tasks Trace. 16. Add stall warnings for RCU Tasks Trace. The new versions of Tasks RCU pass moderate rcutorture testing, and more severe testing is in the offing. They are not yet ready for production use, however! Thanx, Paul ------------------------------------------------------------------------ Documentation/admin-guide/kernel-parameters.txt | 5 include/linux/rcupdate.h | 9 include/linux/rcupdate_trace.h | 84 include/linux/rcupdate_wait.h | 19 include/linux/sched.h | 8 include/linux/wait.h | 2 init/init_task.c | 4 kernel/fork.c | 4 kernel/rcu/Kconfig | 34 kernel/rcu/Kconfig.debug | 4 kernel/rcu/rcu.h | 2 kernel/rcu/rcutorture.c | 96 kernel/rcu/tasks.h | 1730 +++++++++--- kernel/rcu/tree_stall.h | 38 kernel/rcu/update.c | 370 -- kernel/sched/core.c | 49 tools/testing/selftests/rcutorture/configs/rcu/CFLIST | 2 tools/testing/selftests/rcutorture/configs/rcu/RUDE01 | 10 tools/testing/selftests/rcutorture/configs/rcu/RUDE01.boot | 1 tools/testing/selftests/rcutorture/configs/rcu/TRACE01 | 10 tools/testing/selftests/rcutorture/configs/rcu/TRACE01.boot | 1 21 files changed, 1702 insertions(+), 780 deletions(-)