On 2022-03-01 21:42:17 [-0800], Paul E. McKenney wrote:
> There have been several different things requiring various pieces of RCU
> to be running quite early, and I need to do a bit of code archaeology to
> check this out. One of these is what forced me to move to expedited grace
> periods once the scheduler started. And the expedited grace periods do
> require the timers be working, at least in their current form.
expedited as in synchronize_rcu_expedited()? Because we don't do this
but then only after boot so it probably doesn't change a thing and makes
other complicated.
> I vaguely recall tracing requiring call_rcu() extremely early (as
> in before rcu_init() is called), but that was not a problem because
> there was no requirement that the callback be invoked before the first
> couple of sets of initcalls had completed. The only challenge in this
> case was to avoid stomping on the callback lists at rcu_init() time,
> no timers required.
Okay.
> One straightforward approach would be to choose the call site of the
> rcu_tasks_initiate_self_tests() function based on whether or not the
> kernel was built with CONFIG_PREEMPT_RT.
But then we have this change between RT and !RT due to other
limitations. But my understanding is correct that RT does not have a
working synchronize_rcu() in an early_initcall() while !RT does, right?
> The other thought that comes to mind is that you may need the old-style
> softirq until ksoftirqd is spawned. I doubt that real-time response is
> a big deal that early in boot. ;-)
Puh. Given that the whole infrastructure is not prepared for that it is
probably more than a few lines plus a conditional jump in the IRQ
hot path. Also we do have scheduling at this point and IRQ-on/off works
which means possible dead locks because sleeping locks are acquired in
hard-IRQ context.
But let me summarize this: I did this:
|@@ -1600,6 +1600,7 @@ static noinline void __init kernel_init_
|
| rcu_init_tasks_generic();
| do_pre_smp_initcalls();
|+ rcu_tasks_initiate_self_tests();
| lockup_detector_init();
|
| smp_init();
To simply move the test from rcu_init_tasks_generic() to after
do_pre_smp_initcalls(). If we can't move rcu_init_tasks_generic() after
do_pre_smp_initcalls() or at least the test part because we need working
synchronize_rcu() in early_initcall() then I need to move the RT
requirements before. Simple ;)
The requirements are:
--- a/init/main.c
+++ b/init/main.c
@@ -1598,6 +1601,9 @@ static noinline void __init kernel_init_freeable(void)
init_mm_internals();
+ spawn_ksoftirqd();
+ irq_work_init_threads();
+
rcu_init_tasks_generic();
do_pre_smp_initcalls();
lockup_detector_init();
spawn_ksoftirqd() is what I mentioned. What I just figured out is
irq_work_init_threads() due to
call_rcu_tasks_iw_wakeup()
I can't move this to hard-IRQ context because it invokes wake_up() which
acquires sleeping locks. If you say that rtp->cbs_wq has only one waiter
and something like rcuwait_wait_event() / rcuwait_wake_up() would work
as well then call_rcu_tasks_iw_wakeup() could be lifter to hard-IRQ
context and we need to worry only about spawn_ksoftirqd() :)
> Thanx, Paul
>
Sebastian
