On Tue, Mar 21, 2023 at 08:04:20AM -0700, Paul E. McKenney wrote:
> On Tue, Mar 21, 2023 at 04:13:46PM +0800, Zqiang wrote:
> > insmod rcutorture.ko
> > rmmod rcutorture.ko
> >
> > [ 209.437327] WARNING: CPU: 0 PID: 508 at kernel/workqueue.c:3167 __flush_work+0x50a/0x540
> > [ 209.437346] Modules linked in: rcutorture(-) torture [last unloaded: rcutorture]
> > [ 209.437382] CPU: 0 PID: 508 Comm: rmmod Tainted: G W 6.3.0-rc1-yocto-standard+
> > [ 209.437406] RIP: 0010:__flush_work+0x50a/0x540
> > .....
> > [ 209.437758] flush_delayed_work+0x36/0x90
> > [ 209.437776] cleanup_srcu_struct+0x68/0x2e0
> > [ 209.437817] srcu_module_notify+0x71/0x140
> > [ 209.437854] blocking_notifier_call_chain+0x9d/0xd0
> > [ 209.437880] __x64_sys_delete_module+0x223/0x2e0
> > [ 209.438046] do_syscall_64+0x43/0x90
> > [ 209.438062] entry_SYSCALL_64_after_hwframe+0x72/0xdc
> >
> > For srcu objects defined with DEFINE_SRCU() or DEFINE_STATIC_SRCU(),
> > when compiling and loading as modules, the srcu_module_coming() is invoked,
> > allocate memory for srcu structure's->sda and initialize sda structure,
> > due to not fully initialize srcu structure's->sup, so at this time the
> > sup structure's->delaywork.func is null, if not invoke init_srcu_struct_fields()
> > before unloading modules, in srcu_module_going() the __flush_work() find
> > work->func is empty, will raise the warning above.
> >
> > This commit add init_srcu_struct_fields() to initialize srcu structure's->sup,
> > in srcu_module_coming().
> >
> > Signed-off-by: Zqiang <qiang1.zhang@xxxxxxxxx>
>
> Good catch, and thank you for testing the in-module case!
>
> One question below...
>
> Thanx, Paul
>
> > ---
> > kernel/rcu/srcutree.c | 11 ++++++++---
> > 1 file changed, 8 insertions(+), 3 deletions(-)
> >
> > diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
> > index 1fb078abbdc9..42d8720e016c 100644
> > --- a/kernel/rcu/srcutree.c
> > +++ b/kernel/rcu/srcutree.c
> > @@ -1921,7 +1921,8 @@ static int srcu_module_coming(struct module *mod)
> > ssp->sda = alloc_percpu(struct srcu_data);
> > if (WARN_ON_ONCE(!ssp->sda))
> > return -ENOMEM;
> > - init_srcu_struct_data(ssp);
> > + if (WARN_ON_ONCE(init_srcu_struct_fields(ssp, true)))
> > + return -ENOMEM;
>
> Wouldn't it be better to simply delete the init_srcu_struct_data()?
>
> Then the first call to check_init_srcu_struct() would take care of
> the initialization, just as for the non-module case. Or am I missing
> something subtle?
>
> It should also be possible to eliminate duplicate code between the
> in-module and non-module statically allocated initialization cases,
> come to think of it.
But that would amount to only one line of duplicated code, so this
last is probably not worth it.
Thanx, Paul
> > }
> > return 0;
> > }
> > @@ -1931,9 +1932,13 @@ static void srcu_module_going(struct module *mod)
> > {
> > int i;
> > struct srcu_struct **sspp = mod->srcu_struct_ptrs;
> > + struct srcu_struct *ssp;
> >
> > - for (i = 0; i < mod->num_srcu_structs; i++)
> > - cleanup_srcu_struct(*(sspp++));
> > + for (i = 0; i < mod->num_srcu_structs; i++) {
> > + ssp = *(sspp++);
> > + cleanup_srcu_struct(ssp);
> > + free_percpu(ssp->sda);
> > + }
>
> And good catch on another memory leak with this one, looks proper
> to me.
>
> > }
> >
> > /* Handle one module, either coming or going. */
> > --
> > 2.25.1
> >
