On Thu, Feb 20, 2020 at 11:34:21AM +0100, Peter Zijlstra wrote:
> On Wed, Feb 19, 2020 at 08:44:50AM -0800, Paul E. McKenney wrote:
> > On Wed, Feb 19, 2020 at 05:35:35PM +0100, Peter Zijlstra wrote:
>
> > > Possibly, and I suppose the current version is less obviously dependent
> > > on the in_nmi() functionality as was the previous, seeing how Paul
> > > frobbed that all the way into the rcu_irq_enter*() implementation.
> > >
> > > So sure, I can go move it I suppose.
> >
> > No objections here.
>
> It now looks like so:
>
> ---
> Subject: rcu,tracing: Create trace_rcu_{enter,exit}()
> From: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> Date: Wed Feb 12 09:18:57 CET 2020
>
> To facilitate tracers that need RCU, add some helpers to wrap the
> magic required.
>
> The problem is that we can call into tracers (trace events and
> function tracing) while RCU isn't watching and this can happen from
> any context, including NMI.
>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
Reviewed-by: Paul E. McKenney <paulmck@xxxxxxxxxx>
> ---
> include/linux/rcupdate.h | 29 +++++++++++++++++++++++++++++
> 1 file changed, 29 insertions(+)
>
> --- a/include/linux/rcupdate.h
> +++ b/include/linux/rcupdate.h
> @@ -175,6 +175,35 @@ do { \
> #error "Unknown RCU implementation specified to kernel configuration"
> #endif
>
> +/**
> + * trace_rcu_enter - Force RCU to be active, for code that needs RCU readers
> + *
> + * Very similar to RCU_NONIDLE() above.
> + *
> + * Tracing can happen while RCU isn't active yet, for instance in the idle loop
> + * between rcu_idle_enter() and rcu_idle_exit(), or early in exception entry.
> + * RCU will happily ignore any read-side critical sections in this case.
> + *
> + * This function ensures that RCU is aware hereafter and the code can readily
> + * rely on RCU read-side critical sections working as expected.
> + *
> + * This function is NMI safe -- provided in_nmi() is correct and will nest up-to
> + * INT_MAX/2 times.
> + */
> +static inline int trace_rcu_enter(void)
> +{
> + int state = !rcu_is_watching();
> + if (state)
> + rcu_irq_enter_irqsave();
> + return state;
> +}
> +
> +static inline void trace_rcu_exit(int state)
> +{
> + if (state)
> + rcu_irq_exit_irqsave();
> +}
> +
> /*
> * The init_rcu_head_on_stack() and destroy_rcu_head_on_stack() calls
> * are needed for dynamic initialization and destruction of rcu_head
