On Fri, Jan 24, 2025 at 11:25:01PM +0100, Frederic Weisbecker wrote:
> Le Fri, Jan 24, 2025 at 11:40:54AM -0800, Paul E. McKenney a écrit :
> > > > > I'm wondering, what prevents us from removing rcu_state.gp_seq and rely only on
> > > > > the root node for the global state ?
> > > >
> > > > One scenario comes to mind immediately. There may be others.
> > > >
> > > > Suppose we were running with default configuration on a system with
> > > > "only" eight CPUs. Then there is only the one rcu_node structure,
> > > > which is both root and leaf. Without rcu_state.gp_seq, there
> > > > would be no way to communicate the beginning of the grace period to
> > > > get_state_synchronize_rcu_full() without also allowing quiescent states
> > > > to be reported. There would thus be no time in which to check for newly
> > > > onlined/offlined CPUs.
> > >
> > > Heh, that makes sense! Though perhaps that qsmaskinit[next] handling part
> > > could be done before rcu_seq_start()?
> >
> > If we do that, aren't we vulnerable to a CPU coming online just after
> > we handled qsmaskinit{,next} and just before we do rcu_seq_start()?
>
> But then that CPU is guaranteed to see the pre-GP accesses because
> rcutree_report_cpu_starting() holds the rnp lock.
>
> Hmm but it won't see the starting gp_seq and it might therefore be one GP
> too early while accelerating its own callbacks (mis-attributing them the
> current GP in progress instead of a subsequent one) even though other CPUs
> may have reported their own QS already.
>
> Because I only now realize this necessary order while starting a grace period:
>
> 1) rcu_seq_start(rcu_state.gp_seq)
> 2) go through ->qsmaskinit[next] _while holding leaf rnp locks_
> 3) go through the whole tree breadth first to reflect rcu_state.gp_seq to
> rnp->gp_seq. CPUs may start reporting QS concurrently once the first leaf
> node is reached
>
> And the step 2) with its fully ordered locking on leaves which release the write
> to rcu_state.gp_seq is what makes sure that if any CPU from step 3) has already
> reported a QS, it is guaranteed that any call to rcu_seq_snap() under any other
> leaf rnp locking will return the sequence of the subsequent GP number and not the
> current one in progress.
Very good!!!
> This is what makes RCU an entertaining companion. You (think you) understand
> it in the evening and then you forget it all over again in the next morning.
> (And in lunch time you stare at things...).
That sums up many of my entertaining experiences with RCU over the past
three decades. ;-)
Thanx, Paul
