On Wed, Dec 21, 2022 at 5:20 PM Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx> wrote: > > On 2022-12-21 07:11, Frederic Weisbecker wrote: > > On Tue, Dec 20, 2022 at 10:43:25PM -0500, Mathieu Desnoyers wrote: > >> On 2022-12-20 19:58, Frederic Weisbecker wrote: > >>> On Wed, Dec 21, 2022 at 01:49:57AM +0100, Frederic Weisbecker wrote: > >>>> On Tue, Dec 20, 2022 at 07:15:00PM -0500, Joel Fernandes wrote: > >>>>> On Tue, Dec 20, 2022 at 5:45 PM Frederic Weisbecker <frederic@xxxxxxxxxx> wrote: > >>>>> Agreed about (1). > >>>>> > >>>>>> _ In (2), E pairs with the address-dependency between idx and lock_count. > >>>>> > >>>>> But that is not the only reason. If that was the only reason for (2), > >>>>> then there is an smp_mb() just before the next-scan post-flip before > >>>>> the lock counts are read. > >>>> > >>>> The post-flip barrier makes sure the new idx is visible on the next READER's > >>>> turn, but it doesn't protect against the fact that "READ idx then WRITE lock[idx]" > >>>> may appear unordered from the update side POV if there is no barrier between the > >>>> scan and the flip. > >>>> > >>>> If you remove the smp_mb() from the litmus test I sent, things explode. > >>> > >>> Or rather, look at it the other way, if there is no barrier between the lock > >>> scan and the index flip (E), then the index flip can appear to be written before the > >>> lock is read. Which means you may start activating the index before you finish > >>> reading it (at least it appears that way from the readers pont of view). > >> > >> Considering that you can have pre-existing readers from arbitrary index > >> appearing anywhere in the grace period (because a reader can fetch the > >> index and be preempted for an arbitrary amount of time before incrementing > >> the lock count), the grace period algorithm needs to deal with the fact that > >> a newcoming reader can appear in a given index either before or after the > >> flip. > > > > True but the number of preempted tasks is bound and there is a forward progress guarantee. > > > >> I don't see how flipping the index before or after loading the unlock/lock > >> values would break anything (except for unlikely counter overflow situations > >> as previously discussed). > > > > Forward progress guarantee. > > Considering a coherent cache, the store-buffer will ensure that the > index flip eventually reaches all readers. This bounds the time during > which readers can flood the current index, and therefore guarantees > forward progress. AFAIK the Linux kernel does not support architectures > with incoherent caches. > > So I still don't see how having the barrier before or after the index > flip is useful for forward progress. Even though eventually the writes on either side will make it, I think you have little lost opportunity without the "D" memory barrier without satisfying the store-buffer pattern. i.e. even if the idx update was seen by the read-side, its unlock count contributions to the now-inactive idx may not be seen. I think that makes it slightly better for making the grace period end sooner. I think that's what you "scan both" idea also tries to achieve, so for that reason you should like "D" as well. But yes it depends on how forward-progress is defined. If defined as starvation - no the post/pref flip MBs are not helpful for that (eventually the writes will make it). If defined as not losing an opportunity to end the GP sooner, yes the flip-related memory barriers does help that. As for whether or not we need "E", that is up for debate. I am intrigued by Frederick's litmus tests showing that it helps, because my litmus tests don't show it. Thanks.
