Hi, It sounds to me like Paul is missing the main point of Elad's question. So I'm chiming in again. See inline comment below Elad's question. On Wed, 22 Dec 2021 18:29:57 -0800, Paul E. McKenney wrote: > Strictly speaking, all that the rcu_read_lock()/rcu_read_unlock() does > is to ensure that any RCU-protected linked data objects referenced by > the corresponding critical section remain in existence for the full > duration of that critical section. > > And even that overstates things a bit, as this describes not the > underlying RCU API itself, but rather some of that API's use cases. > (Though to be fair, these are the most popular of RCU's use cases.) > So even more strictly speaking, all the rcu_read_lock()/rcu_read_unlock() > does is to make any synchronize_rcu() or call_rcu() invocations starting > after a given rcu_read_lock() wait (synchronously or asynchronously, > respectively) until the corresponding rcu_read_unlock() is reached. > > Returning back to the first paragraph, additional protections can be > arranged, depending on the RCU use case. For example, if the non-pointer > fields of objects added to an RCU-protected linked data structure remain > unchanged while that object is accessible to readers, then the protection > includes not just existence, but also value. And there are other use > cases where those values might change while accessible to readers. > > So the most accurate answer to your question is "That is a design > choice, based on the RCU use case in question." > > My guess is that you are thinking in terms of designs where the > non-pointer fields of an object are constant while that object is > accessible to readers. Is my guess correct? > > Thanx, Paul > > On Wed, Dec 22, 2021 at 04:35:24PM -0500, Elad Lahav wrote: >> I will try to come up with something, but first I wanted to get your >> opinion as to whether this design pattern, where you need to >> dereference a pointer to a data structure containing all of the data >> considered to be protected by the critical section, is inherent to >> RCU, or is it just an idiosyncrasy of the implementation used in the >> Linux kernel? >> >> --Elad I guess Elad is asking more of the design choice of rcu_assign_pointer() and rcu_dereference(). A naive reader might easily miss the asymmetry of rcu_assign_pointer() with store-release and rcu_defeference() *without* load-acquire. IIUC, this asymmetry comes from the use cases where RCU performs best, that is read-mostly. Therefore, rcu_dereference() need to be light-weight as possible and doesn't imply load-acquire. Thankfully quite a lot of use cases can be covered by the pattern where address-dependency is sufficient for read-side. So one approach for Elad's concern would be to add a Quick Quiz on this asymmetry, I suppose. Elad, am I guessing right? Sidenote: RCU and memory accesses are orthogonal. You can use whatever memory access primitives for your needed memory ordering. (Which would be quite likely a tricky thing to do for a naive reader, though.) Thanks, Akira >> >> On Wed, 22 Dec 2021 at 13:19, Paul E. McKenney <paulmck@xxxxxxxxxx> wrote: >>> >>> On Wed, Dec 22, 2021 at 11:35:02PM +0900, Akira Yokosawa wrote: >>>> Hi, >>>> >>>> On Wed, 22 Dec 2021 07:15:54 -0500, Elad Lahav wrote: >>>>> Hi Akira, >>>>> >>>>>> I think this is mostly covered in Section 9.5.2.1 "Publish-Subscribe >>>>>> Mechanism" and Figure 9.10 "Publication/Subscription Constraints". >>>>> >>>>> I don't believe it is. I think you can infer that from reading sections 9.5 >>>>> and chapter 15, but it is never made explicit. >>>> >>>> I agree it is not explicit. >>>> >>>>> >>>>> Sub-section 9.5.2.1 talks about the use of rcu_dereference() to obtain a >>>>> pointer, but it doesn't go very deep into how that's implemented. You would >>>>> think that the writer's use of store-release semantics would necessitate >>>>> the reader's use of load-acquire semantics, but the discussion here, the >>>>> previous examples, and a quick inspection of how rcu_dereference() is >>>>> implemented, suggest that a READ_ONCE() is sufficient (or some less >>>>> Linux-kernel-specific equivalent). >>>>> >>>>> If I am a naive reader (and a lazy one, who haven't read chapter 15 and made >>>>> the necessary connection), I could write something like: >>>>> >>>>> struct foo_s { >>>>> int a; >>>>> } foo; >>>>> int b; >>>>> struct foo_s *g_foop = &foo; >>>>> >>>>> // Reader >>>>> rcu_read_lock(); >>>>> struct foo *l_foop = rcu_dereference(g_foop); >>>>> use(l_foop->a); >>>>> use(b); >>>>> rcu_read_unlock(); >>>>> >>>>> // Writer >>>>> struct foo *l_foop = malloc(sizeof(struct foo)); >>>>> l_foop->a = 1; >>>>> b = 2; >>>>> // Release semantics ensure previous stores are observed >>>>> rcu_assign_pointer(g_foop, l_foop); >>>>> >>>>> But since b has no address dependency to g_foop and since neither >>>>> rcu_read_lock() nor rcu_dereference() impose acquire semantics, then >>>>> the reader may not observe the new value of b. >>>> >>>> No, it may not. >>>> So what you want is the mention of "address dependency" somewhere in >>>> Section 9.5.2.1? >>>> >>>>> >>>>> Again, I may be missing something, but this seems to be a major point >>>>> that needs to be explained and emphasized. >>>> >>>> I guess Paul is intentionally avoiding discussions on memory ordering >>>> here in Section 9.5. >>>> >>>> Such a discussion would easily frighten naive readers... >>>> >>>> Anyway, I guess Paul would have some good compromise to satisfy you. >>> >>> Actually, I am going to ask Elad to propose the location and wording of an >>> addition to Section 9.5 covering this, so that we can discuss and refine >>> it. This addition might be a new paragraph, a footnote, a quick quiz, >>> a citation, or what have you. And the refining might switch back and >>> forth among these options a few times. But we do have to start somewhere. >>> >>> I am thinking in terms of this going in after the release that I was >>> naively planning to do yesterday. (The objective universe had other >>> ideas.) >>> >>>>>> BTW, I couldn't figure out what you meant by "the Spanish >>>>>> Acquisition"... >>>>> It's a reference to an old Monty Python skit. Apologies for the silly pun... >>>> >>>> Ah, now I guess I see the pun of "acquire" and "acquisition". ;-) >>> >>> And here I was hoping that Elad was purchasing a house in Barcelona >>> or some such. ;-) >>> >>> (Sorry, couldn't resist!) >>> >>> Thanx, Paul