Hi,
It sounds to me like Paul is missing the main point of Elad's question.
So I'm chiming in again. See inline comment below Elad's question.
On Wed, 22 Dec 2021 18:29:57 -0800, Paul E. McKenney wrote:
> Strictly speaking, all that the rcu_read_lock()/rcu_read_unlock() does
> is to ensure that any RCU-protected linked data objects referenced by
> the corresponding critical section remain in existence for the full
> duration of that critical section.
>
> And even that overstates things a bit, as this describes not the
> underlying RCU API itself, but rather some of that API's use cases.
> (Though to be fair, these are the most popular of RCU's use cases.)
> So even more strictly speaking, all the rcu_read_lock()/rcu_read_unlock()
> does is to make any synchronize_rcu() or call_rcu() invocations starting
> after a given rcu_read_lock() wait (synchronously or asynchronously,
> respectively) until the corresponding rcu_read_unlock() is reached.
>
> Returning back to the first paragraph, additional protections can be
> arranged, depending on the RCU use case. For example, if the non-pointer
> fields of objects added to an RCU-protected linked data structure remain
> unchanged while that object is accessible to readers, then the protection
> includes not just existence, but also value. And there are other use
> cases where those values might change while accessible to readers.
>
> So the most accurate answer to your question is "That is a design
> choice, based on the RCU use case in question."
>
> My guess is that you are thinking in terms of designs where the
> non-pointer fields of an object are constant while that object is
> accessible to readers. Is my guess correct?
>
> Thanx, Paul
>
> On Wed, Dec 22, 2021 at 04:35:24PM -0500, Elad Lahav wrote:
>> I will try to come up with something, but first I wanted to get your
>> opinion as to whether this design pattern, where you need to
>> dereference a pointer to a data structure containing all of the data
>> considered to be protected by the critical section, is inherent to
>> RCU, or is it just an idiosyncrasy of the implementation used in the
>> Linux kernel?
>>
>> --Elad
I guess Elad is asking more of the design choice of rcu_assign_pointer()
and rcu_dereference().
A naive reader might easily miss the asymmetry of rcu_assign_pointer()
with store-release and rcu_defeference() *without* load-acquire.
IIUC, this asymmetry comes from the use cases where RCU performs best,
that is read-mostly. Therefore, rcu_dereference() need to be
light-weight as possible and doesn't imply load-acquire.
Thankfully quite a lot of use cases can be covered by the pattern
where address-dependency is sufficient for read-side.
So one approach for Elad's concern would be to add a Quick Quiz on
this asymmetry, I suppose.
Elad, am I guessing right?
Sidenote: RCU and memory accesses are orthogonal. You can use whatever
memory access primitives for your needed memory ordering. (Which would
be quite likely a tricky thing to do for a naive reader, though.)
Thanks, Akira
>>
>> On Wed, 22 Dec 2021 at 13:19, Paul E. McKenney <paulmck@xxxxxxxxxx> wrote:
>>>
>>> On Wed, Dec 22, 2021 at 11:35:02PM +0900, Akira Yokosawa wrote:
>>>> Hi,
>>>>
>>>> On Wed, 22 Dec 2021 07:15:54 -0500, Elad Lahav wrote:
>>>>> Hi Akira,
>>>>>
>>>>>> I think this is mostly covered in Section 9.5.2.1 "Publish-Subscribe
>>>>>> Mechanism" and Figure 9.10 "Publication/Subscription Constraints".
>>>>>
>>>>> I don't believe it is. I think you can infer that from reading sections 9.5
>>>>> and chapter 15, but it is never made explicit.
>>>>
>>>> I agree it is not explicit.
>>>>
>>>>>
>>>>> Sub-section 9.5.2.1 talks about the use of rcu_dereference() to obtain a
>>>>> pointer, but it doesn't go very deep into how that's implemented. You would
>>>>> think that the writer's use of store-release semantics would necessitate
>>>>> the reader's use of load-acquire semantics, but the discussion here, the
>>>>> previous examples, and a quick inspection of how rcu_dereference() is
>>>>> implemented, suggest that a READ_ONCE() is sufficient (or some less
>>>>> Linux-kernel-specific equivalent).
>>>>>
>>>>> If I am a naive reader (and a lazy one, who haven't read chapter 15 and made
>>>>> the necessary connection), I could write something like:
>>>>>
>>>>> struct foo_s {
>>>>> int a;
>>>>> } foo;
>>>>> int b;
>>>>> struct foo_s *g_foop = &foo;
>>>>>
>>>>> // Reader
>>>>> rcu_read_lock();
>>>>> struct foo *l_foop = rcu_dereference(g_foop);
>>>>> use(l_foop->a);
>>>>> use(b);
>>>>> rcu_read_unlock();
>>>>>
>>>>> // Writer
>>>>> struct foo *l_foop = malloc(sizeof(struct foo));
>>>>> l_foop->a = 1;
>>>>> b = 2;
>>>>> // Release semantics ensure previous stores are observed
>>>>> rcu_assign_pointer(g_foop, l_foop);
>>>>>
>>>>> But since b has no address dependency to g_foop and since neither
>>>>> rcu_read_lock() nor rcu_dereference() impose acquire semantics, then
>>>>> the reader may not observe the new value of b.
>>>>
>>>> No, it may not.
>>>> So what you want is the mention of "address dependency" somewhere in
>>>> Section 9.5.2.1?
>>>>
>>>>>
>>>>> Again, I may be missing something, but this seems to be a major point
>>>>> that needs to be explained and emphasized.
>>>>
>>>> I guess Paul is intentionally avoiding discussions on memory ordering
>>>> here in Section 9.5.
>>>>
>>>> Such a discussion would easily frighten naive readers...
>>>>
>>>> Anyway, I guess Paul would have some good compromise to satisfy you.
>>>
>>> Actually, I am going to ask Elad to propose the location and wording of an
>>> addition to Section 9.5 covering this, so that we can discuss and refine
>>> it. This addition might be a new paragraph, a footnote, a quick quiz,
>>> a citation, or what have you. And the refining might switch back and
>>> forth among these options a few times. But we do have to start somewhere.
>>>
>>> I am thinking in terms of this going in after the release that I was
>>> naively planning to do yesterday. (The objective universe had other
>>> ideas.)
>>>
>>>>>> BTW, I couldn't figure out what you meant by "the Spanish
>>>>>> Acquisition"...
>>>>> It's a reference to an old Monty Python skit. Apologies for the silly pun...
>>>>
>>>> Ah, now I guess I see the pun of "acquire" and "acquisition". ;-)
>>>
>>> And here I was hoping that Elad was purchasing a house in Barcelona
>>> or some such. ;-)
>>>
>>> (Sorry, couldn't resist!)
>>>
>>> Thanx, Paul
