On Tue, Sep 22, 2020 at 11:31:00AM -0700, Paul E. McKenney wrote:
> > Also, it's not just the p == &global_variable case. Consider:
> >
> > struct a { struct b *b; };
> > struct b { ... };
> >
> > Thread 1:
> >
> > /* one-time initialized data shared by all instances of b */
> > static struct c *c;
> >
> > void init_b(struct a *a)
> > {
> > if (!c)
> > c = alloc_c();
> >
> > smp_store_release(&a->b, kzalloc(sizeof(struct b)));
> > }
> >
> > Thread 2:
> >
> > void use_b_if_present(struct a *a)
> > {
> > struct b *b = READ_ONCE(a->b);
> >
> > if (b) {
> > c->... # crashes because c still appears to be NULL
> > }
> > }
> >
> >
> > So when the *first* "b" is allocated, the global data "c" is initialized. Then
> > when using a "b", we expect to be able to access "c". But there's no
> > data dependency from "b" to "c"; it's a control dependency only.
> > So smp_load_acquire() is needed, not READ_ONCE().
> >
> > And it can be an internal implementation detail of "b"'s subsystem whether it
> > happens to use global data "c".
>
> Given that "c" is static, these two subsystems must be in the same
> translation unit. So I don't see how this qualifies as being internal to
> "b"'s subsystem.
You're missing the point here. b and c could easily be allocated by a function
alloc_b() that's in another file.
> Besides which, control dependencies should be used only by LKMM experts
> at this point.
What does that even mean? Control dependencies are everywhere.
> > This sort of thing is why people objected to the READ_ONCE() optimization during
> > the discussion at
> > https://lkml.kernel.org/linux-fsdevel/20200717044427.68747-1-ebiggers@xxxxxxxxxx/T/#u.
> > Most kernel developers aren't experts in the LKMM, and they want something
> > that's guaranteed to be correct without having to to think really hard about it
> > and make assumptions about the internal implementation details of other
> > subsystems, how compilers have implemented the C standard, and so on.
>
> And smp_load_acquire()is provided for that reason. Its name was
> even based on the nomenclature used in the C standard and elsewhere.
> And again, control dependencies are for LKMM experts, as they are very
> tricky to get right.
How does a developer know that the code they're calling in another subsystem
wasn't written by one of these "experts" and therefore has a control dependency?
>
> But in the LKMM documentation, you are likely to find LKMM experts who
> want to optimize all the way, particularly in cases like the one-time
> init pattern where all the data is often local. And the best basis for
> READ_ONCE() in one-time init is not a control dependency, but rather
> ordering of accesses to a single variable from a single task combined
> with locking, both of which are quite robust and much easier to use,
> especially in comparison to control dependencies.
>
> My goal for LKMM is not that each and every developer have a full
> understanding of every nook and cranny of that model, but instead that
> people can find the primitives supporting the desired point in the
> performance/simplicity tradoff space. And yes, I have more writing
> to do to make more progress towards that goal.
So are you saying people should use smp_load_acquire(), or are you saying people
should use READ_ONCE()?
- Eric
