On Wed, Jan 16, 2019 at 03:34:55PM +1100, Dave Chinner wrote:
> On Tue, Jan 15, 2019 at 09:23:12PM -0500, Jerome Glisse wrote:
> > On Tue, Jan 15, 2019 at 06:01:09PM -0800, Dan Williams wrote:
> > > On Tue, Jan 15, 2019 at 5:56 PM Jerome Glisse <jglisse@xxxxxxxxxx> wrote:
> > > > On Tue, Jan 15, 2019 at 04:44:41PM -0800, John Hubbard wrote:
> > > [..]
> > > > To make it clear.
> > > >
> > > > Lock code:
> > > > GUP()
> > > > ...
> > > > lock_page(page);
> > > > if (PageWriteback(page)) {
> > > > unlock_page(page);
> > > > wait_stable_page(page);
> > > > goto retry;
> > > > }
> > > > atomic_add(page->refcount, PAGE_PIN_BIAS);
> > > > unlock_page(page);
> > > >
> > > > test_set_page_writeback()
> > > > bool pinned = false;
> > > > ...
> > > > pinned = page_is_pin(page); // could be after TestSetPageWriteback
> > > > TestSetPageWriteback(page);
> > > > ...
> > > > return pinned;
> > > >
> > > > Memory barrier:
> > > > GUP()
> > > > ...
> > > > atomic_add(page->refcount, PAGE_PIN_BIAS);
> > > > smp_mb();
> > > > if (PageWriteback(page)) {
> > > > atomic_add(page->refcount, -PAGE_PIN_BIAS);
> > > > wait_stable_page(page);
> > > > goto retry;
> > > > }
> > > >
> > > > test_set_page_writeback()
> > > > bool pinned = false;
> > > > ...
> > > > TestSetPageWriteback(page);
> > > > smp_wmb();
> > > > pinned = page_is_pin(page);
> > > > ...
> > > > return pinned;
> > > >
> > > >
> > > > One is not more complex than the other. One can contend, the other
> > > > will _never_ contend.
> > >
> > > The complexity is in the validation of lockless algorithms. It's
> > > easier to reason about locks than barriers for the long term
> > > maintainability of this code. I'm with Jan and John on wanting to
> > > explore lock_page() before a barrier-based scheme.
> >
> > How is the above hard to validate ?
>
> Well, if you think it's so easy, then please write the test cases so
> we can add them to fstests and make sure that we don't break it in
> future.
>
> If you can't write filesystem test cases that exercise these race
> conditions reliably, then the answer to your question is "it is
> extremely hard to validate" and the correct thing to do is to start
> with the simple lock_page() based algorithm.
>
> Premature optimisation in code this complex is something we really,
> really need to avoid.
Litmus test shows that this never happens, i am attaching 2 litmus
test one with barrier and one without. Without barrier we can see
the double negative !PageWriteback in GUP and !page_pinned() in
test_set_page_writeback() (0:EAX = 0; 1:EAX = 0; below)
~/local/bin/litmus7 -r 100 gup.litmus
...
Histogram (3 states)
2 *>0:EAX=0; 1:EAX=0; x=1; y=1;
4999999:>0:EAX=1; 1:EAX=0; x=1; y=1;
4999999:>0:EAX=0; 1:EAX=1; x=1; y=1;
Ok
Witnesses
Positive: 2, Negative: 9999998
Condition exists (0:EAX=0 /\ 1:EAX=0) is validated
Hash=2d53e83cd627ba17ab11c875525e078b
Observation SB Sometimes 2 9999998
Time SB 3.24
With the barrier this never happens:
~/local/bin/litmus7 -r 10000 gup-mb.litmus
...
Histogram (3 states)
499579828:>0:EAX=1; 1:EAX=0; x=1; y=1;
499540152:>0:EAX=0; 1:EAX=1; x=1; y=1;
880020:>0:EAX=1; 1:EAX=1; x=1; y=1;
No
Witnesses
Positive: 0, Negative: 1000000000
Condition exists (0:EAX=0 /\ 1:EAX=0) is NOT validated
Hash=0dd48258687c8f737921f907c093c316
Observation SB Never 0 1000000000
I do not know any better test than litmus for this kind of thing.
Cheers,
Jérôme
X86 SB
"GUP"
{ x=0; y=0; }
P0 | P1 ;
MOV [x],$1 | MOV [y],$1 ;
MOV EAX,[y] | MOV EAX,[x] ;
locations [x;y;]
exists (0:EAX=0 /\ 1:EAX=0)
X86 SB
"GUP with barrier"
{ x=0; y=0; }
P0 | P1 ;
MOV [x],$1 | MOV [y],$1 ;
MFENCE | MFENCE ;
MOV EAX,[y] | MOV EAX,[x] ;
locations [x;y;]
exists (0:EAX=0 /\ 1:EAX=0)
