On Thu, Sep 10, 2020 at 5:49 PM Michael Larabel
<Michael@xxxxxxxxxxxxxxxxxx> wrote:
>
> I should be able to fire up some benchmarks of the patch overnight to
> see what they show, but guessing something more might be at play. While
> it's plausible this might help the Apache and Nginx web server results
> as they do touch the disk, Hackbench for instance shouldn't really be
> interacting with the file-system. Was the Hackbench perf data useful at
> all or should I generate a longer run of that for more events?
The hackbench data actually does have some of the same patterns with
ext4_write_iter showing up there too, but the perf profile there is
fairly weak (it and nginx both have _much_ fewer profile data points
than the apache run had).
hackbench I also didn't feel was all that interesting, because the
performance impact seemed more mixed there.
NOTE! The whole fair locking issue does show up even without any
lock/unlock/lock patterns, because even if you don't have that
"immediately re-take the lock" thing going on as in the ext4 example,
it's a very easy pattern to trigger by simply having a microbenchmark
that does the same system call over and over again. So the
"lock-unlock-lock-unlock" pattern can be two separate system calls,
each of which just does a single lock-unlock, but they do so at a high
frequency. So the ext4 code I pointed at and that trial patch (maybe)
fixing is just the most egregious case of lock re-taking. It can
easily happen with an external loop too (although normally I'd expect
people to buffer writes enough that the next write certainly shouldn't
be to the same page).
I do kind of wonder why that apache benchmark would have multiple
processes locking the same page, which is what makes me wonder if
there's something else going on. The profile wasn't entirely trivial
to read (the page locking itself does not show up very high on the
profile at all, so ), so I might have missed some other clue.
There were other lock_page cases, ie jbd2_journal_get_write_access()
etc, so I think it's the writing side interacting with the flushing
side.
But the lock-unlock-lock pattern in the ext4 write code made me go
"yeah, that's _exactly_ the kind of thing that would potentialyl slow
down a lot".
Again, that's not saying that other similar patterns don't occur
elsewhere. It's only saying that that's the only really obvious one I
found.
Linus
