On 9/14/20 12:47 PM, Linus Torvalds wrote:
Michael et al,
Ok, I redid my failed "hybrid mode" patch from scratch (original
patch never sent out, I never got it to a working point).
Having learnt from my mistake, this time instead of trying to mix the
old and the new code, instead I just extended the new code, and wrote
a _lot_ of comments about it.
I also made it configurable, using a "page_lock_unfairness" knob,
which this patch defaults to 1000 (which is basically infinite).
That's just a value that says how many times we'll try the old unfair
case, so "1000" means "we'll re-queue up to a thousand times before we
say enough is enough" and zero is the fair mode that shows the
performance problems.
I've only (lightly) tested those two extremes, I think the interesting
range is likely in the 1-5 range.
So you can do
echo 0 > /proc/sys/vm/page_lock_unfairness
.. run test ..
and you should get the same numbers as without this patch (within
noise, of course).
Or do
echo 5 > /proc/sys/vm/page_lock_unfairness
.. run test ..
and get numbers for "we accept some unfairness, but if we have to
requeue more than five times, we force the fair mode".
Again, the default is 1000, which is ludicrously high (it's not a
"this many retries per page" count, it's a "for each waiter" count). I
made it that high just because I have *not* run any numbers for that
interesting range, I just checked the extreme cases, and I wanted to
make sure that Michael sees the old performance (modulo other changes
to 5.9, of course).
Comments? The patch really has a fair amount of comments in it, in
fact the code changes are reasonably small, most of the changes really
are about new and updated comments about what is going on.
I was burnt by making a mess of this the first time, so I proceeded
more thoughtfully this time. Hopefullyt the end result is also better.
(Note that it's a commit and has a SHA1, but it's from my "throw-away
tree for testing", so it doesn't have my sign-off or any real commit
message yet: I'll do that once it gets actual testing and comments).
Linus
Still running more benchmarks and on more systems, but so far at least
as the Apache test is concerned this patch does seem to largely address
the issue. The performance with the default 1000 page_lock_unfairness
was yielding results more similar to 5.8 and in some cases tweaking the
value did help improve the performance. A PLU value of 4~5 seems to
yield the best performance.
The results though with Hackbench and Redis that exhibited similar drops
from the commit in question remained mixed. Overview of the Apache
metrics below, the other tests and system details @
https://openbenchmarking.org/result/2009154-FI-LINUX58CO57 For the other
systems still testing, so far it's looking like similar relative impact
to these results.
Apache Siege 2.4.29
Concurrent Users: 1
Transactions Per Second > Higher Is Better
v5.8 ............. 7684.81
|=================================================
v5.9 Git ......... 7390.86 |===============================================
Default PLU 1000 . 7579.49
|=================================================
PLU 0 ............ 7937.84
|===================================================
PLU 1 ............ 7464.61 |================================================
PLU 2 ............ 7552.61
|=================================================
PLU 3 ............ 7475.96 |================================================
PLU 4 ............ 7638.69
|=================================================
PLU 5 ............ 7735.75
|==================================================
Apache Siege 2.4.29
Concurrent Users: 50
Transactions Per Second > Higher Is Better
v5.8 ............. 39280.51 |===============================================
v5.9 Git ......... 28240.71 |==================================
Default PLU 1000 . 39708.30 |===============================================
PLU 0 ............ 26645.15 |================================
PLU 1 ............ 38709.95 |==============================================
PLU 2 ............ 39712.82 |===============================================
PLU 3 ............ 41959.67
|==================================================
PLU 4 ............ 38870.90 |==============================================
PLU 5 ............ 41301.97
|=================================================
Apache Siege 2.4.29
Concurrent Users: 100
Transactions Per Second > Higher Is Better
v5.8 ............. 51255.73 |=============================================
v5.9 Git ......... 21926.62 |===================
Default PLU 1000 . 42001.86 |=====================================
PLU 0 ............ 21528.43 |===================
PLU 1 ............ 37138.49 |=================================
PLU 2 ............ 38086.58 |==================================
PLU 3 ............ 38057.72 |==================================
PLU 4 ............ 56350.51
|==================================================
PLU 5 ............ 37868.57 |==================================
Apache Siege 2.4.29
Concurrent Users: 200
Transactions Per Second > Higher Is Better
v5.8 ............. 47825.12 |===================================
v5.9 Git ......... 20174.78 |===============
Default PLU 1000 . 48190.05 |===================================
PLU 0 ............ 20095.10 |===============
PLU 1 ............ 48524.44 |====================================
PLU 2 ............ 47823.09 |===================================
PLU 3 ............ 47751.02 |===================================
PLU 4 ............ 68286.02
|==================================================
PLU 5 ............ 47662.08 |===================================
Apache Siege 2.4.29
Concurrent Users: 250
Transactions Per Second > Higher Is Better
v5.8 ............. 55279.65 |=====================================
v5.9 Git ......... 20282.62 |==============
Default PLU 1000 . 67639.46 |=============================================
PLU 0 ............ 20181.98 |==============
PLU 1 ............ 40505.37 |===========================
PLU 2 ............ 56914.07 |======================================
PLU 3 ............ 55285.35 |=====================================
PLU 4 ............ 55499.25 |=====================================
PLU 5 ............ 74347.77
|==================================================
