On 06/03/2013 11:51 AM, Raghavendra K T wrote:
On 06/03/2013 07:10 AM, Raghavendra K T wrote:
On 06/02/2013 09:50 PM, Jiannan Ouyang wrote:
On Sun, Jun 2, 2013 at 1:07 AM, Gleb Natapov <gleb@xxxxxxxxxx> wrote:
High level question here. We have a big hope for "Preemptable Ticket
Spinlock" patch series by Jiannan Ouyang to solve most, if not all,
ticketing spinlocks in overcommit scenarios problem without need for
PV.
So how this patch series compares with his patches on PLE enabled
processors?
No experiment results yet.
An error is reported on a 20 core VM. I'm during an internship
relocation, and will start work on it next week.
Preemptable spinlocks' testing update:
I hit the same softlockup problem while testing on 32 core machine with
32 guest vcpus that Andrew had reported.
After that i started tuning TIMEOUT_UNIT, and when I went till (1<<8),
things seemed to be manageable for undercommit cases.
But I still see degradation for undercommit w.r.t baseline itself on 32
core machine (after tuning).
(37.5% degradation w.r.t base line).
I can give the full report after the all tests complete.
For over-commit cases, I again started hitting softlockups (and
degradation is worse). But as I said in the preemptable thread, the
concept of preemptable locks looks promising (though I am still not a
fan of embedded TIMEOUT mechanism)
Here is my opinion of TODOs for preemptable locks to make it better ( I
think I need to paste in the preemptable thread also)
1. Current TIMEOUT UNIT seem to be on higher side and also it does not
scale well with large guests and also overcommit. we need to have a
sort of adaptive mechanism and better is sort of different TIMEOUT_UNITS
for different types of lock too. The hashing mechanism that was used in
Rik's spinlock backoff series fits better probably.
2. I do not think TIMEOUT_UNIT itself would work great when we have a
big queue (for large guests / overcommits) for lock.
one way is to add a PV hook that does yield hypercall immediately for
the waiters above some THRESHOLD so that they don't burn the CPU.
( I can do POC to check if that idea works in improving situation
at some later point of time)
Preemptable-lock results from my run with 2^8 TIMEOUT:
+-----------+-----------+-----------+------------+-----------+
ebizzy (records/sec) higher is better
+-----------+-----------+-----------+------------+-----------+
base stdev patched stdev %improvement
+-----------+-----------+-----------+------------+-----------+
1x 5574.9000 237.4997 3484.2000 113.4449 -37.50202
2x 2741.5000 561.3090 351.5000 140.5420 -87.17855
3x 2146.2500 216.7718 194.8333 85.0303 -90.92215
4x 1663.0000 141.9235 101.0000 57.7853 -93.92664
+-----------+-----------+-----------+------------+-----------+
+-----------+-----------+-----------+------------+-----------+
dbench (Throughput) higher is better
+-----------+-----------+-----------+------------+-----------+
base stdev patched stdev %improvement
+-----------+-----------+-----------+------------+-----------+
1x 14111.5600 754.4525 3930.1602 2547.2369 -72.14936
2x 2481.6270 71.2665 181.1816 89.5368 -92.69908
3x 1510.2483 31.8634 104.7243 53.2470 -93.06576
4x 1029.4875 16.9166 72.3738 38.2432 -92.96992
+-----------+-----------+-----------+------------+-----------+
Note we can not trust on overcommit results because of softlock-ups
Hi, I tried
(1) TIMEOUT=(2^7)
(2) having yield hypercall that uses kvm_vcpu_on_spin() to do directed
yield to other vCPUs.
Now I do not see any soft-lockup in overcommit cases and results are
better now (except ebizzy 1x). and for dbench I see now it is closer to
base and even improvement in 4x
+-----------+-----------+-----------+------------+-----------+
ebizzy (records/sec) higher is better
+-----------+-----------+-----------+------------+-----------+
base stdev patched stdev %improvement
+-----------+-----------+-----------+------------+-----------+
5574.9000 237.4997 523.7000 1.4181 -90.60611
2741.5000 561.3090 597.8000 34.9755 -78.19442
2146.2500 216.7718 902.6667 82.4228 -57.94215
1663.0000 141.9235 1245.0000 67.2989 -25.13530
+-----------+-----------+-----------+------------+-----------+
+-----------+-----------+-----------+------------+-----------+
dbench (Throughput) higher is better
+-----------+-----------+-----------+------------+-----------+
base stdev patched stdev %improvement
+-----------+-----------+-----------+------------+-----------+
14111.5600 754.4525 884.9051 24.4723 -93.72922
2481.6270 71.2665 2383.5700 333.2435 -3.95132
1510.2483 31.8634 1477.7358 50.5126 -2.15279
1029.4875 16.9166 1075.9225 13.9911 4.51050
+-----------+-----------+-----------+------------+-----------+
IMO hash based timeout is worth a try further.
I think little more tuning will get more better results.
Jiannan, When you start working on this, I can also help
to get best of preemptable lock idea if you wish and share
the patches I tried.
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
