On Mon, Apr 29, 2019 at 08:06:00AM -0700, Paul E. McKenney wrote:
> On Mon, Apr 29, 2019 at 07:49:24AM -0700, Paul E. McKenney wrote:
> > On Mon, Apr 29, 2019 at 10:19:44AM +0200, Sebastian Andrzej Siewior wrote:
> > > On 2019-04-26 06:50:58 [-0700], Paul E. McKenney wrote:
> > > > One place to look is in the summary output:
> > > >
> > > > TREE01 ------- 17540 GPs (58.4667/s) [rcu: g130629 f0x0 ]
> > > >
> > > > The "58.4667/s" is the number of grace periods per second. I would be
> > > > surprised if CONFIG_PARAVIRT_SPINLOCKS made a noticeable difference in
> > > > grace-period rate (given the natural variation), but you never know.
> > >
> > > I did four runs of the different parts of the patch:
> > > - 5.1-rc6
> > > - #1 + kvm64 CPU + some config options
> > > - #2 + tsc-deadline=on and so on (the whole line)
> > > - #3 + CONFIG_PARAVIRT_SPINLOCKS (now everything)
> > >
> > > the test command was
> > > tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 112 --duration 60 --configs "16*TREE08" --memory 4G
> > >
> > > and the results:
> > > | HEAD is now at 085b7755808a... Linux 5.1-rc6
> > > | (28.5942 +27.4658 +28.0203 +27.2061 +28.0731 +26.9078 +27.8494 + 27.3392 +26.4339 +28.025 +27.4797 +27.6775 +28.0653 +28.0742 +27.9581 +28.6508)/ 16
> > > | 27.738775
> > > |
> > > | HEAD is now at 36a12aa9761a... tune #1
> > > | (28.5761 +26.6514 +26.6989 +27.4375 +27.3442 +28.3228 +26.6353 +27.5461+28.5531 +27.7006 +27.8078 +27.9753 +27.4269 +28.0464 +27.6314 +27.8356) / 16
> > > | 27.6368375
> > > |
> > > | HEAD is now at af5cd7196436... tune #2
> > > | (28.4867 +26.3675 +27.6364 +28.3344 +27.4153 +27.9306 +27.1703 +26.8461+27.3194 +28.5486 +27.8975 +27.4356 +28.12 +28.4397 +29.0186 +26.9328 )/ 16
> > > | 27.74371875
> > > |
> > > | HEAD is now at 3701f64943f5... tune #3
> > > | (28.2431 +27.7831 +28.39 +28.2586 +27.7408 +27.9258 +26.6236 +26.7817+29.1178 +26.9564 +29.0525 +27.4258 +27.4931 +27.8928 +26.9308 +28.4833)/ 16
> > > | 27.8187
> > >
> > > This 28.… is the number of GP/s. Based on the results in looks like
> > > noise to me. Also I have no idea why you have more than twice as many
> > > GP/s as I do.
> >
> > My guess is that because you have more CPUs, the for_each_online_cpu()
> > loop takes longer on your system.
>
> OK, that is rather oversimplified, to say the least. A better way to
> put this is that the probability of some CPU holding things up is larger
> the more CPUs you have. RCU doe take explicit steps to slow down grace
> periods, but that doesn't start kicking in until 256 CPUs.
And I ended up with the following variant of your patch. If I don't
hear otherwise from you, I will assume that you are OK with it. So if
something bothers you about it, please don't suffer in silence!
Thanx, Paul
------------------------------------------------------------------------
commit 4a04229cf73ac9bc1ae15357beb32a0b37be1480
Author: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>
Date: Wed Apr 24 09:34:46 2019 +0200
rcutorture: Tweak kvm options
In one of my rcutorture tests the TSC clocksource got marked unstable
due to a large difference in the TSC value. I'm not sure if the guest
run for a long time with disabled interrupts or if the host was very
busy and didn't schedule the guest for some time.
I took a look on the qemu/KVM options and decided to update the options:
- Use kvm{32|64} as CPU. We could probably use `host' (like ARM does)
for maximum available features but since we don't run any userland I'm
not sure if it makes any difference.
- Drop the "noapic" option. There is no history why the APIC was disabled,
I see no reason for it. Once old qemu versions fade away, we can add
"x2apic=on,tsc-deadline=on,hypervisor=on,tsc_adjust=on".
- Additional config options. It ensures that the kernel knowns that it
runs as a kvm guest and can use virt devices like the kvm-clock as
clocksource. The kvm-clock was the main motivation here.
- I didn't add a random HW device. It would make the random device ready
earlier (not it doesn't complete the initialisation at all) but I
doubt that there is any need for this.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>
[ paulmck: The world is not quite ready for CONFIG_PARAVIRT_SPINLOCKS=y
and x2apic, so they are omitted for the time being. ]
Signed-off-by: Paul E. McKenney <paulmck@xxxxxxxxxxxxx>
diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
index 6bcb8b5b2ff2..c3a49fb4d6f6 100644
--- a/tools/testing/selftests/rcutorture/bin/functions.sh
+++ b/tools/testing/selftests/rcutorture/bin/functions.sh
@@ -172,7 +172,7 @@ identify_qemu_append () {
local console=ttyS0
case "$1" in
qemu-system-x86_64|qemu-system-i386)
- echo noapic selinux=0 initcall_debug debug
+ echo selinux=0 initcall_debug debug
;;
qemu-system-aarch64)
console=ttyAMA0
@@ -191,8 +191,19 @@ identify_qemu_append () {
# Output arguments for qemu arguments based on the TORTURE_QEMU_MAC
# and TORTURE_QEMU_INTERACTIVE environment variables.
identify_qemu_args () {
+ local KVM_CPU=""
+ case "$1" in
+ qemu-system-x86_64)
+ KVM_CPU=kvm64
+ ;;
+ qemu-system-i386)
+ KVM_CPU=kvm32
+ ;;
+ esac
case "$1" in
qemu-system-x86_64|qemu-system-i386)
+ echo -machine q35,accel=kvm
+ echo -cpu ${KVM_CPU}
;;
qemu-system-aarch64)
echo -machine virt,gic-version=host -cpu host
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/CFcommon b/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
index d2d2a86139db..e19a444a0684 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
+++ b/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
@@ -1,2 +1,5 @@
CONFIG_RCU_TORTURE_TEST=y
CONFIG_PRINTK_TIME=y
+CONFIG_HYPERVISOR_GUEST=y
+CONFIG_PARAVIRT=y
+CONFIG_KVM_GUEST=y
