|
Hi
Everyone, I’ currently testing KVM arm realtime performance on a hikey960 board. My test benchmark is cyclictest to measure thread wake up latency both on Host linux OS and KVM Guest linux OS. Host OS: hikey960:/mnt/debian/usr/src/linux# cyclictest -p 99 -t 4 -m
-n -a 0-3 -l 100000 # /dev/cpu_dma_latency set to 0us WARN: Running on unknown kernel version...YMMV policy: fifo: loadavg: 0.00 0.00 0.00 1/165 3270 T: 0 ( 3266)
P:99 I:1000 C: 100000 Min: 4 Act: 15 Avg: 15 Max: 139 T: 1 ( 3267)
P:99 I:1500 C: 66736 Min: 4 Act: 15 Avg: 15 Max: 239 T: 2 ( 3268)
P:99 I:2000 C: 50051 Min: 4 Act: 19 Avg: 15 Max: 43 T: 3 ( 3269)
P:99 I:2500 C: 40039 Min: 5 Act: 15 Avg: 16 Max: 74 Guest OS: root@genericarmv8:~# cyclictest -p 99 -t 4 -m
-n -a 0-3 -l 100000 # /dev/cpu_dma_latency set to 0us WARN: Running on unknown kernel version...YMMV policy: fifo: loadavg: 0.13 0.05 0.01 1/70 293 T: 0 ( 290)
P:99 I:1000 C: 100000 Min: 7 Act: 44 Avg: 85 Max: 16111 T: 1 ( 291)
P:99 I:1500 C: 66665 Min: 7 Act: 81 Avg: 90 Max: 15306 T: 2 ( 292)
P:99 I:2000 C: 49995 Min: 7 Act: 88 Avg: 87 Max: 16703 T: 3 ( 293)
P:99 I:2500 C: 39992 Min: 8 Act: 72 Avg: 97 Max: 14976 RT performance on KVM guest OS is poor compared to that on host OS. The average wake up latency is about 6 – 7 times on Guest OS vs on Host OS. I’ve tried some configurations to improve RT in KVM, like: 1 Can be combined with CPU isolation 2 Host OS and Guest OS use RT preempt kernel 3 Host CPU avoid frequency change 4 Configure NO_HZ_FULL for Guest OS There could be a little improvement after apply above configuration, but the RT performance is still very poor. 5 Guest OS use idle poll instead of WFI to avoid trap and switch out diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 2dc0f84..53aef78 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -83,7 +83,7 @@ void arch_cpu_idle(void) * tricks */ trace_cpu_idle_rcuidle(1, smp_processor_id()); - cpu_do_idle(); + cpu_relax(); local_irq_enable(); trace_cpu_idle_rcuidle(PWR_EVENT_EXIT, smp_processor_id()); } root@genericarmv8:~# cyclictest -p 99 -t 4 -m -n -l 100000 # /dev/cpu_dma_latency set to 0us WARN: Running on unknown kernel version...YMMV policy: fifo: loadavg: 0.07 0.03 0.00 1/99 328 T: 0 ( 325) P:99 I:1000 C: 100000 Min: 3 Act: 6 Avg: 13 Max: 4999 T: 1 ( 326) P:99 I:1500 C: 66659 Min: 5 Act: 7 Avg: 14 Max: 3449 T: 2 ( 327) P:99 I:2000 C: 49989 Min: 4 Act: 7 Avg: 9 Max: 11471 T: 3 ( 328) P:99 I:2500 C: 39986 Min: 4 Act: 14 Avg: 14 Max: 11253 The method 5 can improve Guest OS RT performance a lot, the average thread wake up latency on Guest OS is almost same as its on Host OS, but the Max wake up latency is still very poor. Anyone has any idea to improve RT performance on KVM Guest OS? Although method 5 can improve RT performance on Guest OS a lot, I think it is not good idea. Thanks, Steven |
_______________________________________________ kvmarm mailing list kvmarm@xxxxxxxxxxxxxxxxxxxxx https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
