On 2021-07-15 10:44, Qu Wenruo wrote:
On 2021/7/15 下午5:28, Robin Murphy wrote:
On 2021-07-15 09:55, Qu Wenruo wrote:
Hi,
Recently I'm playing around the Nvidia Xavier AGX board, which has
VHE extension support.
In theory, considering the CPU and memory, it should be pretty
powerful compared to boards like RPI CM4.
But to my surprise, KVM runs pretty poor on Xavier.
Just booting the edk2 firmware could take over 10s, and 20s to fully
boot the kernel.
Even my VM on RPI CM4 has way faster boot time, even just running on
PCIE2.0 x1 lane NVME, and just 4 2.1Ghz A72 core.
This is definitely out of my expectation, I double checked to be sure
that it's running in KVM mode.
But further digging shows that, since Xavier AGX CPU supports VHE,
kvm is running in VHE mode other than HYP mode on CM4.
Is there anyway to manually disable VHE mode to test the more common
HYP mode on Xavier?
According to kernel-parameters.txt, "kvm-arm.mode=nvhe" (or its
low-level equivalent "id_aa64mmfr1.vh=0") on the command line should
do that.
Thanks for this one, I stupidly only searched modinfo of kvm, and didn't
even bother to search arch/arm64/kvm...
However I'd imagine the discrepancy is likely to be something more
fundamental to the wildly different microarchitectures. There's
certainly no harm in giving non-VHE a go for comparison, but I
wouldn't be surprised if it turns out even slower...
You're totally right, with nvhe mode, it's still the same slow speed.
BTW, what did you mean by the "wildly different microarch"?
Is ARMv8.2 arch that different from ARMv8 of RPI4?
I don't mean Armv8.x architectural features, I mean the actual
implementation of NVIDIA's Carmel core is very, very different from
Cortex-A72 or indeed our newer v8.2 Cortex-A designs.
And any extra methods I could try to explore the reason of the slowness?
I guess the first check would be whether you're trapping and exiting the
VM significantly more. I believe there are stats somewhere, but I don't
know exactly where, sorry - I know very little about actually *using* KVM :)
If it's not that, then it might just be that EDK2 is doing a lot of
cache maintenance or system register modification or some other
operation that happens to be slower on Carmel compared to Cortex-A72.
Robin.
At least RPI CM4 is beyond my expectation and is working pretty fine.
Thanks,
Qu
Robin.
BTW, this is the dmesg related to KVM on Xavier, running v5.13
upstream kernel, with 64K page size:
[ 0.852357] kvm [1]: IPA Size Limit: 40 bits
[ 0.857378] kvm [1]: vgic interrupt IRQ9
[ 0.862122] kvm: pmu event creation failed -2
[ 0.866734] kvm [1]: VHE mode initialized successfully
While on CM4, the host runs v5.12.10 upstream kernel (with downstream
dtb), with 4K page size:
[ 1.276818] kvm [1]: IPA Size Limit: 44 bits
[ 1.278425] kvm [1]: vgic interrupt IRQ9
[ 1.278620] kvm [1]: Hyp mode initialized successfully
Could it be the PAGE size causing problem?
Thanks,
Qu
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@xxxxxxxxxxxxxxxxxxx
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
_______________________________________________
kvmarm mailing list
kvmarm@xxxxxxxxxxxxxxxxxxxxx
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
