On Fri, Nov 22, 2019 at 7:29 AM Marc Zyngier <maz@xxxxxxxxxx> wrote: > > Hi Daniel, > > [please use my kernel.org address, as I'm not employed by ARM anymore] > > On 2019-11-21 20:16, Daniel Verkamp wrote: > > Hi Marc, > > > > I noticed that in recent kernels (4.19 in our case), KVM no longer > > works for 32-bit processes running on arm64 machines. We > > (Crostini/crosvm, Linux VM support on Chromebooks) use this > > KVM_COMPAT > > on all arm64-based Chromebooks that support VMs, since our entire > > userspace (including crosvm) is 32-bit. > > > > I found this commit (KVM: arm64: Prevent KVM_COMPAT from being > > selected): > > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=37b65db85f9b2fc98267eee4a18d7506492e6e8c > > > > Is there any particular reason the compat ioctl is disabled (e.g. > > actual bugs/breakage), or would it be possible to re-enable > > KVM_COMPAT > > for ARM64? > > Well, it never worked the first place! :( The whole API is entirely > different, > and you should feel extremely lucky that it ever seemed to work! I hadn't looked at this before, but it seems like the Rust bindings for KVM we are using are actually generated from the aarch64 C headers (e.g. struct kvm_regs contains sp_el1, etc. rather than svc_regs): https://chromium.googlesource.com/chromiumos/platform/crosvm/+/37bd738b783053c2e94b4c215c32496bae5081ed/kvm_sys/src/aarch64/bindings.rs So we are really passing the native 64-bit structs to the 32-bit compat ioctl, which is probably bogus. I can see how this is unlikely to work in a normal C program compiled against the 32-bit ARM uapi headers. > > In my brief testing with a 4.19-stable-based kernel on a new arm64 > > board, everything seems to work fine with the commit above reverted. > > The key word here is *seems*. It may happen that the userspace ABI > somehow align for some of the basic stuff, but it will fail for > anything moderately serious (I really doubt QEMU would be able to > run at all). > > I suppose to tried this with crosvm? 32 or 64bit guest? Yes, I am able to run the full Crostini environment using 32-bit crosvm on an aarch64 Chromebook kernel based on 4.19.84 stable. The guest kernel and userspace is 64 bit in this case (we don't support 32-bit guest). I also just tried a 32-bit build of kvmtool (lkvm), since we have a convenient ebuild to cross-compile it in the Chrome OS SDK, but I wasn't able to get it to boot (it fails at a call to KVM_GET_ONE_REG). I didn't attempt to debug it, but I'm assuming it's due to the ABI mismatch you mentioned. It sounds like disabling the compat ioctl is probably the right thing to do in this case; I'll have to see if we can build crosvm as a 64-bit executable instead in the future (not sure how easily this will be given the way the Chromium OS build system works). Thanks, -- Daniel _______________________________________________ kvmarm mailing list kvmarm@xxxxxxxxxxxxxxxxxxxxx https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
