Hi Marc, On 5/17/23 16:12, Marc Zyngier wrote: > On Wed, 17 May 2023 09:59:45 +0100, > Eric Auger <eauger@xxxxxxxxxx> wrote: >> >> Hi Marc, >>Hi Marc, >> On 5/16/23 22:28, Marc Zyngier wrote: >>> On Tue, 16 May 2023 17:53:14 +0100, >>> Eric Auger <eauger@xxxxxxxxxx> wrote: >>>> >>>> Hi Marc, >>>> >>>> On 5/15/23 19:30, Marc Zyngier wrote: >>>>> This is the 4th drop of NV support on arm64 for this year. >>>>> >>>>> For the previous episodes, see [1]. >>>>> >>>>> What's changed: >>>>> >>>>> - New framework to track system register traps that are reinjected in >>>>> guest EL2. It is expected to replace the discrete handling we have >>>>> enjoyed so far, which didn't scale at all. This has already fixed a >>>>> number of bugs that were hidden (a bunch of traps were never >>>>> forwarded...). Still a work in progress, but this is going in the >>>>> right direction. >>>>> >>>>> - Allow the L1 hypervisor to have a S2 that has an input larger than >>>>> the L0 IPA space. This fixes a number of subtle issues, depending on >>>>> how the initial guest was created. >>>>> >>>>> - Consequently, the patch series has gone longer again. Boo. But >>>>> hopefully some of it is easier to review... >>>>> >>>>> [1] https://lore.kernel.org/r/20230405154008.3552854-1-maz@xxxxxxxxxx >>>> >>>> I have started testing this and when booting my fedora guest I get >>>> >>>> [ 151.796544] kvm [7617]: Unsupported guest sys_reg access at: >>>> 23f425fd0 [80000209] >>>> [ 151.796544] { Op0( 3), Op1( 3), CRn(14), CRm( 3), Op2( 1), func_write }, >>>> >>>> as soon as the host has kvm-arm.mode=nested >>>> >>>> This seems to be triggered very early by EDK2 >>>> (ArmPkg/Drivers/TimerDxe/TimerDxe.c). >>>> >>>> If I am not wrong this CNTV_CTL_EL0. Do you have any idea? >>> >>> So here's my current analysis: >>> >>> I assume you are running EDK2 as the L1 guest in a nested >>> configuration. I also assume that you are not running on an Apple >>> CPU. If these assumptions are correct, then EDK2 runs at vEL2, and is >>> in nVHE mode. >>> >>> Finally, I'm going to assume that your implementation has FEAT_ECV and >>> FEAT_NV2, because I can't see how it could fail otherwise. >> all the above is correct. >>> >>> In these precise conditions, KVM sets the CNTHCTL_EL2.EL1TVT bit so >>> that we can trap the EL0 virtual timer and faithfully emulate it (it >>> is otherwise written to memory, which isn't very helpful). >> >> indeed >>> >>> As it turns out, we don't handle these traps. I didn't spot it because >>> my test machines are all Apple boxes that don't have a nVHE mode, so >>> nothing on the nVHE path is getting *ANY* coverage. Hint: having >>> access to such a machine would help (shipping address on request!). >>> Otherwise, I'll eventually kill the nVHE support altogether. >>> >>> I have written the following patch, which compiles, but that I cannot >>> test with my current setup. Could you please give it a go? >> >> with the patch below, my guest boots nicely. You did it great on the 1st >> shot!!! So this fixes my issue. I will continue testing the v10. > > Thanks a lot for reporting the issue and testing my hacks. I'll > eventually fold it into the rest of the series. > > By the way, what are you using as your VMM? I'd really like to > reproduce your setup. Sorry I missed your reply. I am using libvirt + qemu (feat Miguel's RFC) and fedora L1 guest. Thanks to your fix, this boots fine. But at the moment it does not reboot and hangs in edk2 I think. Unfortunately this time I have no trace on host :-( While looking at your series I will add some traces. Eric > > Cheers, > > M. >
