On 05.06.20 17:18, Ard Biesheuvel wrote:
> On Fri, 5 Jun 2020 at 17:14, Heinrich Schuchardt <xypron.glpk@xxxxxx> wrote:
>>
>> On 05.06.20 16:57, Ard Biesheuvel wrote:
>>> On Fri, 5 Jun 2020 at 16:54, Heinrich Schuchardt <xypron.glpk@xxxxxx> wrote:
>>>>
>>>> On 05.06.20 14:39, Ard Biesheuvel wrote:
>>>>> On Fri, 5 Jun 2020 at 14:20, Heinrich Schuchardt <xypron.glpk@xxxxxx> wrote:
>>>>>>
>>>>>> On 05.06.20 13:52, Ard Biesheuvel wrote:
>>>>>>> EFI on ARM only supports short descriptors, and given that it mandates
>>>>>>> that the MMU and caches are on, it is implied that booting in HYP mode
>>>>>>> is not supported.
>>>>>>>
>>>>>>> However, implementations of EFI exist (i.e., U-Boot) that ignore this
>>>>>>> requirement, which is not entirely unreasonable, given that it does
>>>>>>> not make a lot of sense to begin with.
>>>>>>
>>>>>> Hello Ard,
>>>>>>
>>>>>> thanks for investigating the differences between EDK2 and U-Boot.
>>>>>>
>>>>>> What I still do not understand is if there is a bug in U-Boot where
>>>>>> U-Boot does not conform to the UEFI specification? In this case I would
>>>>>> expect a fix in U-Boot.
>>>>>>
>>>>>
>>>>> U-Boot violates the EFI spec, yes. The spec is very clear on how the
>>>>> MMU is configured, and this rules out booting with the caches off, or
>>>>> booting in HYP mode.
>>>>>
>>>>> However, given that this is the situation today, we still need to deal
>>>>> with it on the Linux side.
>>>>> In parallel, fixing it in U-boot may be appropriate. However, I think
>>>>> the EFI spec is too detailed here - instead of 'booting at the highest
>>>>> non-secure privilege mode', it tells you which exact bits to set in
>>>>> SCTLR, which seems overzealous to me. In other words, even though it
>>>>> violates the letter of the spec, I don't mind dealing with this
>>>>> exception in the Linux side, since the requirement is somewhat
>>>>> unreasonable to begin with.
>>>>>
>>>>>> Or is it simply a deficiency of Linux that it does not properly support
>>>>>> HYP/EL2 mode on 32-bit ARM?
>>>>>>
>>>>>
>>>>> No, this is definitely not the fault of Linux.
>>>>>
>>>>>> Up to now I never experience a problem booting a 32bit board via U-Boot
>>>>>> -> GRUB-EFI -> Linux. Where did you have a problem when booting via
>>>>>> U-Boot's UEFI implementation and the current Linux kernel?
>>>>>>
>>>>>
>>>>> I haven't managed to make it fail, but my only 32-bit HYP capable
>>>>> platform is QEMU. I am not 100% convinced that the EFI+HYP+U-Boot case
>>>>> is rock solid, and I may have gotten lucky with QEMU (which uses
>>>>> emulation not virtualization when running at HYP)
>>>>>
>>>>> Do you have any A7/A15 based boards that don't print 'WARNING: Caches
>>>>> not enabled' at boot?
>>>>
>>>> Hello Ard,
>>>>
>>>> I have no board that prints this. Where did you actually see this output?
>>>>
>>>
>>> In U-boot
>>>
>>> arch/arm/lib/cache.c: * Default implementation of enable_caches()
>>> arch/arm/lib/cache.c- * Real implementation should be in platform code
>>> arch/arm/lib/cache.c- */
>>> arch/arm/lib/cache.c:__weak void enable_caches(void)
>>> arch/arm/lib/cache.c-{
>>> arch/arm/lib/cache.c- puts("WARNING: Caches not enabled\n");
>>> arch/arm/lib/cache.c-}
>>> arch/arm/lib/cache.c-
>>>
>>> The QEMU port does not override that routine. This may be the reason
>>> it doesn't work for me under KVM, but only under emulation.
>>>
>>>> The string "Caches not enabled" does not exist in Linux next-20200505
>>>> according to "git grep -n 'ache not enabled'".
>>>>
>>>> Here is some sample output for an Orange Pi PC with a quad core
>>>> Allwinner H3 Soc. "ARMv7 Processor rev 5 (v7l)" according to
>>>> /proc/cpuinfo, compatible to "arm,cortex-a7" according to the device tree.
>>>>
>>>> $ uname -m
>>>> Linux orangepi-pc 5.6.0-2-armmp-lpae #1 SMP Debian 5.6.14-1 (2020-05-23)
>>>> armv7l GNU/Linux
>>>>
>>>
>>> Could you check whether it boots in HYP mode?
>>>
>>> [ 0.381460] CPU: All CPU(s) started in SVC mode.
>>>
>>> vs
>>>
>>> [ 0.135626] CPU: All CPU(s) started in HYP mode.
>>>
>>>
>>> ?
>>>
>>
>> Booted via GRUB-EFI:
>>
>> sudo dmesg -H | grep 'started in'
>> [ +0.000017] CPU: All CPU(s) started in HYP mode.
>>
>> Booted via Linux stub:
>>
>> $ sudo dmesg -H | grep 'started in'
>> [ +0.000016] CPU: All CPU(s) started in HYP mode.
>>
>
> Thanks
>
> Which uboot config is that? Which version of enable_caches() does it use?
>
orangepi_pc_defconfig
arch/arm/mach-sunxi/board.c:350:void enable_caches(void)
There has been work on simulating ARMv7 caches on QEMU. But I guess it
has not been upstreamed.
T. V. Dung, I. Taniguchi and H. Tomiyama, "Cache Simulation for
Instruction Set Simulator QEMU," 2014 IEEE 12th International Conference
on Dependable, Autonomic and Secure Computing, Dalian, 2014, pp.
441-446, doi: 10.1109/DASC.2014.85.
https://ipsj.ixsq.nii.ac.jp/ej/?action=repository_action_common_download&item_id=103050&item_no=1&attribute_id=1&file_no=1
Best regards
Heinrich
