On Wed, 25 Jan 2023 at 17:32, Nathan Chancellor <nathan@xxxxxxxxxx> wrote:
>
> Hi Ard,
>
> On Wed, Jan 11, 2023 at 11:22:34AM +0100, Ard Biesheuvel wrote:
> > If we enter with the MMU on, there is no need for explicit cache
> > invalidation for stores to memory, as they will be coherent with the
> > caches.
> >
> > Let's take advantage of this, and create the ID map with the MMU still
> > enabled if that is how we entered, and avoid any cache invalidation
> > calls in that case.
> >
> > Signed-off-by: Ard Biesheuvel <ardb@xxxxxxxxxx>
> > ---
> > arch/arm64/kernel/head.S | 5 +++--
> > 1 file changed, 3 insertions(+), 2 deletions(-)
> >
> > diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
> > index c3b898efd3b5288d..d75f419206451d07 100644
> > --- a/arch/arm64/kernel/head.S
> > +++ b/arch/arm64/kernel/head.S
> > @@ -89,9 +89,9 @@
> > SYM_CODE_START(primary_entry)
> > bl record_mmu_state
> > bl preserve_boot_args
> > + bl create_idmap
> > bl init_kernel_el // w0=cpu_boot_mode
> > mov x20, x0
> > - bl create_idmap
> >
> > /*
> > * The following calls CPU setup code, see arch/arm64/mm/proc.S for
> > @@ -377,12 +377,13 @@ SYM_FUNC_START_LOCAL(create_idmap)
> > * accesses (MMU disabled), invalidate those tables again to
> > * remove any speculatively loaded cache lines.
> > */
> > + cbnz x19, 0f // skip cache invalidation if MMU is on
> > dmb sy
> >
> > adrp x0, init_idmap_pg_dir
> > adrp x1, init_idmap_pg_end
> > bl dcache_inval_poc
> > - ret x28
> > +0: ret x28
> > SYM_FUNC_END(create_idmap)
> >
> > SYM_FUNC_START_LOCAL(create_kernel_mapping)
> > --
> > 2.39.0
> >
>
> Our CI started reporting a boot failure in QEMU with defconfig +
> CONFIG_CPU_BIG_ENDIAN=y after this patch as commit 32b135a7fafe ("arm64:
> head: avoid cache invalidation when entering with the MMU on") in the
> arm64 tree (and now next-20230125).
>
> https://github.com/ClangBuiltLinux/continuous-integration2/actions/runs/4001750912/jobs/6868612292
>
> $ timeout --foreground 3m qemu-system-aarch64 \
> -cpu max,pauth-impdef=true \
> -machine virt,gic-version=max,virtualization=true \
> -kernel Image.gz \
> -append "console=ttyAMA0 earlycon" \
> -display none \
> -initrd rootfs.cpio
> -m 512m \
> -nodefaults \
> -no-reboot \
> -serial mon:stdio
> qemu-system-aarch64: terminating on signal 15 from pid 389 (timeout)
>
> defconfig is fine at the same change.
>
> There is no output, which makes sense since this is pretty early in
> boot. We are not booting via EFI, in case that matters. This does not
> appear to be a toolchain problem, as I can reproduce it with the
> kernel.org GCC toolchains.
>
Thanks for the report.
With this patch, the ID map is populated before the switch to BE mode,
and so the descriptors are written in the wrong byte order.
This should be easy to fix - I'll have a patch out shortly.
