Hi,
I am investigating a data abort affecting the barebox bootloader built for aarch64
that only manifests with qemu-system-aarch64 -enable-kvm.
The issue happens when using the post-indexed form of LDR on a MMIO address:
ldr x0, =0x9000fe0 // MMIO address
ldr w1, [x0], #4 // data abort, but only with -enable-kvm
Here's a full example to reproduce this:
start:
mov w10, 'o'
bl putch
bl amba_device_read_pid
mov w10, 'k'
bl putch
ret
amba_device_read_pid:
ldr x0, =0x9000fe0
ldr w1, [x0], #4 // data abort unless #4 is removed or KVM disabled
ret
putch:
ldr x8, =0x9000018
1:
/* Wait until there is space in the FIFO */
ldr w9, [x8]
tbnz w9, #5, 1b
/* Send the character */
mov x9, #0x9000000
str w10, [x9]
2:
/* Wait to make sure it hits the line, in case we die too soon. */
ldr w9, [x8]
tbnz w9, #5, 2b
ret
It assumes 0x9000000-0x9000fff is a PL011 UART as is the case on the Virt platform.
It will print an 'o', try to access 0x9000fe0, which contains the first byte
of the AMBA Product ID and then print a 'k' using the same PL011.
To build:
aarch64-linux-gnu-as reproducer.S -o reproducer.o
aarch64-linux-gnu-objcopy -O binary reproducer.o reproducer.bin
To run (whether -bios or -kernel doesn't seem to matter):
taskset -a --cpu-list 2-5 qemu-system-aarch64 -bios reproducer.bin -machine virt,secure=off \
-cpu max -m 1024M -nographic -serial mon:stdio -trace file=/dev/null
When run _without_ kvm, this will output:
ok
When run with -enable-kvm, this will trigger a data abort in amba_device_read_pid:
o
The data abort can also be avoided by removing the post index-increment:
-ldr w1, [x0], #4
+ldr w1, [x0]
This doesn't introduce a functional difference, because x0 isn't used again anyway,
but it makes the code work under KVM.
I am using debian-arm64 Bookworm on an Amlogic A311D (4x CA72, 2x CA53) with:
QEMU emulator version 9.1.50 (v9.1.0-704-g423be09ab9)
Linux 6.11-arm64 #1 SMP Debian 6.11-1~exp1 (2024-09-19) aarch64 GNU/Linux
This issue was first noticed by openembedded-core CI while testing this patch series
adding support for testing the barebox and U-Boot bootloaders:
https://lore.kernel.org/all/ee536d88a5b6468b20e37f3daabe4aa63047d1ad.camel@xxxxxxxxxxxxxx/
AFAICS, the U-Boot breakage has the same root cause as barebox', except that it's
a str instruction that has the post-increment and the PCI MMIO region is what's
being accessed.
I haven't been successful in getting QEMU/GDB to trap on data aborts, so here's an
excerpt from my original barebox stack trace instead that sent me down the rabbit
hole:
DABT (current EL) exception (ESR 0x96000010) at 0x0000000009030fe0
elr: 000000007fb2221c lr : 000000007fb22250
[...]
Call trace:
[<7fb2221c>] (amba_device_get_pid.constprop.0.isra.0+0x10/0x34) from [<7fb01e3c>] (start_barebox+0x88/0xb4)
[...]
This looks pretty much like a bug to me, but then I would have expected more
software to be affected by this.. Maybe it's memory mapping related?
I only tested with either MMU disabled or MMIO region mapped strongly-ordered.
Please let me know if there's further information I can provide.
Thanks,
Ahmad
--
Pengutronix e.K. | |
Steuerwalder Str. 21 | http://www.pengutronix.de/ |
31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
