We observed an issue with NXP 5.15 LTS kernel that dma_alloc_coherent()
may fail sometimes when there're multiple processes trying to allocate
CMA memory.
This issue can be very easily reproduced on MX6Q SDB board with latest
linux-next kernel by writing a test module creating 16 or 32 threads
allocating random size of CMA memory in parallel at the background.
Or simply enabling CONFIG_CMA_DEBUG, you can see endless of CMA alloc
retries during booting:
[ 1.452124] cma: cma_alloc(): memory range at (ptrval) is busy,retrying
....
(thousands of reties)
The root cause of this issue is that since commit a4efc174b382
("mm/cma.c: remove redundant cma_mutex lock"), CMA supports concurrent
memory allocation.
It's possible that the memory range process A try to alloc has already
been isolated by the allocation of process B during memory migration.
The problem here is that the memory range isolated during one allocation
by start_isolate_page_range() could be much bigger than the real size we
want to alloc due to the range is aligned to MAX_ORDER_NR_PAGES.
Taking an ARMv7 platform with 1G memory as an example, when MAX_ORDER_NR_PAGES
is big (e.g. 32M with max_order 14) and CMA memory is relatively small
(e.g. 128M), there're only 4 MAX_ORDER slot, then it's very easy that
all CMA memory may have already been isolated by other processes when
one trying to allocate memory using dma_alloc_coherent().
Since current CMA code will only scan one time of whole available CMA
memory, then dma_alloc_coherent() may easy fail due to contention with
other processes.
This patchset introduces a retry mechanism to rescan CMA bitmap for -EBUSY
error in case the target pageblock may has been temporarily isolated
by others and released later.
It also improves the CMA allocation performance by trying the next
MAX_ORDER_NR_PAGES range during reties rather than looping within the
same isolated range in small steps which wasting CPU mips.
The following test is based on linux-next: next-20211213.
Without the fix, it's easily fail.
# insmod cma_alloc.ko pnum=16
[ 274.322369] CMA alloc test enter: thread number: 16
[ 274.329948] cpu: 0, pid: 692, index 4 pages 144
[ 274.330143] cpu: 1, pid: 694, index 2 pages 44
[ 274.330359] cpu: 2, pid: 695, index 7 pages 757
[ 274.330760] cpu: 2, pid: 696, index 4 pages 144
[ 274.330974] cpu: 2, pid: 697, index 6 pages 512
[ 274.331223] cpu: 2, pid: 698, index 6 pages 512
[ 274.331499] cpu: 2, pid: 699, index 2 pages 44
[ 274.332228] cpu: 2, pid: 700, index 0 pages 7
[ 274.337421] cpu: 0, pid: 701, index 1 pages 38
[ 274.337618] cpu: 2, pid: 702, index 0 pages 7
[ 274.344669] cpu: 1, pid: 703, index 0 pages 7
[ 274.344807] cpu: 3, pid: 704, index 6 pages 512
[ 274.348269] cpu: 2, pid: 705, index 5 pages 148
[ 274.349490] cma: cma_alloc: reserved: alloc failed, req-size: 38 pages, ret: -16
[ 274.366292] cpu: 1, pid: 706, index 4 pages 144
[ 274.366562] cpu: 0, pid: 707, index 3 pages 128
[ 274.367356] cma: cma_alloc: reserved: alloc failed, req-size: 128 pages, ret: -16
[ 274.367370] cpu: 0, pid: 707, index 3 pages 128 failed
[ 274.371148] cma: cma_alloc: reserved: alloc failed, req-size: 148 pages, ret: -16
[ 274.375348] cma: cma_alloc: reserved: alloc failed, req-size: 144 pages, ret: -16
[ 274.384256] cpu: 2, pid: 708, index 0 pages 7
....
With the fix, 32 threads allocating in parallel can pass overnight
stress test.
root@imx6qpdlsolox:~# insmod cma_alloc.ko pnum=32
[ 112.976809] cma_alloc: loading out-of-tree module taints kernel.
[ 112.984128] CMA alloc test enter: thread number: 32
[ 112.989748] cpu: 2, pid: 707, index 6 pages 512
[ 112.994342] cpu: 1, pid: 708, index 6 pages 512
[ 112.995162] cpu: 0, pid: 709, index 3 pages 128
[ 112.995867] cpu: 2, pid: 710, index 0 pages 7
[ 112.995910] cpu: 3, pid: 711, index 2 pages 44
[ 112.996005] cpu: 3, pid: 712, index 7 pages 757
[ 112.996098] cpu: 3, pid: 713, index 7 pages 757
...
[41877.368163] cpu: 1, pid: 737, index 2 pages 44
[41877.369388] cpu: 1, pid: 736, index 3 pages 128
[41878.486516] cpu: 0, pid: 737, index 2 pages 44
[41878.486515] cpu: 2, pid: 739, index 4 pages 144
[41878.486622] cpu: 1, pid: 736, index 3 pages 128
[41878.486948] cpu: 2, pid: 735, index 7 pages 757
[41878.487279] cpu: 2, pid: 738, index 4 pages 144
[41879.526603] cpu: 1, pid: 739, index 3 pages 128
[41879.606491] cpu: 2, pid: 737, index 3 pages 128
[41879.606550] cpu: 0, pid: 736, index 0 pages 7
[41879.612271] cpu: 2, pid: 738, index 4 pages 144
...
v1:
https://patchwork.kernel.org/project/linux-mm/cover/20211215080242.3034856-1-aisheng.dong@xxxxxxx/
v2:
https://patchwork.kernel.org/project/linux-mm/cover/20220112131552.3329380-1-aisheng.dong@xxxxxxx/
Dong Aisheng (2):
mm: cma: fix allocation may fail sometimes
mm: cma: try next MAX_ORDER_NR_PAGES during retry
mm/cma.c | 23 +++++++++++++++++++++--
1 file changed, 21 insertions(+), 2 deletions(-)
--
2.25.1
