On 1/4/23 07:43, Mel Gorman wrote:
[snip]
What is of interest to Broadcom customers is to better distribute user space
accesses across each memory controller to improve the bandwidth available to
user space dominated work flows. With no ZONE_MOVABLE, the BCM7278 SoC with
1GB of memory on each memory controller will place the 1GB on the low
address memory controller in ZONE_DMA and the 1GB on the high address memory
controller in ZONE_NORMAL. With this layout movable allocation requests will
only fallback to the ZONE_DMA (low memory controller) once the ZONE_NORMAL
(high memory controller) is sufficiently depleted of free memory.
Adding ZONE_MOVABLE memory above ZONE_NORMAL with the current movablecore
behavior does not improve this situation other than forcing more kernel
allocations off of the high memory controller. User space allocations are
even more likely to be on the high memory controller.
But it's a weak promise that interleaving will happen. If only a portion
of ZONE_MOVABLE is used, it might still be all on the same channel. This
might improve over time if enough memory was used and the system was up
for long enough.
It is indeed a weak promise for user-space allocations out of
ZONE_MOVABLE, however the other consumer of the DMB region is a kernel
driver (typically a video decoder engine) which is directly tied to a
specific memory controller/DMB region. For the kernel driver using the
DMB region there is a hard guarantee from the kernel that it gets memory
from a specific PFN range mapping directly to the desired memory
controller and thus it is meeting the desired bandwidth
allocation/deadlines/bursts etc.
We care about both sides of the coin, though we acknowledge that
"controlling" where user-space allocations are coming from such that
they be steered towards a specific memory controller is a much harder
task and so having some amount of non-determinism is acceptable here.
--
Florian
