On 28.01.25 18:07, Juan Yescas wrote:
On Tue, Jan 28, 2025 at 1:58 AM David Hildenbrand <david@xxxxxxxxxx> wrote:
On 28.01.25 02:04, Juan Yescas wrote:
Hi LSF organizers,
I would like to continue discussing this topic with the mm community:
"CMA reservation optimizations"
Note: There is already an email in the linux-mm mailing list that is
discussing this issue. The title is:
"CMA reservations require 32MiB alignment in 16KiB page size kernels
instead of 8MiB in 4KiB page size kernel"
Background
When the drivers reserve CMA memory in 16KiB kernels, the minimum
alignment is 32 MiB as per CMA_MIN_ALIGNMENT_BYTES. However, in 4KiB
kernels, the CMA alignment is 4MiB.
I'm curious, here you say 4 MiB, above 8 MiB.
My bad, it is a typo. I meant 4 MiB.
That makes sense.
But nowadays it's usually 2 MiB (pageblock size), no?
That's right for the case when THPs are enabled in 4KiB page size configs.
#define pageblock_order MIN_T(unsigned int, HPAGE_PMD_ORDER, MAX_PAGE_ORDER)
https://elixir.bootlin.com/linux/v6.13/source/include/linux/pageblock-flags.h#L50
This evals to pageblock_order = min(21 - 12, 10) = 9
#define CMA_MIN_ALIGNMENT_PAGES pageblock_nr_pages
#define CMA_MIN_ALIGNMENT_BYTES (PAGE_SIZE * CMA_MIN_ALIGNMENT_PAGES)
https://elixir.bootlin.com/linux/v6.13/source/include/linux/cma.h#L21
CMA_MIN_ALIGNMENT_BYTES = (4096 * 2 ^ 9) = (4096 * 512) = 2097152 = 2 MiB
However, when THPs are disabled, we get:
#define pageblock_order MAX_PAGE_ORDER // 10
https://elixir.bootlin.com/linux/v6.13/source/arch/arm64/Kconfig#L1630
https://elixir.bootlin.com/linux/v6.13/source/include/linux/pageblock-flags.h#L55
CMA_MIN_ALIGNMENT_BYTES = (4096 * 2 ^ 10) = (4096 * 1024) = 4194304 = 4 MiB
Right, and it can depend on ARCH_FORCE_MAX_ORDER.
I've been wondering for a while if pageblock_order should nowadays
default to HPAGE_PMD_ORDER, with the option to make it smaller/larger
(likely smaller) -- as discussed.
As discussed, the topic you are touching on is also relevant for
virtio-mem, which can add/remove memory currently in pageblock
granularity: 512 MiB on arm64 are not particularly helpful. I think we
could support adding/removing smaller granularity, but it requires a bit
of work, and always isolating 512MiB worth of pages just to effectively
allocate e.g., 2 MiB worth of pages is rather suboptimal. Same applies
to CMA I assume.
So there is more infrastructure that could benefit from pageblocks to
rather be on the smaller side, even when hugetlb+THP might not be around
in a config.
--
Cheers,
David / dhildenb
