On Tue, Sep 3, 2024 at 8:08 PM Theodore Ts'o <tytso@xxxxxxx> wrote:
>
> On Tue, Sep 03, 2024 at 04:50:46PM +0800, Zhaoyang Huang wrote:
> > > I'd also sugest only trying to use this is the file system has
> > > journaling enabled. If the file system is an ext2 file system without
> > > a journal, there's no reason avoid using the CMA region
> > agree.
> > > assume the reason why the buffer cache is trying to use the moveable
> > > flag is because the amount of non-CMA memory might be a precious
> > > resource in some systems.
> >
> > I don't think so. All migrate type page blocks possess the same
> > position as each other as they could fallback to all migrate types
> > when current fails. I guess the purpose could be to enlarge the scope
> > of available memory as __GFP_MOVEABLE has the capability of recruiting
> > CMA.
>
> Well, I guess I'm a bit confused why the buffer cache is trying to use
> __GFP_MOVEABLE in the first place. In general CMA is to allow systems
> to be able to allocate big chunks of memory which have to be
> physically contiguous because the I/O device(s) are too primitive to
> be able to do scatter-gather, right? So why are we trying to use CMA
> eligible memory for 4k buffer cache pages? Presumably, because
> there's not enough non-CMA eligible memory?
I suppose maybe you missed the way of how CMA work as the second
client as the special fallback of MIGRATE_MOVABLE during normal
alloc_pages.(cma_alloc is the first client of CMA area)
//MIGRATE_MOVABLE->ALLOC_CMA
gfp_to_alloc_flags_cma
{
#ifdef CONFIG_CMA
if (gfp_migratetype(gfp_mask) == MIGRATE_MOVABLE)
alloc_flags |= ALLOC_CMA;
#endif
return alloc_flags;
}
//ALLOC_CMA->__rmqueue_cma_fallback
__rmqueue(struct zone *zone, unsigned int order, int migratetype,
unsigned int alloc_flags)
{
struct page *page;
if (IS_ENABLED(CONFIG_CMA)) {
if (alloc_flags & ALLOC_CMA &&
zone_page_state(zone, NR_FREE_CMA_PAGES) >
zone_page_state(zone, NR_FREE_PAGES) / 2) {
page = __rmqueue_cma_fallback(zone, order);
if (page)
return page;
}
}
page = __rmqueue_smallest(zone, order, migratetype);
...
}
>
> After all, using GFP_MOVEABLE memory seems to mean that the buffer
> cache might get thrashed a lot by having a lot of cached disk buffers
> getting ejected from memory to try to make room for some contiguous
> frame buffer memory, which means extra I/O overhead. So what's the
> upside of using GFP_MOVEABLE for the buffer cache?
To my understanding, NO. using GFP_MOVEABLE memory doesn't introduce
extra IO as they just be migrated to free pages instead of ejected
directly when they are the target memory area. In terms of reclaiming,
all migrate types of page blocks possess the same position.
>
> Just curious, because in general I'm blessed by not having to use CMA
> in the first place (not having I/O devices too primitive so they can't
> do scatter-gather :-). So I don't tend to use CMA, and obviously I'm
> missing some of the design considerations behind CMA. I thought in
> general CMA tends to used in early boot to allocate things like frame
> buffers, and after that CMA doesn't tend to get used at all? That's
> clearly not the case for you, apparently?
Yes. CMA is designed for contiguous physical memory and has been used
via cma_alloc during the whole lifetime especially on the system
without SMMU, such as DRM driver. In terms of MIGRATE_MOVABLE page
blocks, they also could have compaction path retry for many times
which is common during high-order alloc_pages.
>
> Cheers,
>
> - Ted
