Re: [LSF/MM/BPF TOPIC] Migrating the un-migratable

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Jan 29, 2025 at 8:10 AM David Hildenbrand <david@xxxxxxxxxx> wrote:
>
> Hi,
>
> ___GFP_MOVABLE allocations are supposed to be movable -> migratable: the
> page allocator can place them on
> MIGRATE_CMA/ZONE_MOVABLE/MIGRATE_MOVABLE areas: areas where the
> expectation is that allocations can be migrated (somewhat reliably) to
> different memory areas on demand.
>
> Mechanisms that turn such allocations unmigratable, such as long-term
> page pinning (FOLL_LONGTERM), migrate these allocations at least out of
> MIGRATE_CMA/ZONE_MOVABLE areas first.
>
> Ideally, we'd only perform this migration if really required (e.g.,
> long-term pinning), and rather "fix" other cases to not turn allocations
> unmigratable.
>
> However, we have some rather obscure cases that can turn migratable
> allocations effectively unmigratable for a long/indeterminate time,
> possibly controlled by unprivileged user space.
>
> Possible effects include:
> * CMA allocations failing
> * Memory hotunplug not making progress
> * Memory compaction not working as expected
>
> Some cases I can fix myself [1], others are harder to tackle.
>
> As one example, in context of FUSE we recently discovered that folios
> that are under writeback cannot be migrated, and user space in control
> of when writeback will end. Something similar can happen ->readahead()
> where user space is in charge of supplying page content. Networking
> filesystems in general seem to be prone to this as well.
>
> As another example, failing to split large folios can prevent migration
> if memory is fragmented. XFS (IOMAP in general) refuses to split folios
> that are dirty [3]. Splitting of folios and page migration have a lot in
> common.
>
> This session is to collect cases that are known to be problematic, and
> to start discussing possible approaches to make some of these
> un-migratable allocations migratable, or alternative strategies to deal
> with this.
>
>
> [1] https://lkml.kernel.org/r/20250129115411.2077152-1-david@xxxxxxxxxx
> [2]
> https://lkml.kernel.org/r/CAJnrk1ZCgff6ZWmqKzBXFq5uAEbms46OexA1axWS5v-PCZFqJg@xxxxxxxxxxxxxx
> [3]
> https://lkml.kernel.org/r/4febc035-a4ff-4afe-a9a0-d127826852a9@xxxxxxxxxx
>
> --
> Cheers,
>
> David / dhildenb
>
>

We have run in to the same issues (especially the writeback one), so a
definite +1 on this topic from me.

- Frank





[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [NTFS 3]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [NTFS 3]     [Samba]     [Device Mapper]     [CEPH Development]

  Powered by Linux