On Mon, Nov 02, 2020 at 06:51:09PM +0100, David Hildenbrand wrote: > > > Assume you have a system with quite some ZONE_MOVABLE memory (esp. in > > > virtualized environments), eating up a significant amount of !ZONE_MOVABLE > > > memory dynamically at runtime can lead to non-obvious issues. It looks like > > > you have plenty of free memory, but the kernel might still OOM when trying > > > to do kernel allocations e.g., for pagetables. With CMA we at least know > > > what we're dealing with - it behaves like ZONE_MOVABLE except for the owner > > > that can place unmovable pages there. We can use it to compute statically > > > the amount of ZONE_MOVABLE memory we can have in the system without doing > > > harm to the system. > > > > Why would you say that secretmem allocates from !ZONE_MOVABLE? > > If we put boot time reservations aside, the memory allocation for > > secretmem follows the same rules as the memory allocations for any file > > descriptor. That means we allocate memory with GFP_HIGHUSER_MOVABLE. > > Oh, okay - I missed that! I had the impression that pages are unmovable and > allocating from ZONE_MOVABLE would be a violation of that? > > > After the allocation the memory indeed becomes unmovable but it's not > > like we are eating memory from other zones here. > > ... and here you have your problem. That's a no-no. We only allow it in very > special cases where it can't be avoided - e.g., vfio having to pin guest > memory when passing through memory to VMs. > > Hotplug memory, online it to ZONE_MOVABLE. Allocate secretmem. Try to unplug > the memory again -> endless loop in offline_pages(). > > Or have a CMA area that gets used with GFP_HIGHUSER_MOVABLE. Allocate > secretmem. The owner of the area tries to allocate memory - always fails. > Purpose of CMA destroyed. > > > > > > Ideally, we would want to support page migration/compaction and allow for > > > allocation from ZONE_MOVABLE as well. Would involve temporarily mapping, > > > copying, unmapping. Sounds feasible, but not sure which roadblocks we would > > > find on the way. > > > > We can support migration/compaction with temporary mapping. The first > > roadblock I've hit there was that migration allocates 4K destination > > page and if we use it in secret map we are back to scrambling the direct > > map into 4K pieces. It still sounds feasible but not as trivial :) > > That sounds like the proper way for me to do it then. Although migration of secretmem pages sounds feasible now, there maybe other issues I didn't see because I'm not very familiar with migration/compaction code. I've looked again at CMA and I'm inclined to agree with you that using CMA for secretmem allocations could be the right thing. -- Sincerely yours, Mike.
