Assume you have a system with quite some ZONE_MOVABLE memory (esp. in
virtualized environments), eating up a significant amount of !ZONE_MOVABLE
memory dynamically at runtime can lead to non-obvious issues. It looks like
you have plenty of free memory, but the kernel might still OOM when trying
to do kernel allocations e.g., for pagetables. With CMA we at least know
what we're dealing with - it behaves like ZONE_MOVABLE except for the owner
that can place unmovable pages there. We can use it to compute statically
the amount of ZONE_MOVABLE memory we can have in the system without doing
harm to the system.
Why would you say that secretmem allocates from !ZONE_MOVABLE?
If we put boot time reservations aside, the memory allocation for
secretmem follows the same rules as the memory allocations for any file
descriptor. That means we allocate memory with GFP_HIGHUSER_MOVABLE.
Oh, okay - I missed that! I had the impression that pages are unmovable
and allocating from ZONE_MOVABLE would be a violation of that?
After the allocation the memory indeed becomes unmovable but it's not
like we are eating memory from other zones here.
... and here you have your problem. That's a no-no. We only allow it in
very special cases where it can't be avoided - e.g., vfio having to pin
guest memory when passing through memory to VMs.
Hotplug memory, online it to ZONE_MOVABLE. Allocate secretmem. Try to
unplug the memory again -> endless loop in offline_pages().
Or have a CMA area that gets used with GFP_HIGHUSER_MOVABLE. Allocate
secretmem. The owner of the area tries to allocate memory - always
fails. Purpose of CMA destroyed.
Ideally, we would want to support page migration/compaction and allow for
allocation from ZONE_MOVABLE as well. Would involve temporarily mapping,
copying, unmapping. Sounds feasible, but not sure which roadblocks we would
find on the way.
We can support migration/compaction with temporary mapping. The first
roadblock I've hit there was that migration allocates 4K destination
page and if we use it in secret map we are back to scrambling the direct
map into 4K pieces. It still sounds feasible but not as trivial :)
That sounds like the proper way for me to do it then.
But again, there is nothing in the current form of secretmem that
prevents allocation from ZONE_MOVABLE.
Oh, there is something: That the pages are not movable.
--
Thanks,
David / dhildenb
