On 2/15/21 8:27 AM, Michal Hocko wrote: > On Mon 15-02-21 23:36:49, Muchun Song wrote: > [...] >>> There shouldn't be any real reason why the memory allocation for >>> vmemmaps, or handling vmemmap in general, has to be done from within the >>> hugetlb lock and therefore requiring a non-sleeping semantic. All that >>> can be deferred to a more relaxed context. If you want to make a >> >> Yeah, you are right. We can put the freeing hugetlb routine to a >> workqueue. Just like I do in the previous version (before v13) patch. >> I will pick up these patches. > > I haven't seen your v13 and I will unlikely have time to revisit that > version. I just wanted to point out that the actual allocation doesn't > have to happen from under the spinlock. There are multiple ways to go > around that. Dropping the lock would be one of them. Preallocation > before the spin lock is taken is another. WQ is certainly an option but > I would take it as the last resort when other paths are not feasible. Sorry for jumping in late, Monday was a US holiday ... IIRC, the point of moving the vmemmap allocations under the hugetlb_lock was just for simplicity. The idea was to modify the allocations to be non-blocking so that allocating pages and restoring vmemmap could be done as part of normal huge page freeing where we are holding the lock. Perhaps that is too simplistic of an approach. IMO, using the workque approach as done in previous patches introduces too much complexity. Michal did bring up the question "Do we really want to do all the vmemmap allocation (even non-blocking) and manipulation under the hugetlb lock? I'm thinking the answer may be no. For 1G pages, this will require 4094 calls to alloc_pages. Even with non-blocking calls this seems like a long time. If we are not going to do the allocations under the lock, then we will need to either preallocate or take the workqueue approach. One complication with preallocation is that we do not for sure we will be freeing the huge page to buddy until we take the hugetlb_lock. This is because the decision to free or not is based on counters protected by the lock. We could of course check counters without the lock to guess if we will be freeing the page, and then check again after acquiring the lock. This may not be too bad in the case of freeing a single page, but would become more complex when doing bulk freeing. After a little thought, the workqueue approach may even end up simpler. However, I would suggest a very simple workqueue implementation with non-blocking allocations. If we can not quickly get vmemmap pages, put the page back on the hugetlb free list and treat as a surplus page. -- Mike Kravetz
