On 03/28/2016 11:14 AM, Nikolay Borisov wrote:
Hello, On kernel 4.4 I observe that the memory gets really fragmented fairly quickly. E.g. there are no order > 4 pages even after 2 days of uptime. This leads to certain data structures on XFS (in my case order 4/order 5 allocations) not being allocated and causes the server to stall. When this happens either someone has to log on the server and manually invoke the memory compaction or plain reboot the server. Before that the server was running with the exact same workload but with 3.12.52 kernel and no such issue were observed. That is - memory was fragmented but allocation didn't fail, maybe alloc_pages_direct_compact was doing a better job? FYI the allocation is performed with GFP_KERNEL | GFP_NOFS
GFP_NOFS is indeed excluded from memory compaction in the allocation context (i.e. direct compaction).
Manual compaction usually does the job, however I'm wondering why isn't invoking __alloc_pages_direct_compact from within __alloc_pages_nodemask satisfying the request if manual compaction would do the job. Is there a difference in the efficiency of manually invoking memory compaction and the one invoked from the page allocator path?
Manual compaction via /proc is known to be safe in not holding any locks that XFS might be holding. Compaction relies on page migration and IIRC some filesystems cannot migrate dirty pages unless there's writeback, and if that writeback called back to xfs, it would be a deadlock. However, we could investigate if the async compaction would be safe.
In any case, such high-order allocations should always have an order-0 fallback. You're suggesting there's an infinite loop around the allocation attempt instead? Do you have the full backtrace?
Even an infinite loop should eventually proceed by having kswapd do the compaction work with unrestricted context. But it turns out kswapd compaction was broken. Hopefully good news is that 4.6-rc1 has kcompactd for that, which should work better. If you're able to test such experimental kernel, I would be interested to hear if it helped.
Another question for my own satisfaction - I created a kernel module which allocate pages of very high order - 8/9) then later when those pages are returned I see the number of unmovable pages increase by the amount of pages returned. So should freed pages go to the unmovable category?
Your kernel was likely doing unmovable allocations (e.g. GFP_KERNEL), so freed pages are returned to unmovable freelists. But once they merge to order-9 or higher, it doesn't really matter, as it's trivial to transform them to movable lists in response to movable allocation demand, without causing permanent fragmentation.
-- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>
-- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>