On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: > On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: > > The basic idea as outlined by Mel Gorman in [2] is: > > > > 1) On first fault in a sufficiently sized range, allocate a huge page > > sized and aligned block of base pages. Map the base page > > corresponding to the fault address and hold the rest of the pages in > > reserve. > > 2) On subsequent faults in the range, map the pages from the reservation. > > 3) When enough pages have been mapped, promote the mapped pages and > > remaining pages in the reservation to a huge page. > > 4) When there is memory pressure, release the unused pages from their > > reservations. > > I haven't yet read the patch in details, but I'm skeptical about the > approach in general for few reasons: > > - PTE page table retracting to replace it with huge PMD entry requires > down_write(mmap_sem). It makes the approach not practical for many > multi-threaded workloads. > > I don't see a way to avoid exclusive lock here. I will be glad to > be proved otherwise. > That problem is somewhat fundamental to the mmap_sem itself and conceivably it could be alleviated by range-locking (if that gets completed). The other thing to bear in mind is the timing. If the promotion is in-place due to reservations, there isn't the allocation overhead and the hold times *should* be short. > - The promotion will also require TLB flush which might be prohibitively > slow on big machines. > Which may be offset by either a) setting the threshold to 1 in cases where the promtotion should always be immediate or b) offset by reduced memory consumption potentially avoiding premature reclaim in others. > - Short living processes will fail to benefit from THP with the policy, > even with plenty of free memory in the system: no time to promote to THP > or, with synchronous promotion, cost will overweight the benefit. > Short-lived processes are also not going to be dominated by the TLB refill cost so I think that's somewhat unfair. Potential means of mediating this include per-task promotion thresholds via either prctl or a task-wide policy inherited across exec > The goal to reduce memory overhead of THP is admirable, but we need to be > careful not to kill THP benefit itself. The approach will reduce number of > THP mapped in the system and/or shift their allocation to later stage of > process lifetime. > While I agree with you, I also had suggested in review that the threshold initially be set to 1 so it can be experiemented with by people who are more concerned about memory consumption than reduced TLB misses. While the general idea is not free of problems, I believe they are fixable rather than fundamental. > Prove me wrong with performance data. :) > Agreed that this should be accompanied by performance data but I think I laid out a reasonable approach here. If the default is a threshold of 1 and that is shown to be performance-neutral then incremental progress can be made as opposed to an "all or nothing" approach. -- Mel Gorman SUSE Labs
