On 03/15/23 21:03, David Hildenbrand wrote: > On 10.03.23 19:28, Stefan Roesch wrote: > > Stefan, can you do me a favor and investigate which pages we end up > deduplicating -- especially if it's mostly only the zeropage and if it's > still that significant when disabling THP? > > I'm currently investigating with some engineers on playing with enabling KSM > on some selected processes (enabling it blindly on all VMAs of that process > via madvise() ). > > One thing we noticed is that such (~50 times) 20MiB processes end up saving > ~2MiB of memory per process. That made me suspicious, because it's the THP > size. > > What I think happens is that we have a 2 MiB area (stack?) and only touch a > single page. We get a whole 2 MiB THP populated. Most of that THP is zeroes. > > KSM somehow ends up splitting that THP and deduplicates all resulting > zeropages. Thus, we "save" 2 MiB. Actually, it's more like we no longer > "waste" 2 MiB. I think the processes with KSM have less (none) THP than the > processes with THP enabled, but I only took a look at a sample of the > process' smaps so far. > > I recall that there was a proposal to split underutilized THP and free up > the zeropages (IIRC Rik was involved). > > I also recall that Mike reported memory waste due to THP. Interesting! 2MB stacks were also involved in our case. That stack would first get a write fault allocating a THP. The write fault would be followed by a mprotect(PROT_NONE) of the 4K page at the bottom of the stack to create a guard page. The mprotect would result in the THP being split resulting in 510 zero filled pages. I suppose KSM could dedup those zero pages. -- Mike Kravetz
