On Sun, 1 Nov 2015, Daniel Micay wrote: > It can definitely be improved to cooperate well with THP too. I've been > following the progress, and most of the problems seem to have been with > the THP and that's a very active area of development. Seems best to deal > with that after a simple, working implementation lands. > > The best aspect of MADV_FREE is that it completely avoids page faults > when there's no memory pressure. Making use of the freed memory only > triggers page faults if the pages had to be dropped because the system > ran out of memory. It also avoids needing to zero the pages. The memory > can also still be freed at any time if there's memory pressure again > even if it's handed out as an allocation until it's actually touched. > > The call to madvise still has significant overhead, but it's much > cheaper than MADV_DONTNEED. Allocators will be able to lean on the > kernel to make good decisions rather than implementing lazy freeing > entirely on their own. It should improve performance *and* behavior > under memory pressure since allocators can be more aggressive with it > than MADV_DONTNEED. > > A nice future improvement would be landing MADV_FREE_UNDO feature to > allow an attempt to pin the pages in memory again. It would make this > work very well for implementing caches that are dropped under memory > pressure. Windows has this via MEM_RESET (essentially MADV_FREE) and > MEM_RESET_UNDO. Android has it for ashmem too (pinning/unpinning). I > think browser vendors would be very interested in it. > This sounds similar to what I was proposing to prevent thp splits when there is no memory pressure. MADV_SPLITTABLE marks ranges of memory as free and the underlying thp may be split if there is no memory pressure. Under memory pressure, it acts identical to MADV_DONTNEED. Without memory pressure, the range is enqueued on an lru for the memcg that the vma's mm owner belongs to (global for !CONFIG_MEMCG). It is also linked on a per-vma list for the range. Anytime the vma is manipulated, the MADV_SPLITTABLE ranges are also fixed up. On subsequent memory pressure, the memcg hierarchy lru list is iterated (global for !CONFIG_MEMCG) and the MADV_SPLITTABLE ranges are actually zapped (including thp split if necessary) and the memory is really freed to the system. MADV_UNSPLITTABLE marks ranges of memory that have already been freed through MADV_SPLITTABLE as being used again. If there was no memory pressure and the MADV_SPLITTABLE was simply enqueued on the lru list, it is removed from that list after the range has been zeroed with the same user-facing semantics as MADV_DONTNEED. Otherwise, nothing is done since the ptes are already zapped and we'll incur a refault. The change to tcmalloc is simple: use MADV_SPLITTABLE instead of MADV_DONTNEED when freeing memory to the system and use MADV_UNSPLITTABLE when returning memory that has been already freed to the system. This works well in experimentation when 100% of heap backed by thp with no memory pressure. This is a type of lazy-free that prevents thp memory from being split without memory pressure. I was wondering if this could become part of MADV_FREE behavior with the MADV_FREE_UNDO behavior as the equivalent to my MADV_UNSPLITTABLE. If there is no ground to be shared, mine is just implemented seperately, but I'm trying to avoid additional system calls required for malloc implemenations. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>