On Thu, Oct 22, 2020 at 09:42:11AM -0700, Mike Kravetz wrote: > On 10/21/20 7:33 PM, Roman Gushchin wrote: > > On Wed, Oct 21, 2020 at 05:15:53PM -0700, Mike Kravetz wrote: > >> On 10/16/20 3:52 PM, Roman Gushchin wrote: > >>> This small patchset makes cma_release() non-blocking and simplifies > >>> the code in hugetlbfs, where previously we had to temporarily drop > >>> hugetlb_lock around the cma_release() call. > >>> > >>> It should help Zi Yan on his work on 1 GB THPs: splitting a gigantic > >>> THP under a memory pressure requires a cma_release() call. If it's > >>> a blocking function, it complicates the already complicated code. > >>> Because there are at least two use cases like this (hugetlbfs is > >>> another example), I believe it's just better to make cma_release() > >>> non-blocking. > >>> > >>> It also makes it more consistent with other memory releasing functions > >>> in the kernel: most of them are non-blocking. > >> > >> Thanks for looking into this Roman. > > > > Hi Mike, > > > >> > >> I may be missing something, but why does cma_release have to be blocking > >> today? Certainly, it takes the bitmap in cma_clear_bitmap and could > >> block. However, I do not see why cma->lock has to be a mutex. I may be > >> missing something, but I do not see any code protected by the mutex doing > >> anything that could sleep? > >> > >> Could we simply change that mutex to a spinlock? > > > > I actually have suggested it few months ago, but the idea was > > opposed by Joonsoo: https://lkml.org/lkml/2020/4/3/12 . > > > > The time of a bitmap operation is definitely not an issue in my context, > > but I can't speak for something like an embedded/rt case. > > > > I wonder if it may be time to look into replacing the cma area bitmap > with some other data structure? Joonsoo was concerned about the time > required to traverse the bitmap for an 8GB area. With new support for > allocating 1GB hugetlb pages from cma, I can imagine someone setting > up a cma area that is hundreds of GB if not TB in size. It is going > take some time to traverse a bitmap describing such a huge area. If the cma area is used exclusively for 1 GB allocations, the bitmap can have only 1 bit per GB, so it shouldn't be a big problem. Long-term I have some hopes to be able to allocate 1 GB pages without a need to reserve a cma area: we can try to group pages based on their mobility on a 1 GB scale, so that all non-movable pages will reside in few 1 GB blocks. I'm looking into that direction, but don't have any results yet. If this idea fails and we'll end up allocating a large cma area unconditionally and shrink it on demand (I think Rik suggested something like this), replacing the bitmap with something else sounds like a good idea to me. As now, I want to unblock Zi Yan on his work on 1 GB THPs, so maybe we can go with introducing cma_release_nowait(), as I suggested in the other e-mail in this thread? Do you have any objections? Thanks!
