On 3/24/22 17:02, Ray Fucillo wrote: > >> On Mar 24, 2022, at 6:41 PM, Mike Kravetz <mike.kravetz@xxxxxxxxxx> wrote: >> >> I also seem to remember thinking about the possibility of >> avoiding the synchronization if pmd sharing was not possible. That may be >> a relatively easy way to speed things up. Not sure if pmd sharing comes >> into play in your customer environments, my guess would be yes (shared >> mappings ranges more than 1GB in size and aligned to 1GB). > > Hi Mike, > > This is one very large shared memory segment allocated at database startup. It's common for it to be hundreds of GB. We allocate it with shmget() passing SHM_HUGETLB (when huge pages have been reserved for us). Not sure if that answers... Yes, so there would be shared pmds for that large shared mapping. I assume this is x86 or arm64 which are the only architectures which support shared pmds. So, the easy change of "don't take semaphore if pmd sharing is not possible" would not apply. >> Also, do you have any specifics about the regressions your customers are >> seeing? Specifically what paths are holding i_mmap_rwsem in write mode >> for long periods of time. I would expect something related to unmap. >> Truncation can have long hold times especially if there are may shared >> mapping. Always worth checking specifics, but more likely this is a general >> issue. > > We've seen the write lock originate from calling shmat(), shmdt() and process exit. We've also seen it from a fork() off of one of the processes that are attached to the shared memory segment. Some evidence suggests that fork is a more costly case. However, while there are some important places where we'd use fork(), it's more unusual because most process creation will vfork() and execv() a new database process (which then attaches with shmat()). Thanks. I will continue to look at this. A quick check of the fork code shows the semaphore held in read mode for the duration of the page table copy. -- Mike Kravetz
