On 10.11.21 15:38, Jason Gunthorpe wrote: > On Wed, Nov 10, 2021 at 02:25:50PM +0100, David Hildenbrand wrote: >> On 10.11.21 13:56, Jason Gunthorpe wrote: >>> On Wed, Nov 10, 2021 at 06:54:13PM +0800, Qi Zheng wrote: >>> >>>> In this patch series, we add a pte_refcount field to the struct page of page >>>> table to track how many users of PTE page table. Similar to the mechanism of >>>> page refcount, the user of PTE page table should hold a refcount to it before >>>> accessing. The PTE page table page will be freed when the last refcount is >>>> dropped. >>> >>> So, this approach basically adds two atomics on every PTE map >>> >>> If I have it right the reason that zap cannot clean the PTEs today is >>> because zap cannot obtain the mmap lock due to a lock ordering issue >>> with the inode lock vs mmap lock. >> >> There are different ways to zap: madvise(DONTNEED) vs >> fallocate(PUNCH_HOLE). It depends on "from where" we're actually >> comming: a process page table walker or the rmap. > > AFAIK rmap is the same issue, it can't lock the mmap_sem > >> The way locking currently works doesn't allow to remove a page table >> just by holding the mmap lock, not even in write mode. > > I'm not sure I understand this. If the goal is to free the PTE tables > then the main concern is use-after free on page table walkers (which > this series is addressing). Ignoring bugs, we have only three ways to > read the page table: Yes, use-after-free and reuse-while-freeing are the two challenges AFAIKs. > > - Fully locked. Under the PTLs (gup slow is an example) > - Semi-locked. Under the read mmap lock and no PTLs (hmm is an example) > - hw-locked. Barriered with TLB flush (gup fast is an example) Three additions as far as I can tell: 1. Fully locked currently needs the read mmap lock OR the rmap lock in read. PTLs on their own are not sufficient AFAIKT. 2. #1 and #2 can currently only walk within VMA ranges. 3. We can theoretically walk page tables outside VMA ranges with the write mmap lock, because page tables get removed with the mmap lock in read mode and heavy-weight operations (VMA layout, khugepaged) are performed under the write mmap lock. The rmap locks protect from modifications where we want to exclude rmap walkers similarly to how we grab the mmap lock in write, where the PTLs are not sufficient. See mm/mremap.c:move_ptes() as an example which performs VMA layout + page table modifications. See khugepagd which doesn't perform VMA layout modifications but page table modifications. > > #1 should be completely safe as the PTLs will protect eveything > #2 is safe so long as the write side is held during any layout changes > #3 interacts with the TLB flush, and is also safe with zap > > rmap itself is a #1 page table walker, ie it gets the PTLs under > page_vma_mapped_walk(). When you talk about PTLs, do you mean only PTE-PTLs or also PMD-PTLs? Because the PMD-PTLs re usually not taken in case we know there is a page table (nothing would currently change it without heavy locking). And if they are taken, they are only held while allocating/checking a PMDE, not while actually *using* the page table that's mapped in that entry. For example, walk_page_range() requires the mmap lock in read and grabs the PTE-PTLs. > > The sin we have comitted here is that both the mmap lock and the PTLs > are being used to protect the page table itself with a very > complicated dual semantic. > > Splitting the sleeping mmap lock into 'covers vma' and 'covers page > tables' lets us solve the lock ordering and semi-locked can become > more fully locked by the new lock, instead of by abusing mmap sem. It would still be a fairly coarse-grained locking, I am not sure if that is a step into the right direction. If you want to modify *some* page table in your process you have exclude each and every page table walker. Or did I mis-interpret what you were saying? > > I'd suggest to make this new lock a special rwsem which allows either > concurrent read access OR concurrent PTL access, but not both. This I looked into such a lock recently in similar context and something like that does not exist yet (and fairness will be challenging). You either have a single reader or multiple writer. I'd be interested if someone knows of something like that. -- Thanks, David / dhildenb
