On Wed, Nov 09, 2022 at 03:23:00PM +0500, Muhammad Usama Anjum wrote: > Soft-dirty PTE bit of the memory pages can be read by using the pagemap > procfs file. The soft-dirty PTE bit for the whole memory range of the > process can be cleared by writing to the clear_refs file. There are other > methods to mimic this information entirely in userspace with poor > performance: > - The mprotect syscall and SIGSEGV handler for bookkeeping > - The userfaultfd syscall with the handler for bookkeeping Userfaultfd is definitely slow in this case because it needs the messaging roundtrip that happens in two different threads synchronously, so at least more schedule effort even than mprotect. I saw the other patch on vma merging with SOFTDIRTY, didn't look deeper there but IIUC it won't really help much if the other commit (34228d47) can't be reverted then it seems to help nothing. And, it does looks risky to revert that because in the same commit it mentioned the case where one can clear ref right before a vma merge, so definitely worth more thoughts and testings, which I agree with you. I'm thinking whether the vma issue can be totally avoided. For example by providing an async version of uffd-wp. Currently uffd-wp must be synchronous and it'll be slow but it services specific purposes. And this is definitely not the 1st time any of us thinking about uffd-wp being async, it's just that we need to solve the problem of storage on the dirty information. Actually we can also use other storage form but so far I didn't think of anything that's easy and clean. Current soft-dirty bit also has its defects (e.g. the need to take mmap lock and walk the pgtables), but that part will be the same as soft-dirty for now. Now I'm wildly thinking whether we can just reuse the soft-dirty bit in the ptes already defined. The GET interface could be similar as proposed here, or at least a separate issue. So _maybe_ we can have a feature (bound to the uffd context) for uffd that enables async uffd-wp, in which case the wr-protect fault is not sending any message anymore (nor enqueuing) but instead setting the soft-dirty then quickly resolving the write bit immediately and continue the fault. Clearing of the soft-dirty bit needs to be done in UFFDIO_WRITEPROTECT alongside of clearing uffd-wp bit. So on that part the current GET+CLEAR interface for pagemap may need to be replaced. And frankly, it feels weird to me to allow change mm layout in pagemap ioctls.. With this we can keep the pagemap interface to only fetch information, like before. A major benefit of using uffd is that uffd is by nature pte-based, so no fiddling with vma needed at all. Firstly, no need to worry about merging vmas with tons of false positives. Meanwhile, one can wr-protect in page-size granule easily. All the wr-protect is not governed by vma flag anymore but based on uffd-wp flag, so no extra overhead too on any page that the monitor is not interested. There's already infrastructure code for persisting uffd-wp bit, so it'll naturally work similarly for an async mode if to come to the world. It's just that we'll also need to consider exclusive use of the bit, so we'll need to fail clear_refs on vmas where we have VM_UFFD_WP and also the async feature enabled. I would hope that's very rare, but worth thinking about its side effect. The same will need to apply to UFFDIO_REGISTER on async wp mode when soft-dirty enabled, we'll need to bailout too. Said that, this is not a suggestion of a new design, but just something I thought about when reading this, and quickly writting this down. Thanks, -- Peter Xu
