On Wed, Apr 12, 2023 at 11:18:31AM +1000, Alistair Popple wrote: > Device exclusive page table entries are used to prevent CPU access to a > page whilst it is being accessed from a device. Typically this is used to > implement atomic operations when the underlying bus does not support > atomic access. When a CPU thread encounters a device exclusive entry it > locks the page and restores the original entry after calling mmu notifiers > to signal drivers that exclusive access is no longer available. > > The device exclusive entry holds a reference to the page making it safe to > access the struct page whilst the entry is present. However the fault > handling code does not hold the PTL when taking the page lock. This means > if there are multiple threads faulting concurrently on the device > exclusive entry one will remove the entry whilst others will wait on the > page lock without holding a reference. > > This can lead to threads locking or waiting on a folio with a zero > refcount. Whilst mmap_lock prevents the pages getting freed via munmap() > they may still be freed by a migration. This leads to warnings such as > PAGE_FLAGS_CHECK_AT_FREE due to the page being locked when the refcount > drops to zero. > > Fix this by trying to take a reference on the folio before locking it. > The code already checks the PTE under the PTL and aborts if the entry is > no longer there. It is also possible the folio has been unmapped, freed > and re-allocated allowing a reference to be taken on an unrelated folio. > This case is also detected by the PTE check and the folio is unlocked > without further changes. > > Link: https://lkml.kernel.org/r/20230330012519.804116-1-apopple@xxxxxxxxxx > Fixes: b756a3b5e7ea ("mm: device exclusive memory access") > Signed-off-by: Alistair Popple <apopple@xxxxxxxxxx> > Reviewed-by: Ralph Campbell <rcampbell@xxxxxxxxxx> > Reviewed-by: John Hubbard <jhubbard@xxxxxxxxxx> > Acked-by: David Hildenbrand <david@xxxxxxxxxx> > Cc: Matthew Wilcox (Oracle) <willy@xxxxxxxxxxxxx> > Cc: Christoph Hellwig <hch@xxxxxxxxxxxxx> > Cc: <stable@xxxxxxxxxxxxxxx> > Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> > (cherry picked from commit 7c7b962938ddda6a9cd095de557ee5250706ea88) > --- > mm/memory.c | 16 +++++++++++++++- > 1 file changed, 15 insertions(+), 1 deletion(-) All backports now queued up, thanks. greg k-h
