On Thu, Aug 19, 2021 at 11:00 AM Sierra Guiza, Alejandro (Alex) <alex.sierra@xxxxxxx> wrote: > > > On 8/18/2021 2:28 PM, Ralph Campbell wrote: > > On 8/17/21 5:35 PM, Felix Kuehling wrote: > >> Am 2021-08-17 um 8:01 p.m. schrieb Ralph Campbell: > >>> On 8/12/21 11:31 PM, Alex Sierra wrote: > >>>> From: Ralph Campbell <rcampbell@xxxxxxxxxx> > >>>> > >>>> ZONE_DEVICE struct pages have an extra reference count that > >>>> complicates the > >>>> code for put_page() and several places in the kernel that need to > >>>> check the > >>>> reference count to see that a page is not being used (gup, compaction, > >>>> migration, etc.). Clean up the code so the reference count doesn't > >>>> need to > >>>> be treated specially for ZONE_DEVICE. > >>>> > >>>> v2: > >>>> AS: merged this patch in linux 5.11 version > >>>> > >>>> v5: > >>>> AS: add condition at try_grab_page to check for the zone device type, > >>>> while > >>>> page ref counter is checked less/equal to zero. In case of device > >>>> zone, pages > >>>> ref counter are initialized to zero. > >>>> > >>>> Signed-off-by: Ralph Campbell <rcampbell@xxxxxxxxxx> > >>>> Signed-off-by: Alex Sierra <alex.sierra@xxxxxxx> > >>>> --- > >>>> arch/powerpc/kvm/book3s_hv_uvmem.c | 2 +- > >>>> drivers/gpu/drm/nouveau/nouveau_dmem.c | 2 +- > >>>> fs/dax.c | 4 +- > >>>> include/linux/dax.h | 2 +- > >>>> include/linux/memremap.h | 7 +-- > >>>> include/linux/mm.h | 13 +---- > >>>> lib/test_hmm.c | 2 +- > >>>> mm/internal.h | 8 +++ > >>>> mm/memremap.c | 68 > >>>> +++++++------------------- > >>>> mm/migrate.c | 5 -- > >>>> mm/page_alloc.c | 3 ++ > >>>> mm/swap.c | 45 ++--------------- > >>>> 12 files changed, 46 insertions(+), 115 deletions(-) > >>>> > >>> I haven't seen a response to the issues I raised back at v3 of this > >>> series. > >>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Flinux-mm%2F4f6dd918-d79b-1aa7-3a4c-caa67ddc29bc%40nvidia.com%2F&data=04%7C01%7Calex.sierra%40amd.com%7Cd2bd2d4fbf764528540908d9627e5dcd%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637649117156919772%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=P7FxYm%2BkJaCkMFa3OHtuKrPOn7SvytFRmYQdIzq7rN4%3D&reserved=0 > >>> > >>> > >>> > >>> Did I miss something? > >> I think part of the response was that we did more testing. Alex added > >> support for DEVICE_GENERIC pages to test_hmm and he ran DAX tests > >> recommended by Theodore Tso. In that testing he ran into a WARN_ON_ONCE > >> about a zero page refcount in try_get_page. The fix is in the latest > >> version of patch 2. But it's already obsolete because John Hubbard is > >> about to remove that function altogether. > >> > >> I think the issues you raised were more uncertainty than known bugs. It > >> seems the fact that you can have DAX pages with 0 refcount is a feature > >> more than a bug. > >> > >> Regards, > >> Felix > > > > Did you test on a system without CONFIG_ARCH_HAS_PTE_SPECIAL defined? > > In that case, mmap() of a DAX device will call insert_page() which calls > > get_page() which would trigger VM_BUG_ON_PAGE(). > > > > I can believe it is OK for PTE_SPECIAL page table entries to have no > > struct page or that MEMORY_DEVICE_GENERIC struct pages be mapped with > > a zero reference count using insert_pfn(). > Hi Ralph, > We have tried the DAX tests with and without CONFIG_ARCH_HAS_PTE_SPECIAL > defined. > Apparently none of the tests touches that condition for a DAX device. Of > course, > that doesn't mean it could happen. > > Regards, > Alex S. > > > > > > > I find it hard to believe that other MM developers don't see an issue > > with a struct page with refcount == 0 and mapcount == 1. > > > > I don't see where init_page_count() is being called for the > > MEMORY_DEVICE_GENERIC or MEMORY_DEVICE_PRIVATE struct pages the AMD > > driver allocates and passes to migrate_vma_setup(). > > Looks like svm_migrate_get_vram_page() needs to call init_page_count() > > instead of get_page(). (I'm looking at branch > > origin/alexsierrag/device_generic > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FRadeonOpenCompute%2FROCK-Kernel-Driver.git&data=04%7C01%7Calex.sierra%40amd.com%7Cd2bd2d4fbf764528540908d9627e5dcd%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637649117156919772%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=IXe8HP2s8x5OdJdERBkGOYJCQk3iqCu5AYkwpDL8zec%3D&reserved=0) > Yes, you're right. My bad. Thanks for catching this up. I didn't realize > I was missing > to define CONFIG_DEBUG_VM on my build. Therefore this BUG was never caught. > It worked after I replaced get_pages by init_page_count at > svm_migrate_get_vram_page. However, I don't think this is the best way > to fix it. You definitly don't want to do that. reiniting the page refcounter is wrong it should be done once. Nouveau is not a good example here. > Ideally, get_pages call should work for device pages with ref count > equal to 0 > too. Otherwise, we could overwrite refcounter if someone else is > grabbing the page > concurrently. > I was thinking to add a special condition in get_pages for dev pages. > This could > also fix the insert_page -> get_page call from a DAX device. What is the issue here exactly ? > > Regards, > Alex S. > > > > > > Also, what about the other places where is_device_private_page() is > > called? > > Don't they need to be updated to call is_device_page() instead? > > One of my goals for this patch was to remove special casing reference > > counts > > for ZONE_DEVICE pages in rmap.c, etc. > Correct, is_device_private_page is still used in rmap, memcontrol and > migrate.c files > Looks like rmap and memcontrol should be replaced by is_device_page > function. No you do not want to do that. The private case is special case for a reason. Jerome
