Thanks Dan, this is great. I think the changes in this series are cleaner and more understandable than the patch set I had sent earlier. However, I found a couple minor issues with this patch: On 2019-03-29 9:27 a.m., Dan Williams wrote: > static void pci_p2pdma_release(void *data) > { > struct pci_dev *pdev = data; > @@ -103,12 +110,12 @@ static void pci_p2pdma_release(void *data) > if (!pdev->p2pdma) > return; > > - wait_for_completion(&pdev->p2pdma->devmap_ref_done); > - percpu_ref_exit(&pdev->p2pdma->devmap_ref); > + /* Flush and disable pci_alloc_p2p_mem() */ > + pdev->p2pdma = NULL; > + synchronize_rcu(); > > gen_pool_destroy(pdev->p2pdma->pool); I missed this on my initial review, but it became obvious when I tried to test the series: this is a NULL dereference seeing pdev->p2pdma was set to NULL a few lines up. When I fix this by storing p2pdma in a local variable, the patch set works and never seems to crash when I hot remove p2pdma memory. > void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size) > { > - void *ret; > + void *ret = NULL; > + struct percpu_ref *ref; > > + rcu_read_lock(); > if (unlikely(!pdev->p2pdma)) > - return NULL; Using RCU here makes sense to me, however I expect we should be using the proper rcu_assign_pointer(), rcu_dereference() and __rcu tag with pdev->p2pdma. If only to better document what's being protected with the new RCU calls. Logan