On Fri, Jan 20, 2023 at 10:41:13AM +0100, Christian König wrote: > Am 20.01.23 um 10:19 schrieb Lukas Wunner: > > On surprise removal, pciehp_unconfigure_device() and acpiphp's > > trim_stale_devices() call pci_dev_set_disconnected() to mark removed > > devices as permanently offline. Thereby, the PCI core and drivers know > > to skip device accesses. > > > > However pci_dev_set_disconnected() takes the device_lock and thus waits > > for a concurrent driver bind or unbind to complete. As a result, the > > driver's ->probe and ->remove hooks have no chance to learn that the > > device is gone. > > Who is reading dev->error_state in this situation and especially do we have > the necessary read barrier in place? > > Alternatively if this is just opportunistically we should document that > somehow. dev->error_state is read by pci_dev_is_disconnected() and by various drivers. None of them has a specific read barrier AFAICS or is using the device_lock to protect read access. For pci_dev_is_disconnected(), the read is indeed opportunistic. It's an optimization that prevents the kernel from performing a lot of non-posted requests to a removed device and waiting for them to time out. Or worse, having them be received by a replacement device. > > That doesn't make any sense, so drop the device_lock and instead use > > atomic xchg() and cmpxchg() operations to update the device state. > > You use xchg() instead of WRITE_ONCE() for the memory barrier here? xchg() implies a full memory barrier to ensure that the new state is immediately visible to all CPUs. WRITE_ONCE() would be weaker, it's just a compiler barrier. The desire to immediately make the state change visible is why we absolutely do not want locking here. > > As a byproduct, an AB-BA deadlock reported by Anatoli is fixed which > > occurs on surprise removal with AER concurrently performing a bus reset. > > Well this byproduct is probably the main fix in this patch. I'm wondering > why lockdep didn't complained about that more drastically in our testing. Well I guess I could rephrase the commit message if you feel I'm burying the lede. Thanks, Lukas
