[cc += Keith]
On Fri, Jul 17, 2020 at 09:02:22AM -0500, Ian May wrote:
> I do now have a "better" patch that I was going to submit to the list
> where I converted the pci_slot_mutex to a rw_semaphore. Do you see
> any potential problems with changing the lock type? I attached the
> patch if you are interested in checking it over.
The question is, if pci_slot_mutex is an rw_semaphore, can it happen
that pciehp acquires it for writing, provoking a deadlock like this:
Hotplug AER
---------------------------- ---------------------------
1 down_read(&ctrl->reset_lock)
2 down_read(&pci_slot_mutex)
3 down_write(&pci_slot_mutex)
4 down_write(&ctrl->reset_lock)
** DEADLOCK **
I think this can happen if the device inserted into the hotplug slot
contains a PCIe switch which itself has hotplug ports. That's the
case with Thunderbolt: Every Thunderbolt device contains a PCIe
switch with hotplug ports to extend the Thunderbolt chain. E.g.
the PCIe hierarchy looks like this for a Thunderbolt host controller
with a chain of two devices:
Root - Upstream - Downstream - Upstream - Downstream - Upstream - Downstream
(host ...) (1st device ...) (2nd device ...)
When a Thunderbolt device is attached, pci_slot_mutex would be taken
for writing in pci_create_slot():
pciehp_configure_device()
pci_scan_slot()
pci_scan_single_device()
pci_scan_device()
pci_setup_device()
pci_dev_assign_slot() # acquire pci_slot_mutex for reading
pci_device_add() # match_driver = false; device_add()
pci_bus_add_devices()
pci_bus_add_device() # match_driver = true; device_attach()
device_attach()
__device_attach()
__device_attach_driver()
driver_probe_device()
pcie_portdrv_probe()
pcie_port_device_register()
pcie_device_init()
device_register()
device_add()
bus_probe_device()
device_initial_probe()
__device_attach()
__device_attach_driver()
driver_probe_device()
pciehp_probe()
init_slot()
pci_hp_initialize()
pci_create_slot()
down_write(pci_slot_mutex)
(You may want to double-check that I got this right.)
In principle, Keith did the right thing to acquire pci_slot_mutex in
pci_bus_error_reset() for accessing the bus->slots list.
I need to think some more to come up with a solution for this particular
deadlock. Maybe using a klist and traversing it with klist_iter_init()
(holds a ref on each slot, allowing concurrent list access) or something
along those lines...
Thanks,
Lukas
