On 08/08/2012 02:11 AM, Don Dutile wrote: > On 08/07/2012 12:10 PM, Jiang Liu wrote: >> From: Jiang Liu<liuj97@xxxxxxxxx> >> >> This is the second take to resolve race conditions when hot-plugging PCI >> devices/host bridges. Instead of using a globla lock to serialize all hotplug >> operations as in previous version, now we introduce a state machine and bit >> lock mechanism for PCI buses to serialize hotplug operations. For discussions >> related to previous version, please refer to: >> http://comments.gmane.org/gmane.linux.kernel.pci/15007 >> >> This patch-set is still in early stages, so sending it out just requesting >> for comments. Any comments are welcomed, especially about whether it's the >> right/suitable way to solve these race condition issues. >> >> patch 1-5: >> Preparing for coming PCI bus lock >> patch 6-7: >> Core of the new PCI bus lock mechanism. >> patch 8-13: >> Enhance PCI core to support PCI bus lock mechanism. >> patch 14-18: >> Enhance several PCI hotplug drivers to use PCI bus lock to serialize >> hotplug operations. >> patch 19-20: >> Enable PCI bus lock mechanism for x86 and IA64, still need to enable >> PCI bus lock for other archs. >> patch 21-22: >> Cleanups for unsed code. >> >> There are multiple methods to trigger PCI hotplug requests/operations >> concurrently, such as: >> 1. Sysfs interfaces exported by the PCI core subsystem >> /sys/devices/pcissss:bb/ssss:bb:dd.f/.../remove >> /sys/devices/pcissss:bb/ssss:bb:dd.f/.../rescan >> /sys/devices/pcissss:bb/ssss:bb:dd.f/.../pci_bus/ssss:bb/rescan >> /sys/bus/pci/rescan >> 2. Sysfs interfaces exported by the PCI hotplug subsystem >> /sys/bus/pci/slots/xx/power >> 3. PCI hotplug events triggered by PCI Hotplug Controllers >> 4. ACPI hotplug events for PCI host bridges >> 5. Driver binding/unbinding events >> binding/unbinding pci drivers with SR-IOV support >> > 6. PCI reset > --> a PCIe device-level reset is done by KVM when it assigns a device > to a guest. a PCI config-save before reset, and PCI config-restore after reset > is done in this case. > --> VF devices are interesting, since they are reset, then bound to > pci-stub driver. when more than 1 VF is enabled in a PF, > and several device-assignments are done simultaneously, you > get a storm of reset (save/restore pci cfg space), and pci-stub binding > (pci cfg read for resource allocation/deallocation), and depending on > the hw design: an AER caused by the FLR reset -- not suppose to, but > hw has bugs too! ;-) > PCI locking is 'challenged' in the above scenario. > > So, I ask: have you tried your patch set doing something like: > a) modprobe an SRIOV device with > 1 vf enabled > you may also have to do: > b) while assigning another SRIOV device's VF to another KVM guest > > Unfortunately, the PCI cfg-space locking, esp. on x86 (ok, I'll say it: > damn, mutually exclusive, IO-port-based cfg registers), doesn't lend itself > to this multi-task, dynamic PCI scenario. > Much less complicated on linearly-mapped, PCI-mmconf-only accesses. > > - Don Hi Don, Thanks for your comments. Haven't done such tests for SR-IOV yet. We will try to find some NICs with SR-IOV capability for testing and will send the result to you once done. Regards! Gerry -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
