| From: Sheng Yang <sheng@xxxxxxxxxxxxxxx>
| Date: Tuesday, July 13, 2010 05:53 pm
|
| On Wednesday 14 July 2010 04:41:01 Casey Leedom wrote:
| >
| > It looks like the Linux KVM kernel support code issues an FLR against
| > an Assigned Device when the device is assigned and when it's freed
| > but it's not clear when those actions are taken. For instance,
| > if a device is assigned to a VM and then the VM reboots itself, is
| > that counted as another assignment point? I.e. will KVM issue a
| > new pci_reset_function() on the device so it shows up reset in
| > the newly rebooted VM?
|
| The assignment/deassignment happened when guest was created/destroyed.
| Currently it wouldn't issue a FLR when guest reset.
Hhrrmmm, this seems like a semantic error. "Resetting" the Vm should be
morally equivalent to resetting a real physical machine. And when a real
physical machine is reset, all of its buses, etc. get reset which propagates to
Device Resets on those buses ... I think that Assigned Devices should be reset
for reboots and power off/on ...
| > And what happens if the VM OS/Driver attempts to write the PCI Pass
| > Through Device's PCI-E FLR bit? I assume that that write (and the
| > following polling reads) are trapped by the KVM code but I can't find the
| > code which implements the PCI Configuration Space emulation to see if the
| > FLR is implemented there. For instance, if I run Linux 2.6.30 in the VM
| > and my Device Driver calls pci_reset_function() in its "probe()" function
| > will that result in a Device FLR? it doesn't appear to be the case ...
|
| The PCI configuration space emulated is in QEmu rather than KVM. You can
| check qemu-kvm/hw/device-assignment.c. We didn't emulate FLR capability
| now. (OK, some other device specific reset method may involved, you can
| check pci_dev_reset())
Okay, I think that this is also going to be an issue for supporting Assigned
Devices. For PCI-E SR-IOV Virtual Functions which are assigned to a VM, they
need to be reset at reboot/power off/power on and the Configuration Space
emulation needs to support the Guest OS/VM Device Driver issuing an FLR ...
| > Note that it's impossible for a Device Driver to call
| > pci_reset_function() under Linux 2.6.31 and later
| > because a call to device_lock() was added to
| > pci_dev_reset() in chageset 8e9394ce on Feb
| > 17, 2010 by Greg Kroah-Hartman. This means
| > that a call to pci_reset_function() in a device
| > driver's "probe()" routine will result in an
| > immediate deadlock.
|
| What I saw the code is like this:
|
| static int pci_dev_reset(struct pci_dev *dev, int probe)
| {
| int rc;
|
| might_sleep();
|
| if (!probe) {
| pci_block_user_cfg_access(dev);
| /* block PM suspend, driver probe, etc. */
| device_lock(&dev->dev);
| }
| [...]
|
| So seems it's fine with _probe_ set, to use with probe() routine.
You're looking at a local routine in drivers/pci/pci.c. That routine is
called twice in pci_reset_function(). The "probe" parameter is used to indicate
whether the caller wants to "probe" for the ability to perform a PCI Function
Reset or to actually _do_ the reset. pci_reset_function() first calls
pci_dev_reset() is probe=1 and, if that returns an error code, it returns
immediately with the error. Otherwise it saves the PCI State of the device,
makes another call to pci_dev_reset() with probe=0, and then restores the
device's PCI State. Thus, this "probe" in pci_dev_reset() doesn't have anything
to do with the possibility that a device's own (pci_dev *)->driver->probe()
routine happens to be calling pci_reset_function(). Since, apparently, the
device's own ...->probe() routine is called with the device's (pci_dev *)->lock
held, a call to pci_reset_function() on itself will result in an immediate
deadlock from Linux 2.6.31 on ...
Casey
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
