On Thursday 15 July 2010 02:01:29 Casey Leedom wrote:
> | From: Sheng Yang <sheng@xxxxxxxxxxxxxxx>
> | Date: Tuesday, July 13, 2010 05:53 pm
(Please use reply to all next time.)
> |
> | On Wednesday 14 July 2010 04:41:01 Casey Leedom wrote:
> | > It looks like the Linux KVM kernel support code issues an FLR against
> | >
> | > an Assigned Device when the device is assigned and when it's freed
> | > but it's not clear when those actions are taken. For instance,
> | > if a device is assigned to a VM and then the VM reboots itself, is
> | > that counted as another assignment point? I.e. will KVM issue a
> | > new pci_reset_function() on the device so it shows up reset in
> | > the newly rebooted VM?
> |
> | The assignment/deassignment happened when guest was created/destroyed.
> | Currently it wouldn't issue a FLR when guest reset.
> Hhrrmmm, this seems like a semantic error. "Resetting" the Vm should be
> morally equivalent to resetting a real physical machine. And when a real
> physical machine is reset, all of its buses, etc. get reset which
> propagates to Device Resets on those buses ... I think that Assigned
> Devices should be reset for reboots and power off/on ...
Yes, it should be done when reset. And power on/off should be there, because that's
means the create and destroy the guest.
>
> | > And what happens if the VM OS/Driver attempts to write the PCI Pass
> | >
> | > Through Device's PCI-E FLR bit? I assume that that write (and the
> | > following polling reads) are trapped by the KVM code but I can't find
> | > the code which implements the PCI Configuration Space emulation to see
> | > if the FLR is implemented there. For instance, if I run Linux 2.6.30
> | > in the VM and my Device Driver calls pci_reset_function() in its
> | > "probe()" function will that result in a Device FLR? it doesn't appear
> | > to be the case ...
> |
> | The PCI configuration space emulated is in QEmu rather than KVM. You can
> | check qemu-kvm/hw/device-assignment.c. We didn't emulate FLR capability
> | now. (OK, some other device specific reset method may involved, you can
> | check pci_dev_reset())
>
> Okay, I think that this is also going to be an issue for supporting
> Assigned Devices. For PCI-E SR-IOV Virtual Functions which are assigned
> to a VM, they need to be reset at reboot/power off/power on and the
> Configuration Space emulation needs to support the Guest OS/VM Device
> Driver issuing an FLR ...
You can add the FLR support for your device if you need to call it in the guest.
But I don't quite understand why you need to call it in the guest. KVM should has
already done that, and it's not necessary for native device.
>
> | > Note that it's impossible for a Device Driver to call
> | > pci_reset_function() under Linux 2.6.31 and later
> | > because a call to device_lock() was added to
> | > pci_dev_reset() in chageset 8e9394ce on Feb
> | > 17, 2010 by Greg Kroah-Hartman. This means
> | > that a call to pci_reset_function() in a device
> | > driver's "probe()" routine will result in an
> | > immediate deadlock.
> |
> | What I saw the code is like this:
> |
> | static int pci_dev_reset(struct pci_dev *dev, int probe)
> | {
> |
> | int rc;
> |
> | might_sleep();
> |
> | if (!probe) {
> |
> | pci_block_user_cfg_access(dev);
> | /* block PM suspend, driver probe, etc. */
> | device_lock(&dev->dev);
> |
> | }
> |
> | [...]
> |
> | So seems it's fine with _probe_ set, to use with probe() routine.
>
> You're looking at a local routine in drivers/pci/pci.c. That routine is
> called twice in pci_reset_function(). The "probe" parameter is used to
> indicate whether the caller wants to "probe" for the ability to perform a
> PCI Function Reset or to actually _do_ the reset. pci_reset_function()
> first calls pci_dev_reset() is probe=1 and, if that returns an error code,
> it returns immediately with the error. Otherwise it saves the PCI State
> of the device, makes another call to pci_dev_reset() with probe=0, and
> then restores the device's PCI State. Thus, this "probe" in
> pci_dev_reset() doesn't have anything to do with the possibility that a
> device's own (pci_dev *)->driver->probe() routine happens to be calling
> pci_reset_function(). Since, apparently, the device's own ...->probe()
> routine is called with the device's (pci_dev *)->lock held, a call to
> pci_reset_function() on itself will result in an immediate deadlock from
> Linux 2.6.31 on ...
So you want to issue FLR(rather than probing the feature) when driver->probe()
called? Current seems KVM is the only user of pci_reset_function(), so I think we
can update it after your driver posted to the mailing list. Also I am not sure why
you want to reset function when probing. KVM should cover them all I think.
--
regards
Yang, Sheng
>
> Casey
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
