On 08/11/2015 10:13 PM, Alex Williamson wrote:
> On Tue, 2015-08-11 at 19:26 -0400, Laine Stump wrote:
>> (Alex - I cc'ed you because I addressed a question or two your way down
>> towards the bottom).
>>
>> On 08/11/2015 02:52 AM, Pavel Fedin wrote:
>>> Hello!
>>>
>>>> The original patches to support pcie-root severely restricted what could
>>>> plug into what because in real hardware you can't plug a PCI device into
>>>> a PCIe slot (physically it doesn't work)
>>> But how do you know whether the device is PCI or PCIe ? I don't see anything like this in the code, i see that for example "all network cards are PCI", which is, BTW, not true in the real world.
>> Two years ago when I first added support for q35-based machinetypes and
>> the first pcie controllers, I had less information than I do now. When
>> I looked in the ouput of "qemu-kvm -device ?" I saw that each device
>> listed the type of bus it connected to (PCI or ISA), and assumed that
>> even though at the time qemu didn't differentiate between PCI and PCIe
>> there, since the two things *are* different in the real world eventually
>> they likely would. I wanted the libvirt code to be prepared for that
>> eventuality. Of course every example device (except the PCIe controllers
>> themselves) ends up with the flag saying that it can connect to a PCI
>> bus, not PCIe).
>>
>> Later I was told that, unlike the real world where, if nothing else, the
>> physical slots themselves limit you, any normal PCI device in qemu could
>> be plugged into a PCI or PCIe slot. There still are several restrictions
>> though, which showed themselves as more complicated than just the naive
>> PCI vs PCIe that I originally imagined - just look at the restrictions
>> on the different PCIe controllers:
>>
>> ("pcie-sw-up-port" == "pcie-switch-upstream-port", "pcie-sw-dn-port" ==
>> "pcie-switch-downstream-port")
>>
>> name upstream downstream
>> ----------------- ----------------- -------------------
>> pcie-root none any endpoint
>> pcie-root-port
>> dmi-to-pci-bridge
>> pci-bridge
>> 31 ports NO hotplug
>>
>> dmi-to-pci-bridge pcie-root any endpoint device
>> pcie-root-port pcie-sw-up-port
>> pcie-sw-dn-port
>> NO hotplug 32 ports NO hotplug
> Hmm, pcie-sw-up-port on the downstream is a stretch here. pci-bridge
> should be allowed downstream though.
You're right, I messed up the chart. pcie-sw-up-port can only plug into
pcie-root-port or pcie-sw-dn-port. And I forgot to add in pci-bridge.
Of course my main objective was to graphically point out that you can't
just plug "anything" into "anything" :-)
>> pcie-root-port pcie-root-only any endpoint
>> NO hotplug dmi-to-pci-bridge
>> pcie-sw-up-port
>> 1 port hotpluggable
>>
>> pcie-sw-up-port pcie-root-port pcie-sw-dn-port
>> pcie-sw-dn-port 32 ports "kind of" hotpluggable
>> "kind of" hotpluggable
>>
>> pcie-sw-dn-port pcie-sw-up-port any endpoint
>> "kind of" hotplug pcie-sw-up-port
>> 1 port hotpluggable
>>
>> pci-bridge pci-root any endpoint
>> pcie-root pci-bridge
>> dmi-to-pci-bridge 32 ports hotpluggable
>> pcie-root-port
>> pcie-sw-dn-port
>> NO hotplug (now)
>>
>> So the original restrictions I placed on what could plugin where were
>> *too* restrictive for endpoint devices, but other restrictions were
>> useful, and the framework came in handy as I learned the restrictions of
>> each new pci controller model.
> System software ends up being pretty amiable as well since PCIe is
> software compatible with conventional PCI. If we have a guest-based
> IOMMU though, things could start to get interesting because the
> difference isn't so transparent. The kernel starts to care about
> whether a device is express and expects certain compatible upstream
> devices as it walks the topology. Thankfully though real hardware gets
> plenty wrong too, so we only have to be not substantially worse than
> real hardware ;)
>
--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list