On Tue, Dec 13, 2016 at 09:15:37AM -0600, Benjamin Herrenschmidt wrote: > On Tue, 2016-12-13 at 14:25 +0200, Marcel Apfelbaum wrote: > > > > Hrm, the suggestion of providing both a vanilla-PCI and PCI-E host > > > > bridge came up before. I think one of us spotted a problem with that, > > > > but I don't recall what it was now. I guess one is how libvirt would > > > > map it's stupid-fake-domain-numbers to which root bus to use. > > > > This would be a weird configuration, I never heard of something like that > > on a bare metal machine, but I never worked on pseries, who knows... > > That's the thing though, it's *not* bare metal ;-) > > On bare metal, we use the "powernv" platform for which we haven't yet upstreamed > the PCIE support, but there we have real PCIe root ports with all what's exepcted > on them. > > They come on physically different PHB, one root complex per PHB, but they are > "proper" PCIe. Hotplug is a bit weird still because it has to go through some > FW interactions as the HW doesn't use the PCIe standard slot control bits (and > our qemu model doesn't handle it yet) but otherwise it's standard. > > "pseries" on the other hand is a paravirtualized platform. It's the environment > presented to guests by our propritary hypervisor (PowerVM, aka pHyp) and > KVM/qemu. I think there's a public spec these days but to cut the story short, > it doesn't expose PCIe "properly" for bad historical reasons. > > It tends to hide the parent, ie, the root port for slots connected directly to > a PHB or the entire hierarchy from the root to (and including) the downstream > switch port for slots under as switch. > > So you end up with PCIe devices devoid of a proper "parent" which is a bit > weird. > > Hotplug is implemented using specific firmware interfaces that are identical > with pre-PCIe (ie PCI/PCI-X) systems. The FW has interface for supporting 3 > kinds of hotplug: > > - Entire PHBs > - P2P Bridges > - Individual devices on an existing PHB or P2P bridge > > In practice these days mostly the first one is used for everything under pHyp, > though I think we have a mix of 1 and 3 for KVM/qemu. Alas, no, we only have 3 on KVM/qemu. Well, possibly you could do (2) by plugging a bridge device using (3) then a device onto the bridge. Whole PHB hotplug has never been implemented for qemu/KVM.. but we probably want to. -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Attachment:
signature.asc
Description: PGP signature
-- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list
