On Mon, Apr 08, 2013 at 03:32:07PM -0400, Laine Stump wrote:
> On 04/08/2013 12:48 PM, Daniel P. Berrange wrote:
> > On Mon, Apr 08, 2013 at 12:37:49PM -0400, Laine Stump wrote:
> >> I think we're starting to get closer to the concrete problem that's
> >> bothering me. As I understand it (and again - "what I understand" has
> >> repeatedly been shown to be incorrect in this thread :-):
> >>
> >> * Ihere are multiple different types of devices that provide a bus with
> >> 1 or more "slots" that PCI devices (e.g., the virtio-net-pci device, the
> >> e1000 network device, etc) can be plugged into.
> >>
> >> * In the config for those devices, there is a required (auto-generated
> >> if not explicitly provided) <address> element that indicates what
> >> controller that device is plugged into e.g.:
> >>
> >> <interface type='direct'>
> >> ...
> >> <address type='pci' domain='0' bus='0' slot='3' function='0'/>
> >> ...
> >> </interface>
> >>
> >> * domain is always hardcoded to 0, and in the past bus was also always
> >> hardcoded to 0 because until now there has only been a single place
> >> where PCI devices could be connected - the builtin pci.0 bus, which is a
> >> part of the basic "pc" (and some others) virtual machine and provides 32
> >> slots.
> >>
> >> * Now we are adding the ability to define new PCI buses, for now just a
> >> single kind - a pci-bridge controller, which itself must connect to an
> >> existing PCI slot, and provides 32 new PCI slots. But in the future
> >> there will be more different types of controllers that provide one or
> >> more PCI slots where PCI devices/controllers can be plugged in.
> >>
> >> * In these patches adding support for pci-bridge, we are making the
> >> assumption that there is a 1:1 correspondence between the "index='n'"
> >> attribute of the pci-bridge controller and the "bus='n'" attribute of
> >> the <address> element in devices that will be plugged into that
> >> controller. So for example if we have:
> >>
> >>
> >> <controller type='pci-bridge' index='1'>
> >> <address type='pci' domain='0' bus='0' slot='10' function='0'/>
> >> </controller>
> >>
> >> and then change the <interface> definition above to say "bus='1'", that
> >> interface device will plug into this new bus at slot 3.
> >>
> >> * So let's assume that we add a new controller called "dmi-to-pci-bridge":
> >>
> >> <controller type='dmi-to-pci-bridge' index='0'/>
> >>
> >> Ignoring for now the question of what address we give in the definition
> >> of *this* device (which is itself problematic - do we need a new "pcie"
> >> address type?), if some device is then defined with
> >>
> >>
> >> <address type='pci bus='0' .../>
> >>
> >> How do we differentiate between that meaning "the pci-ptp controller
> >> that is index='0'" and "the pci-bridge controller that is index='0'"? Do
> >> we need to expand our <address> element further? If, as I think you
> >> suggest, we have multiple different kinds of controllers that provide
> >> PCI slots, each with its own namespace, the current pci address element
> >> is inadequate to unambiguously describe where a pci device should be
> >> plugged in.
> > Hmm yes, you're right - as long as we only have <adress type='pci'>
> > then all <controller> elements should use type='pci' too, and we should
> > just distinguish based on the model name of the controller. So ignore
> > my previous suggestion to have 'pci-bridge' and 'pci-root' types, we
> > can only use type='pci' on <controller> elements.
>
> Okay, so that means we preserve the correlation between
>
> <controller type='pci' index='1'>
>
> and
>
> <address type='pci' bus='1' ..../>
>
>
> Should the <controller> device use, e.g. <model type='pci-bridge'/> for
> the model, as is done for <interface> devices? One notable difference is
> that in the case of <interface> (with the exception of "<model
> type='virtio'/>"), the model isn't used for anything except passing
> directly through to qemu (and very recently validating against a list of
> known interface models), while in the case of controllers with
> type='pci', different models will have different rules about what they
> can connect to and what can connect to them, and they will affect what
> is valid in other devices.
>
> An example on a "pc" machinetype that has the builtin PCI bus, one extra
> pci-pci bridge, and an interface device plugged into slot 3 of the
> pci-bridge:
>
> <controller type='pci' index='0'>
> <model type='pci-root'/> <!-- builtin pci bus -->
> </controller>
> <controller type='pci' index='1'>
> <model type='pci-bridge'/>
> </controller>
> <interface type='direct'>
> ...
> <address type='pci' bus='1' slot='3'/>
> </controller>
>
> And for a q35 machinetype that has the root pcie, an i82801b11-bridge
> connected to slot 1e of that, a pci bridge connected to slot 1 of the
> i82801b11-bridge, and an interface plugged into slot 3 of the pci-bridge:
>
> <controller type='pci' index='0'>
> <model type='pcie-root'/>
> </controller>
> <controller type='pci' index='1'>
> <model type='i82801b11-bridge'/> <!-- [*] -->
> <address type='pci' bus='0' slot='0x1e'/>
> </controller>
> <controller type='pci' index='2'>
> <model type='pci-bridge'/>
> <address type='pci' bus='1' slot='1'/>
> </controller>
> <interface type='direct'>
> ...
> <address type='pci' bus='2' slot='3'/>
> </controller>
>
> (note that controllers with model='(pci|pcie)-root' will not have any
> <address> element, because they exist in the basic machine so we don't
> need to connect them to anywhere.)
Actually I do wonder if we should reprent a PCI root as two
<controller> elements, one representing the actual PCI root
device, and the other representing the host bridge that is
built-in.
Also we should use the actual model names, not 'pci-root' or
'pcie-root' but rather i440FX for "pc" machine type, and whatever
the q35 model name is.
- One PCI root with built-in PCI bus (ie todays' setup)
<controller type="pci-root" index="0">
<model name="i440FX"/>
</controller>
<controller type="pci" index="0"> <!-- Host bridge -->
<address type='pci' domain='0' bus='0' slot='0''/>
</controller>
<interface type='direct'>
...
<address type='pci' domain='0' bus='0' slot='3'/>
</controller>
- One PCI root with built-in PCI bus and extra PCI bridge
<controller type="pci-root" index="0">
<model name="i440FX"/>
</controller>
<controller type="pci" index="0"> <!-- Host bridge -->
<address type='pci' domain='0' bus='0' slot='0'/>
</controller>
<controller type="pci" index="1"> <!-- Additional bridge -->
<address type='pci' domain='0' bus='0' slot='1'/>
</controller>
<interface type='direct'>
...
<address type='pci' domain='0' bus='1' slot='3'/>
</controller>
- One PCI root with built-in PCI bus, PCI-E bus and and extra PCI bridge
(ie possible q35 setup)
<controller type="pci-root" index="0">
<model name="i440FX"/>
</controller>
<controller type="pci" index="0"> <!-- Host bridge -->
<address type='pci' domain='0' bus='0' slot='0'/>
</controller>
<controller type="pci" index="1"> <!-- Additional bridge -->
<address type='pci' domain='0' bus='0' slot='1'/>
</controller>
<controller type="pci" index="1"> <!-- Additional bridge -->
<address type='pci' domain='0' bus='0' slot='1'/>
</controller>
<interface type='direct'>
...
<address type='pci' domain='0' bus='1' slot='3'/>
</controller>
So if we later allowed for mutiple PCI roots, then we'd have something
like
<controller type="pci-root" index="0">
<model name="i440FX"/>
</controller>
<controller type="pci-root" index="1">
<model name="i440FX"/>
</controller>
<controller type="pci" index="0"> <!-- Host bridge 1 -->
<address type='pci' domain='0' bus='0' slot='0''/>
</controller>
<controller type="pci" index="0"> <!-- Host bridge 2 -->
<address type='pci' domain='1' bus='0' slot='0''/>
</controller>
<interface type='direct'> <!-- NIC on host bridge 2 -->
...
<address type='pci' domain='1' bus='0' slot='3'/>
</controller>
NB this means that 'index' values can be reused against the
<controller>, provided they are setup on different pci-roots.
> (also note that it might happen that the bus number in libvirt's config
> will correspond to the bus numbering that shows up in the guest OS, but
> that will just be a happy coincidence)
>
> Does this make sense?
Yep, I think we're fairly close.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list