On Thu, 3 Nov 2022 00:13:16 +0200 Amnon Ilan <ailan@xxxxxxxxxx> wrote: > On Wed, Nov 2, 2022 at 6:47 PM Laine Stump <laine@xxxxxxxxxx> wrote: > > > On 11/2/22 11:58 AM, Igor Mammedov wrote: > > > On Wed, 2 Nov 2022 15:20:39 +0000 > > > Daniel P. Berrangé <berrange@xxxxxxxxxx> wrote: > > > > > >> On Wed, Nov 02, 2022 at 04:08:43PM +0100, Igor Mammedov wrote: > > >>> On Wed, 2 Nov 2022 10:43:10 -0400 > > >>> Laine Stump <laine@xxxxxxxxxx> wrote: > > >>> > > >>>> On 11/1/22 7:46 AM, Igor Mammedov wrote: > > >>>>> On Mon, 31 Oct 2022 14:48:54 +0000 > > >>>>> Daniel P. Berrangé <berrange@xxxxxxxxxx> wrote: > > >>>>> > > >>>>>> On Mon, Oct 31, 2022 at 04:32:27PM +0200, Edward Haas wrote: > > >>>>>>> Hi Igor and Laine, > > >>>>>>> > > >>>>>>> I would like to revive a 2 years old discussion [1] about > > consistent network > > >>>>>>> interfaces in the guest. > > >>>>>>> > > >>>>>>> That discussion mentioned that a guest PCI address may change in > > two cases: > > >>>>>>> - The PCI topology changes. > > >>>>>>> - The machine type changes. > > >>>>>>> > > >>>>>>> Usually, the machine type is not expected to change, especially if > > one > > >>>>>>> wants to allow migrations between nodes. > > >>>>>>> I would hope to argue this should not be problematic in practice, > > because > > >>>>>>> guest images would be made per a specific machine type. > > >>>>>>> > > >>>>>>> Regarding the PCI topology, I am not sure I understand what changes > > >>>>>>> need to occur to the domxml for a defined guest PCI address to > > change. > > >>>>>>> The only think that I can think of is a scenario where > > hotplug/unplug is > > >>>>>>> used, > > >>>>>>> but even then I would expect existing devices to preserve their > > PCI address > > >>>>>>> and the plug/unplug device to have a reserved address managed by > > the one > > >>>>>>> acting on it (the management system). > > >>>>>>> > > >>>>>>> Could you please help clarify in which scenarios the PCI topology > > can cause > > >>>>>>> a mess to the naming of interfaces in the guest? > > >>>>>>> > > >>>>>>> Are there any plans to add the acpi_index support? > > >>>>>> > > >>>>>> This was implemented a year & a half ago > > >>>>>> > > >>>>>> https://libvirt.org/formatdomain.html#network-interfaces > > >>>>>> > > >>>>>> though due to QEMU limitations this only works for the old > > >>>>>> i440fx chipset, not Q35 yet. > > >>>>> > > >>>>> Q35 should work partially too. In its case acpi-index support > > >>>>> is limited to hotplug enabled root-ports and PCIe-PCI bridges. > > >>>>> One also has to enable ACPI PCI hotplug (it's enled by default > > >>>>> on recent machine types) for it to work (i.e.it's not supported > > >>>>> in native PCIe hotplug mode). > > >>>>> > > >>>>> So if mgmt can put nics on root-ports/bridges, then acpi-index > > >>>>> should just work on Q35 as well. > > >>>> > > >>>> With only a few exceptions (e.g. the first ich9 audio device, which is > > >>>> placed directly on the root bus at 00:1B.0 because that is where the > > >>>> ich9 audio device is located on actual Q35 hardware), libvirt will > > >>>> automatically put all PCI devices (including network interfaces) on a > > >>>> pcie-root-port. > > >>>> > > >>>> After seeing reports that "acpi index doesn't work with Q35 > > >>>> machinetypes" I just assumed that was correct and didn't try it. But > > >>>> after seeing the "should work partially" statement above, I tried it > > >>>> just now and an <interface> of a Q35 guest that had its PCI address > > >>>> auto-assigned by libvirt (and so was placed on a pcie-root-port)m and > > >>>> had <acpi index='4'/> was given the name "eno4". So what exactly is it > > >>>> that *doesn't* work? > > >>> > > >>> From QEMU side: > > >>> acpi-index requires: > > >>> 1. acpi pci hotplug enabled (which is default on relatively new q35 > > machine types) > > >>> 2. hotpluggble pci bus (root-port, various pci bridges) > > >>> 3. NIC can be cold or hotplugged, guest should pick up acpi-index of > > the device > > >>> currently plugged into slot > > >>> what doesn't work: > > >>> 1. device attached to host-bridge directly (work in progress) > > >>> (q35) > > >>> 2. devices attached to any PXB port and any hierarchy hanging of it > > (there are not plans to make it work) > > >>> (q35, pc) > > >> > > >> I'd say this is still a relatively important, as the PXBs are needed > > >> to create a NUMA placement aware topology for guests, and I'd say it > > >> is undesirable to loose acpi-index if a guest is updated to be NUMA > > >> aware, or if a guest image can be deployed in either normal or NUMA > > >> aware setups. ... > How big of a project would it be to enable ACPI-indexing/hotplug with PXB? > Since native PCI was improved, we can still compromise on switching to > native-PCI-hotplug when PXB is required (and no fixed indexing) My guesstimate would be it's not terribly difficult. Maybe we could even marry native hotplug & acpi-index after the later is decoupled from ACPI PCI hotplug as much as possible. > > Thanks, > Amnon > > > > > > > Anyway, it sounds like (*within the confines of how libvirt constructs > > the PCI topology*) we actually have functional parity of acpi-index > > between 440fx and Q35. > > > >
