> -----Original Message----- > From: qemu-devel-bounces+benve=cisco.com@xxxxxxxxxx [mailto:qemu-devel- > bounces+benve=cisco.com@xxxxxxxxxx] On Behalf Of Markus Armbruster > Sent: Monday, October 31, 2011 7:05 AM > To: Daniel P. Berrange > Cc: libvir-list@xxxxxxxxxx; David Wang (dwang2); Ram Durairaj > (radurair); qemu-devel@xxxxxxxxxx; Sumit Naiksatam (snaiksat) > Subject: Re: [Qemu-devel] RFC decoupling VM NIC provisioning > fromVM NIC connection to backend networks > > "Daniel P. Berrange" <berrange@xxxxxxxxxx> writes: > > > On Fri, Oct 28, 2011 at 04:15:41PM -0700, Sumit Naiksatam (snaiksat) > wrote: > >> Hi, > >> > >> In its current implementation Libvirt makes sure that the network > >> interfaces that it passes/provision to a VM (for example to qemu[- > kvm]) > >> are already connected to its backend (interfaces/networks) by the > time > >> the VM starts its boot process. In a non virtualized setup it would > be > >> like booting a machine with the Ethernet cable already plugged into > a > >> router/switch port. While in a non virtualized setup you can boot a > >> machine first (with no physical connection to a router/switch) and > later > >> connect its NIC/s to the switch/router, when you boot a VM via > Libvirt > >> it is not possible to decouple the two actions (VM boot, cable > >> plug/unplug). > >> > >> An example of case where the capability of decoupling the two > actions > >> mentioned above is a requirement in Quantum/NetStack which is the > >> network service leveraged by OpenStack. The modular design of > OpenStack > >> allows you to: > >> - provision VMs with NIC/s > >> - create networks > >> - create ports on networks > >> - plug/unplug a VM NIC into/from a given port on a network (at > runtime) > >> > >> Note that this runtime plug/unplug requirement has nothing to do > with > >> hot plug/unplug of NICs. > >> The idea is more that of decoupling the provisioning of a VM from > the > >> connection of the VM to the network/s. > >> This would make it possible to change (at run-time too) the networks > the > >> NIC/s of a given VM are connected to. > >> > >> For example, when a VM boots, its interfaces should be in link down > >> state if the network admin has not connected the VM NIC/s to any > >> "network" yet. > >> Even though libvirt already provides a way to change the link state > of > >> an a VM NIC, link state and physical connection are two different > things > >> and should be manageable independently. > >> > >> Ideally the configuration syntax should be interface type and > hypervisor > >> type agnostic. > >> > >> Let's take QEMU[-kvm] as an example - when Libvirt starts a QEMU VM, > it > >> passes to QEMU a number of file descriptors that map to host backend > >> interfaces (for example macvtap interfaces). > >> > >> In order to introduce this runtime plug/unplug capability, we need a > >> mechanism that permits to delay the binding between the host macvtap > >> interfaces and the guest taps (because you cannot know the fd of the > >> macvtap interfaces before you create them). This means you need a > >> mechanism that allows you to change such fd/s at runtime: > >> > >> - you can close/reset an fd (ie, when you disconnect a VM NIC from > its > >> network) > >> - you can open/set an fd (ie, when you connect a VM NIC to a > network) > >> > >> This could probably be a libvirt command that translates to a QEMU > >> monitor command. > >> > >> Can the runtime plug/unplug capability described above be achieved > >> (cleanly) with another mechanism? > >> > >> Is anybody working on implementing something similar? > > > > No, but I've long thought about doing this & it is quite > straightforward > > todo really. Ordinarily when we start QEMU we do > > > > qemu ... -device e1000,id=nic0,netdev=netdevnic0 \ > > -netdev user,id=netdevnic0 > > > > Todo what you describe we need to be able to: > > > > 1. Start QEMU with a NIC, but no netdev > > 2. Add a netdev to running QEMU. > > 3. Remove a netdev from a running QEMU > > 4. Associate a netdev with a NIC in running QEMU > > > > We can do 1: > > > > $ qemu ... -device e1000,id=nic0 > > > > But QEMU prints an annoying warning > > > > Warning: nic nic0 has no peer > > > > We can do 2 via the monitor: > > > > (qemu) netdev_add type=user,id=netdevnic0 > > > > We can do 3 via the monitor: > > > > (qemu) netdev_del netdevnic0 > > > > > > The problem is 4 - AFAICT we can't connect the existing NIC upto the > newly > > hotplugged netdev, since we can't update the 'netdev' property in the > NIC > > device. > > Yes, that's the missing piece. Can we go ahead and add this missing piece to QEMU? Is there any concern on the QEMU side? /Chris > > Also if we delete the netdev, we can't clear out the 'netdev' > > property in the NIC, so its dangling to a netdev that no longer > exists. > > Err, this sounds like we had a dangling pointer there. We don't. > > Until commit a083a89d, netdev_del disconnected the backend netdev from > the frontend NIC (assigning null to property "netdev"), then deleted > the > netdev. > > Since then, we delay the actual deletion until the NIC goes away, > because "removing host netdev peer while guest is active leads to > guest-visible inconsistency and/or crashes". > > I figure that needs to be fixed differently to enable dynamic network > backend switching. > > > The latter is fairly harmless, since we can just re-use the name if > adding > > a new backend later. The first problem is a bit of a pain, unless we > plug > > in a 'user' backend on the CLI, and immediately netdev_del it before > starting > > the CPUs. Ideally we'd have some way to set qdev properties for > devices so we > > can associate the NIC with the new netdev. > > > > eg when adding a netdev: > > > > (qemu) netdev_add type=user,id=netdevnic0 > > (qemu) set nic0 netdev=netdevnic0 > > > > Or removing one > > > > (qemu) netdev_add netdevnic0 > > (qemu) unset nic0 netdev > > Please work with Anthony to get this use case covered in QOM. > > > WRT to libvirt XML config. Normally you specifiy a NIC like > > > > <interface type='network'> > > <mac address='52:54:00:0f:7d:ad'/> > > <source network='default'/> > > <model type='virtio'/> > > </interface> > > > > To boot a guest without any netdev backend present, we'd introduce a > > new network type="none". eg > > > > <interface type='none'> > > <mac address='52:54:00:0f:7d:ad'/> > > <model type='virtio'/> > > </interface> > > > > The existing API 'virDomainUpdateDevice', can then be used to change > > the interface config on the fly, adding or removing the netdev by > > passing in new XML with a different 'type' attribute & <source> > > element. > > > > Finally, when adding & removing the netdev backends to a running > guest, > > we likely want to be able to set the NIC's link carrier, so the > guest > > OS sees that it has lost / gain its network connection & will thus > > retry DHCP / IPv6 autoconfig. There is already a QEMU montior command > > 'set_link' for changing the NIC link carrier. A minor problem is > that, > > AFAICT, we can't specify the link carrier state on the command line > > when specifying the NIC hardware, eg we would want something like > > this when starting a guest without a netdev back > > > > qemu ... -device e1000,id=nic0,link=down > > Should be easy enough to do. > > > And when adding a netdev we would do > > > > (qemu) netdev_add user,id=netdevnic0 > > (qemu) set nic0 netdev=netdevnic0 > > (qemu) set_link nic0 up > > > > Or when removing a netdev > > > > (qemu) set_link nic0 down > > (qemu) unset nic0 netdev > > (qemu) netdev_del user,id=netdevnic0 -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list