Re: <interface type='hostdev'>vf configuration cleanup when VM is delete

Laine Stump <laine@xxxxxxxxx> · Tue, 15 Dec 2015 14:45:27 -0500



    On 12/15/2015 01:34 PM, Laine Stump
      wrote:

    
      On 12/13/2015 10:51 AM, Moshe Levi
        wrote:

      
          Hi, 
           
          I have a setup with libvirt 1.3.0 and
            OpenStack trunk.  
          Before launched the VM ip link command
            show the following VF mac/vlan configuration [1]
          When I launch a VM with <interface
            type='hostdev'> via openstack api (OpenStack direct port)
            
          I can see that the VF get the mac/vlan
            according to libvrit xml [2] and ip link command  [3], but
            when I delete the VM the mac/vlan config are still shown as
            in [3] and not restored to [1]
          Shouldn’t  libvirt restore the mac/vlan
            to [1].
           
          The same problem exists when using
            <interface type='direct'> (OpenStack macvtap port)
             but just for the MAC configuration of the VF. 
        
      
      What libvirt does is to restore the MAC address to whatever it was
      before we set it up for use with a guest. Although there are some
      sriov net drivers that (for some unfathomable reason) think it's
      cool to assign a random MAC address to each VF at boot time, the
      "normal" thing is for the VFs to have a MAC address of all 0's to
      start with. So libvirt should be saving 00:00:00:00:00:00 (it will
      be in the file /var/run/libvirt/hostdevmgr/$ifname_vf$vfnum) then
      setting the MAC to use; when done, libvirt will read the
      00:00:00:00:00:00 and use netlink to set the MAC address, but this
      is apparently failing.

      
      I checked on my Fedora 22 system with the igb driver, and found
      that if the MAC address was originally set to something other than
      0's, it was restored properly by libvirt, but if it was set to all
      0's originally, the attempt to set it back to 0 would fail.

      
      I then tried doing the same thing with the "ip" utility:

      
          # ip link set dev p4p2 vf 0 mac 00:00:00:00:00:00

      
      and I get the following response:

      
          RTNETLINK answers: Invalid argument

      
      So it appears that either the kernel or the NIC driver is refusing
      to set the MAC address to all 0's. I'm reasonably certain this is
      a regression in the kernel,
    

    Sigh. It appears that this has "always" been the case - I just
    checked on a 2.6.32-573 RHEL kernel, and a 3.10.x RHEL7.2 kernel,
    and 4.1 (Fedora 22) and both of them also refuse to set the MAC
    address to 00:00:00:00:00:00. I'm not sure if this limitation is in
    the NIC driver or some basic code in the kernel.

    
      although I can't say how long it's been there, as I don't normally
      pay attention to this (and as I said, many SRIOV NIC drivers don't
      default their VFs to 0 MAC addresses)

      
      What distro and kernel are you using for your tests?

      
          [1]  - 24: enp3s0f0:
            <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq
            master ovs-system state UP mode DEFAULT group default qlen
            1000
              link/ether e4:1d:2d:a5:f1:22 brd
            ff:ff:ff:ff:ff:ff
              vf 0 MAC 00:00:00:00:00:00, spoof
            checking off, link-state auto
              vf 1 MAC 00:00:00:00:00:00, spoof
            checking off, link-state auto
              vf 2 MAC 00:00:00:00:00:00, spoof
            checking off, link-state auto
              vf 3 MAC 00:00:00:00:00:00, spoof
            checking off, link-state auto
           
          [2] - <interface type='hostdev'
managed='yes'>                                                         
            <mac address=' fa:16:3e:11:af:fe
            '/>                          
                                               
            <driver
name='kvm'/>                                                                           
            <source>                                                                                       

            
              <address type='pci'
            domain='0x0000' bus='0x02' slot='0x00'
            function='0x7'/>                   
            </source>                                                                                      

            
            <vlan>                                                                       

                              
              <tag
id='190'/>                                                                              
            </vlan>
            <alias name='hostdev0'/>
            <address type='pci' domain='0x0000'
            bus='0x00' slot='0x04' function='0x0'/>
          </interface>
           
           
          [3] 24: enp3s0f0:
            <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq
            master ovs-system state UP mode DEFAULT group default qlen
            1000
              link/ether e4:1d:2d:a5:f1:22 brd
            ff:ff:ff:ff:ff:ff
              vf 0 MAC 00:00:00:00:00:00, spoof
            checking off, link-state auto
              vf 1 MAC 00:00:00:00:00:00, spoof
            checking off, link-state auto
              vf 2 MAC 00:00:00:00:00:00, spoof
            checking off, link-state auto
              vf 3 MAC fa:16:3e:11:af:fe, vlan 190,
            spoof checking off, link-state enable
           
        
        --
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list
      
      F15

      
      --
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list
    
    
--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list