Re: proper bridging technoque

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 


On Nov 21, 2013, at 10:48 AM, Digimer wrote:

What you do in the VMs does not impact the hosts, so I didn't speak to
that. Having the bridge, interfaces, switches and vnets at 9000 (for
example) doesn't immediately enable large frames in the virtual servers.
It simply means that all of the links between the VM and other devices
on the network are ready for JFs.

Imagine this;

{real switch}
|
{ethX + ethY}
|
{bondX}
|
{vbr0}
|
{vnetX}
|
{VM's eth0}

All of these devices need to have their MTU set to your desires value.
If any one of these is still 1500, then only standard frames will be
able to traverse them.

* real switch; Log into it and make sure jumbo frames are enabled

* ethX + ethY; If you are using bonding, be sure both/all slaved
interfaces are set to use a large frame.

* bondX; Again if you use a bond, make sure the bondX interface has a
large frame.

* vbr0; The bridge can not be set to a specific MTU size. It will use
the lowest MTU of the various devices / interfaces connected to it.

* vnetX; These are the "virtual network cables" that are used to "plug
in" a VM's interface to the bridge. This is not new by any means. In the
real world, network cables don't have setable MTUs of course. In the
virtual world though, they do. These interfaces are spontaneously
created and destroyed as VMs come and go. This is what the udev rule is
for because these "virtual network cables" don't have traditional
ifcfg-X files.

* VM's eth0; This is the (emulated) network card in your virtual server.
If you told the hypervisor to replicate an e1000 intel card or use the
virtio-net driver, you can set a large MTU. However, if you used
something like an emulated realtek card, those don't support jumbo
frames, so their emulated counterparts will not support large frames either.

hth

digimer

On 21/11/13 13:32, Nico Kadel-Garcia wrote:
I was under the impression that the relevant MTU settings were on the
*node's* local ifcfg-eth* configurations. Did something change with
KVM internal networking in the last year?

On Thu, Nov 21, 2013 at 1:03 PM, Digimer <lists@xxxxxxxxxx> wrote:
The problem is that there are no ifcfg-vnetX config files. They are
dynamically created as VMs are created or migrated to a node. You could
manually (or via script) change the MTU, but that would mean that the
MTU on the bridge would drop momentarily when new VMs start. This could
break network traffic for any existing VMs (or real devices) using large
frames.

I'm not a fan of udev either, but in this case, it is the best option.
Of course, I am certainly open to hearing alternative methods if they exist.

On 21/11/13 08:39, Nico Kadel-Garcia wrote:
Stay out of udev if you can. It's often overwritten by component
addition and manipulation MTU is parsed, and overridden, by options in
/etc/sysconfig/network-scripts/ifcfg-[device]. I find it much safer to
read and manage there, and if new devices are added or replaced, the
behavior is dominated by the "HWADDR" associated config files there,
no matter what "udev" thinks the device number or name should be..

<snip>


Another update;

 To make sure the VMs' vnetX devices are created with a larger MTU, you
*sill* need to update udev[1].

 Append to /etc/udev/rules.d/70-persistent-net.rules;

====
# Make all VMs' vnetX devices come up with an MTU of 9000.
SUBSYSTEM=="net", ACTION="" KERNEL=="vnet*", ATTR{mtu}="9000"
====

 Assuming you find that you can use an MTU of '9000', of course. No
need to reboot or even restart networking. Just add that line and then
provision/boot your VMs. If the VMs are already running, you can adjust
the MTU of the existing 'vnetX' devices with:

ifconfig vnetX mtu 9000

Cheers!

PS - Credit for the udev rule:

http://linuxaleph.blogspot.ca/2013/01/how-to-network-jumbo-frames-to-kvm-guest.html

--
Digimer

Hi,

I seem to lack a vnet to bridge device.

When I go to change my interface on the VM using the GUI, I do not see an option for "Host device vnet# (Bridge 'br6')

Instead I see "host device eth6 (Bridge 'br6')  So before creating one via;

brctl addif...

Let me explain my config;

eth0 - standard MTU
eth1 - disabled
*eth6 - 10Gb at jumbo
* This card was added after KVM was setup and running.

My ifconfig output;


br0       Link encap:Ethernet  HWaddr 00:25:90:63:9F:7A  
          inet addr:10.0.10.218  Bcast:10.0.255.255  Mask:255.255.0.0
          inet6 addr: fe80::225:90ff:fe63:9f7a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:754670 errors:0 dropped:0 overruns:0 frame:0
          TX packets:43162 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:160904242 (153.4 MiB)  TX bytes:51752758 (49.3 MiB)

br6       Link encap:Ethernet  HWaddr 00:05:33:48:7B:29  
          inet addr:10.0.10.220  Bcast:10.0.255.255  Mask:255.255.0.0
          inet6 addr: fe80::205:33ff:fe48:7b29/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:4130 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11150 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:131498 (128.4 KiB)  TX bytes:513156 (501.1 KiB)

eth0      Link encap:Ethernet  HWaddr 00:25:90:63:9F:7A  
          inet6 addr: fe80::225:90ff:fe63:9f7a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3379929 errors:18 dropped:0 overruns:0 frame:18
          TX packets:3565007 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:840911383 (801.9 MiB)  TX bytes:3519831013 (3.2 GiB)
          Memory:fbbe0000-fbc00000 

eth6      Link encap:Ethernet  HWaddr 00:05:33:48:7B:29  
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Memory:fbd40000-fbd7ffff 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:185130 errors:0 dropped:0 overruns:0 frame:0
          TX packets:185130 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:138905226 (132.4 MiB)  TX bytes:138905226 (132.4 MiB)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:CE:7A:65  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11139 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:512410 (500.4 KiB)

vnet0     Link encap:Ethernet  HWaddr FE:30:48:7E:65:72  
          inet6 addr: fe80::fc30:48ff:fe7e:6572/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:1045 errors:0 dropped:0 overruns:0 frame:0
          TX packets:697730 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:119723 (116.9 KiB)  TX bytes:175334262 (167.2 MiB)

vnet1     Link encap:Ethernet  HWaddr FE:16:36:0E:E7:F4  
          inet6 addr: fe80::fc16:36ff:fe0e:e7f4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:3494450 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3243369 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:3466241191 (3.2 GiB)  TX bytes:822212316 (784.1 MiB)

brctl show;

bridge name bridge id STP enabled interfaces
br0 8000.002590639f7a no eth0
vnet0
vnet1
br6 8000.000533487b29 no eth6
virbr0 8000.525400ce7a65 yes virbr0-nic


Should I have a virbr6?

I'm obviously pretty lost and must admin I sorta hate bridging in KVM.

- aurf







_______________________________________________
CentOS-virt mailing list
CentOS-virt@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos-virt
[Index of Archives]     [CentOS Users]     [Linux Media]     [Asterisk]     [DCCP]     [Netdev]     [X.org]     [Xfree86]     [Linux USB]

  Powered by Linux