Re: Finally switching from Xen to KVM - question about networking

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 6/11/2014 10:56 AM, Zoltan Frombach wrote:
>
> On 6/11/2014 4:44 PM, Steve Campbell wrote:
>> On 6/11/2014 10:13 AM, Zoltan Frombach wrote:
>>> On 6/11/2014 4:10 PM, Steve Campbell wrote:
>>>> On 6/11/2014 10:03 AM, Zoltan Frombach wrote:
>>>>> On 6/11/2014 3:50 PM, Steve Campbell wrote:
>>>>>> On 6/10/2014 4:00 PM, Zoltan Frombach wrote:
>>>>>>> On 6/10/2014 9:51 PM, Steve Campbell wrote:
>>>>>>>> On 6/10/2014 3:38 PM, Zoltan Frombach wrote:
>>>>>>>>> On 6/10/2014 9:27 PM, Steve Campbell wrote:
>>>>>>>>>> On 6/10/2014 3:09 PM, Zoltan Frombach wrote:
>>>>>>>>>>> Steve,
>>>>>>>>>>>
>>>>>>>>>>> Try the following config.
>>>>>>>>>>>
>>>>>>>>>>> On your host:
>>>>>>>>>>>
>>>>>>>>>>> /etc/sysconfig/network-scripts/ifcfg-eth0    :
>>>>>>>>>>>
>>>>>>>>>>> DEVICE=eth0
>>>>>>>>>>> BOOTPROTO=none
>>>>>>>>>>> ONBOOT=yes
>>>>>>>>>>> NM_CONTROLLED=no
>>>>>>>>>>> TYPE=Ethernet
>>>>>>>>>>> HWADDR=xx:xx:xx:xx:xx:xx    <- put your physical NIC's MAC
>>>>>>>>>>> address
>>>>>>>>>>> here
>>>>>>>>>>> BRIDGE=br0
>>>>>>>>>>> USERCTL=no
>>>>>>>>>>> IPV6INIT=no
>>>>>>>>>>> IPV6_AUTOCONF=no
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> /etc/sysconfig/network-scripts/ifcfg-br0    :
>>>>>>>>>>>
>>>>>>>>>>> DEVICE=br0
>>>>>>>>>>> BOOTPROTO=static
>>>>>>>>>>> ONBOOT=yes
>>>>>>>>>>> NM_CONTROLLED=no
>>>>>>>>>>> TYPE=Bridge
>>>>>>>>>>> IPADDR=10.0.5.16
>>>>>>>>>>> NETMASK=255.255.255.0   <- you may need to adjust this to your
>>>>>>>>>>> network
>>>>>>>>>>> GATEWAY=10.0.5.1
>>>>>>>>>>> DNS1=8.8.8.8
>>>>>>>>>>> DNS2=8.8.4.4
>>>>>>>>>>> PEERDNS=yes
>>>>>>>>>>> DELAY=0
>>>>>>>>>>> STP=off
>>>>>>>>>>> USERCTL=no
>>>>>>>>>>> IPV6INIT=no
>>>>>>>>>>> IPV6_AUTOCONF=no
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Restart networking on the host.
>>>>>>>>>>>
>>>>>>>>>>> Then inside you VM:
>>>>>>>>>>>
>>>>>>>>>>> /etc/sysconfig/network-scripts/ifcfg-eth0    :
>>>>>>>>>>>
>>>>>>>>>>> DEVICE=eth0
>>>>>>>>>>> BOOTPROTO=static
>>>>>>>>>>> ONBOOT=yes
>>>>>>>>>>> NM_CONTROLLED=no
>>>>>>>>>>> TYPE=Ethernet
>>>>>>>>>>> HWADDR=xx:xx:xx:xx:xx:xx    <- it must be a unique MAC address
>>>>>>>>>>> for
>>>>>>>>>>> your VM
>>>>>>>>>>> IPADDR=10.0.5.17
>>>>>>>>>>> NETMASK=255.255.255.0     <- you may need to adjust this to 
>>>>>>>>>>> your
>>>>>>>>>>> network
>>>>>>>>>>> GATEWAY=10.0.5.1
>>>>>>>>>>> DNS1=8.8.8.8
>>>>>>>>>>> DNS2=8.8.4.4
>>>>>>>>>>> PEERDNS=yes
>>>>>>>>>>> USERCTL=no
>>>>>>>>>>> IPV6INIT=no
>>>>>>>>>>> IPV6_AUTOCONF=no
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Then in Virtual Machine Manager make sure that your VM's NIC is
>>>>>>>>>>> connected to the br0 bridge, like this:
>>>>>>>>>>>
>>>>>>>>>>> Network Source: Specify shared device name
>>>>>>>>>>>            Bridge Name: br0
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 6/10/2014 8:16 PM, Steve Campbell wrote:
>>>>>>>>>>>> On 6/10/2014 12:43 PM, Digimer wrote:
>>>>>>>>>>>>> On 10/06/14 12:38 PM, Steve Campbell wrote:
>>>>>>>>>>>>>> On 6/10/2014 12:05 PM, Digimer wrote:
>>>>>>>>>>>>>>> On 10/06/14 11:46 AM, Steve Campbell wrote:
>>>>>>>>>>>>>>>> On 6/10/2014 10:46 AM, Digimer wrote:
>>>>>>>>>>>>>>>>> On 10/06/14 10:03 AM, Steve Campbell wrote:
>>>>>>>>>>>>>>>>>> I had so much trouble putting Centos 6 guest VMs on a
>>>>>>>>>>>>>>>>>> Centos 5
>>>>>>>>>>>>>>>>>> host
>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> I finally switched to a Centos 6 host.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I've not needed more that test VMs, so I've used Virtual
>>>>>>>>>>>>>>>>>> Machine
>>>>>>>>>>>>>>>>>> Manager
>>>>>>>>>>>>>>>>>> on the old system, which worked pretty well, so I
>>>>>>>>>>>>>>>>>> decided to
>>>>>>>>>>>>>>>>>> create my
>>>>>>>>>>>>>>>>>> first KVM guest machine. I noticed when I created it, I
>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>> had the
>>>>>>>>>>>>>>>>>> options of NAT for my network interface, so I used that
>>>>>>>>>>>>>>>>>> (obvious).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Well, after starting the VM, I find I don't have
>>>>>>>>>>>>>>>>>> connectivity
>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> interface. Reading, I find examples where I need to 
>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>> bridges
>>>>>>>>>>>>>>>>>> perhaps. Xen did most of this for me, so it's a little
>>>>>>>>>>>>>>>>>> new to
>>>>>>>>>>>>>>>>>> me.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Can anyone throw me a clue, please?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> steve campbell
>>>>>>>>>>>>>>>>> Setting up a bridge is not that hard, and it will give
>>>>>>>>>>>>>>>>> your VMs
>>>>>>>>>>>>>>>>> direct
>>>>>>>>>>>>>>>>> access to the outside world, and host <-> VM access just
>>>>>>>>>>>>>>>>> fine as
>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Here is a link showing how to setup a bridge connected 
>>>>>>>>>>>>>>>>> to a
>>>>>>>>>>>>>>>>> bond
>>>>>>>>>>>>>>>>> device. Ignore the bond and pretend it is a straight ethX
>>>>>>>>>>>>>>>>> device:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://alteeve.ca/w/AN!Cluster_Tutorial_2#Configuring_our_Bridge.2C_Bonds_and_Interfaces 
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The host has a device named virbr0 that is installed 
>>>>>>>>>>>>>>>> during
>>>>>>>>>>>>>>>> system
>>>>>>>>>>>>>>>> installation. It also has a network device vnet0. There
>>>>>>>>>>>>>>>> are no
>>>>>>>>>>>>>>>> files in
>>>>>>>>>>>>>>>> /etc/sysconfig/network-scripts for these.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Shouldn't I be able to use the virbr0 virtual bridge for
>>>>>>>>>>>>>>>> this?
>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>> tried setting up the VM's device with all of the options
>>>>>>>>>>>>>>>> that is
>>>>>>>>>>>>>>>> listed,
>>>>>>>>>>>>>>>> but to no avail.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Should I need to set up another bridge for this?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> And thanks for the link.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> steve
>>>>>>>>>>>>>>> virbr0 is created and managed by libvirtd. If you open
>>>>>>>>>>>>>>> Virtual
>>>>>>>>>>>>>>> Machine
>>>>>>>>>>>>>>> Manager, connect to localhost and then double-click on
>>>>>>>>>>>>>>> 'localhost',
>>>>>>>>>>>>>>> you will see a tab for creating/managing bridges (NAT'ed,
>>>>>>>>>>>>>>> generally).
>>>>>>>>>>>>>>> I disable 'virbr0' as NAT'ing is generally not what I want.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The 'vnetX' devices are dynamically created to link a VM's
>>>>>>>>>>>>>>> interface
>>>>>>>>>>>>>>> to a bridge. Think of them as virtual network cables. They
>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>> and destroyed as needed.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sorry, but I'm confused:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> My host server has a real NIC and IP address with a real
>>>>>>>>>>>>>> gateway to
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> outside:
>>>>>>>>>>>>>> virtbr0 IP: 192.168.122.1
>>>>>>>>>>>>>> Host IP: 10.0.5.16
>>>>>>>>>>>>>> Gateway IP: 10.0.5.1
>>>>>>>>>>>>>> on eth0 and this works
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> My VM server has all fake stuff currently:
>>>>>>>>>>>>>> Host IP: 10.0.5.17
>>>>>>>>>>>>>> Gateway IP: 10.0.5.1
>>>>>>>>>>>>>> on eth0 and this is like NIC without a cable.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So I need to create a bridge device on both the host and VM
>>>>>>>>>>>>>> (lets
>>>>>>>>>>>>>> say I
>>>>>>>>>>>>>> name it br1). I change the eth0 config file on both host and
>>>>>>>>>>>>>> VM to
>>>>>>>>>>>>>> point
>>>>>>>>>>>>>> to br1 and give the br1 config file on both host and VM the
>>>>>>>>>>>>>> correct
>>>>>>>>>>>>>> IP.
>>>>>>>>>>>>>> But won't this just let the two talk to each other. How will
>>>>>>>>>>>>>> the VM
>>>>>>>>>>>>>> server get outside?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> steve
>>>>>>>>>>>>> The bridge is created only on the host. Think of the 
>>>>>>>>>>>>> bridge as
>>>>>>>>>>>>> being
>>>>>>>>>>>>> like a virtual switch. When a VM is created, you tell it to
>>>>>>>>>>>>> connect to
>>>>>>>>>>>>> the bridge, similar to how you would plug a physical wire
>>>>>>>>>>>>> into a
>>>>>>>>>>>>> real
>>>>>>>>>>>>> switch. That provides the link to the network, and then you
>>>>>>>>>>>>> configure
>>>>>>>>>>>>> the virtual server's network just as if it was a real 
>>>>>>>>>>>>> network.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On the host, you don't set the IP on the ethX device,
>>>>>>>>>>>>> instead you
>>>>>>>>>>>>> tell
>>>>>>>>>>>>> ethX to connect to the bridge (look at 'bond2' in the link
>>>>>>>>>>>>> above
>>>>>>>>>>>>> for
>>>>>>>>>>>>> the 'BRIDGE="..."' line). Then you move the host's IP
>>>>>>>>>>>>> address/config
>>>>>>>>>>>>> to the bridge itself (look at the 'vbr2' in the link above).
>>>>>>>>>>>>>
>>>>>>>>>>>>> If you're still stuck, tell me the IP addresses you want to
>>>>>>>>>>>>> set in
>>>>>>>>>>>>> the
>>>>>>>>>>>>> host and VM and I'll see if I can bang out a couple sample
>>>>>>>>>>>>> 'ifcfg-X'
>>>>>>>>>>>>> files.
>>>>>>>>>>>>>
>>>>>>>>>>>> I'm running into some problems with my config files where I
>>>>>>>>>>>> get a
>>>>>>>>>>>> message that says the network can't determine my interface
>>>>>>>>>>>> names.
>>>>>>>>>>>> I'll
>>>>>>>>>>>> look further, but the IPs above are what I need. 10.0.5.1 is a
>>>>>>>>>>>> firewall
>>>>>>>>>>>> that NATs the 10.0.5.0/24 network to another outside public
>>>>>>>>>>>> firewall.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>> steve
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> CentOS-virt mailing list
>>>>>>>>>>>> CentOS-virt@xxxxxxxxxx
>>>>>>>>>>>> http://lists.centos.org/mailman/listinfo/centos-virt
>>>>>>>>>>>>
>>>>>>>>>> Stuck in a rut now. Restarting the network indicates :
>>>>>>>>>>
>>>>>>>>>> Device eth0 does not seem to be present, delaying 
>>>>>>>>>> initialization.
>>>>>>>>>>
>>>>>>>>>> I thought it might be due to the fact I didn't have the 
>>>>>>>>>> "TYPE" in
>>>>>>>>>> the
>>>>>>>>>> ifcfg-eth0 file, but after adding that, I still get the error.
>>>>>>>>> NM_CONTROLLED=no might help with that.
>>>>>>>> I turned off and stopped NetworkManager and had already specified
>>>>>>>> this.
>>>>>>>>>> I'm using the same hardware address that was in the original 
>>>>>>>>>> eth0
>>>>>>>>>> file.
>>>>>>>>> That is fine.
>>>>>>>>>
>>>>>>>>>>       My bridge device starts fine, but I'll tailor it to your
>>>>>>>>>> specs.
>>>>>>>>> Also, you might want to check the contents of your
>>>>>>>>> /etc/udev/rules.d/70-persistent-net.rules file. Is the MAC 
>>>>>>>>> address
>>>>>>>>> for
>>>>>>>>> eth0 the same as in your 
>>>>>>>>> /etc/sysconfig/network-scripts/ifcfg-eth0
>>>>>>>>> file?
>>>>>>>>>
>>>>>>>> They are the same except the udev file has lower case letters 
>>>>>>>> in it.
>>>>>>> You mean the hexadecimal numbers are lower case? That shouldn't
>>>>>>> matter.
>>>>>>>
>>>>>>> Try to delete the file /etc/udev/rules.d/70-persistent-net.rules 
>>>>>>> and
>>>>>>> reboot your host machine.
>>>>>>> ( See
>>>>>>> http://www.andrewklau.com/device-eth0-does-not-seem-to-be-present-delaying-initialization/ 
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> )
>>>>>>>
>>>>>> All is well on the host now, and I've brought up the VM and 
>>>>>> configured
>>>>>> it to use the bridge. I can ping the host from the VM but I have no
>>>>>> access beyond the local network. The host sees the world, but the VM
>>>>>> only sees the 10.0.5.0/24 network.
>>>>> Did you set the gateway in your VM to 10.0.5.1 ?
>>>>> Can you ping the gateway ( ping 10.0.5.1 ) from the VM?
>>>>> Do you have a firewall configured in your VM (or host) ? If so, 
>>>>> try to
>>>>> disable the firewall temporarily for testing.
>>>>>
>>>> The gateway is set. Can't ping the gateway. Firewalls (iptables and
>>>> ip6tables) are stopped and off. Previous post suggested the route. 
>>>> I'll
>>>> check it.
>>> Add the following lines to your /etc/sysctl.conf file on your host:
>>>
>>> net.ipv4.ip_forward = 1
>>> net.ipv4.conf.default.rp_filter = 0
>>> net.ipv6.conf.all.disable_ipv6 = 1
>>> net.ipv6.conf.default.disable_ipv6 = 1
>>>
>>> then issue the following command:
>>>
>>> sysctl -p
>>>
>> Still no joy. I'm installing another VM just to see if having the bridge
>> defined before installation would make a difference. I find it hard to
>> imagine all of this can't be done will creating the VM (Xen seemed to do
>> it all for you).
>
> What you did on the host machine, only has to be done once when you 
> setup the system to be used as a KVM host server.
>
>> So now, I've got the route, gateway, bridge, and eth0/1 defined on the
>> original. I can ping both the host and the VM from each other. The host
>> can see the world through the gateway. The VM can't see the gateway or
>> the world.
>
> How did you come up with the IP address ( 10.0.5.17 ) for the VM ? Are 
> you sure it is not blocked somehow by your gateway ( at 10.0.5.1 ) ?
>
> Also, maybe your gateway ( at 10.0.5.1 ) is doing MAC address 
> filtering? Since your VM has its own MAC address, this could be a 
> problem.
>
The gateway is a firewall. It's set up to route and NAT from 10.0.5.0/24 
to another inside network on 10.0.0.0/24. This is the true inside 
network that is NATted to the public with our public IPs. It accepts all 
traffic from the 10.0.5.0 network.

steve
_______________________________________________
CentOS-virt mailing list
CentOS-virt@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos-virt




[Index of Archives]     [CentOS Users]     [Linux Media]     [Asterisk]     [DCCP]     [Netdev]     [X.org]     [Xfree86]     [Linux USB]

  Powered by Linux