On Monday 05 April 2010 13.34:26 mathias dufresne wrote:
> Hi,
>
> For networks 1 and 2 (respectively 192.168.1.0/24 and 192.168.2.0/24, I'll
> acll also network 3 for the last one) you'll have to use bridges.
> For network 3 you can use vde which is quite simply to use and gives
> possibility to add new virtual networks without the need of physical
> interface.
>
> For bridges configuration:
> brctl addbr ${brname}
> brctl setfd ${brname} 0
> brctl sethello ${brname} 1
> brctl stp ${brname} off
> brct addif ${brname} ${physical_if}
>
> Here you can add an IP to you're bridge interface if needed for the host
> has access to this network. Host's IP must by set up on bridge interface
> with configuration, if IP is set up on physical interface it is not
> working. I'm wondering why, my network knowledge is not much :/
>
> Then when you'll run qemu to set up a new VM you'll have to use a tap
> device for the VM
>
> One important thing when use several networks in the same host is then vlan
> ID you'll use for networks. These vlans aren't (from my understanding : )
> 802.1Q vlans but pure qemu/kvm concepts. If you don't set up each VM's
> interface on the right kvm's vlan they will all be set up on vlan number 0,
> which means packets from all networks will appear everywhere (tcpdump on
> 192.168.1.0/24 card will see traffic for others networks).
>
> Here is the command I use to launch a VM with two bridged interfaces and
> one other on some vde network:
> screen -d -m qemu-system-x86_64 -m 64 \
> -net tap,vlan=0,ifname=tap0,script=/path/to/scripts/kvm-ifup.br0 \
> -net nic,vlan=0,model=e1000,macaddr=52:54:00:01:ab:cd \
> -net tap,vlan=3,ifname=tap1,script=/path/to/scripts/kvm-ifup.br1 \
> -net nic,vlan=3,model=e1000,macaddr=52:54:00:03:ab:cd \
> -net vde,vlan=2,sock=/var/vde/switches/vdelan0 \
> -net nic,vlan=2,model=e1000,macaddr=52:54:00:02:ab:cd \
> -hda /path/to/disks/vm.vmdk \
> -nographic
>
> Note the "vlan=X" option set on both -net arguments for each interface.
>
> For VDE switchs:
> /usr/bin/vde_switch -m 770 -g qemu -daemon -n 8 -t dummy10 -s
> /var/vde/switches/vdelan0 -M /var/vde/management/vdelan0
>
> I use dummy interface on host to set up vde networks because the host
> mustn't have access on these networks.
> -s directory where VDE's unix sockets will be store for this network
> -M management socket (I haven't yet understood how to use them :p)
> -n number of port on the switch
> other options are quite easy to understand.
>
> Finally my scripts to set up bridged interface are the following:
> ------------------
> #!/bin/sh
> #set -x
> switch=`echo $0 | awk -F\. '{ print $NF}'`
> if [ -n "$1" ];then
> #/usr/bin/tunctl -u `whoami` -t $1
> /sbin/ip link set $1 up
> sleep 0.5s
> /sbin/brctl addif $switch $1
> exit 0
> else
> echo "Error: no interface specified"
> exit 1
> fi
> ------------------
> I use one script per bridge named kvm-ifup.${brname} because the script's
> name define the bridge on which the interface will be attached
> (kvm-ifup.br0 -> br0 and kvm-ifup.br1 -> br1).
>
> I use VDE switch for networks on which host isn't plugged because they are
> simpler to use for me but it is not the only way to proceed.
>
>
>
> A last thing, I have a physical card on which is plugged my modem. As I
> don't want to expose the host on Internet I use a VM to act as a firewall.
> As I just can't give this VM direct access on the physical card dedicated
> to Internet I use a bridge:
>
> modem ------- physical card ----- bridge ----- tap ---- VM's internal card
>
> The modem knows only one MAC address: the one from physical interface but
> this physical interface as no IP. The bridge has same MAC address as
> physical interface but has no more IP. The card with an IP is the VM's
> internal card so for packets pass through all these interface without IP I
> use ebtables to "nat" MAC address between physical interface and VM's
> internal card:
> ebtables -t nat -A PREROUTING -i $PHYS_DEV_NAME -d $PHYS_DEV_MAC -j dnat
> --to-destination $VIRT_DEV_MAC
> ebtables -t nat -A POSTROUTING -o $VIRT_DEV_NAME -s $VIRT_DEV_MAC -j snat
> --to-source $PHYS_DEV_MAC
>
> Finally there are some interesting links:
> forum's thread related to vlan issues
> http://serverfault.com/questions/101477/issue-with-multiple-bridging-for-kv
> m-hosts
>
> something about bridges:
> http://www.savelono.com/linux/using-multiple-interfaces-with-kvm-and-xen.ht
> ml
>
> different method to build networks using qemu/kvm:
> http://people.gnome.org/~markmc/qemu-networking.html<http://people.gnome.or
> g/%7Emarkmc/qemu-networking.html>
>
> Hoping this helps.
>
> Kindly regards,
>
> mathias
>
> 2010/4/5 Dan Johansson <kvm@xxxxxx>
>
> > On Sunday 04 April 2010 22.23:28 Held Bernhard wrote:
> > > Am 04.04.2010 20:02, schrieb Dan Johansson:
> > > > On Sunday 04 April 2010 15.00:26 sudhir kumar wrote:
> > > >> On Sun, Apr 4, 2010 at 5:47 PM, Dan Johansson <kvm@xxxxxx> wrote:
> > > >>> Hi,
> > > >>>
> > > >>> I am new to this list and to KVM (and qemu) so please be gentle
> > > >>> with me. Up until now I have been running my virtualizing using
> > > >>> VMWare-Server. Now I want to try KVM due to some issues with the
> > > >>> VMWare-Server and I am having some troubles with the networking
> > > >>> part
> >
> > of
> >
> > > >>> KVM.
> > > >>>
> > > >>> This is a small example of what I want (best viewed in a fix-font):
> > > >>>
> > > >>> +-----------------------------------+
> > > >>>
> > > >>> | Host |
> > > >>> | +----------+ eth0 |---- 192.168.1.0/24
> > > >>> |
> > > >>> | | eth0|-- + |
> > > >>> | | VM1 eth1|---(---+------- eth1 |---- 192.168.2.0/24
> > > >>> | | eth2|---(---(---+ |
> > > >>> |
> > > >>> | +----------+ | | | |
> > > >>> |
> > > >>> | +----------+ +---(---(--- eth2 |---- 192.168.1.0/24
> > > >>> |
> > > >>> | | eth0|---+ | | |
> > > >>> | | VM2 eth1|-------+ +--- eth3 |---- 192.168.3.0/24
> > > >>> | | eth2|-----------+ |
> > > >>> |
> > > >>> | +----------+ |
> > > >>>
> > > >>> +-----------------------------------+
> > > >>>
> > > >>> Host-eth0 is only for the Host (no VM)
> > > >>> Host-eth1 is shared between the Host and the VM's (VM?-eth1)
> > > >>> Host-eth2 and Host-eth3 are only for the VMs (eth0 and eth2)
> > > >>>
> > > >>> The Host and the VMs all have fixed IPs (no dhcp or likewise).
> > > >>> In this example th IPs could be:
> > > >>> Host-eth0: 192.168.1.1
> > > >>> Host-eth1: 192.168.2.1
> > > >>> Host-eth2: -
> > > >>> Host-eth3: -
> > > >>> VM1-eth0: 192.168.1.11
> > > >>> VM1-eth1: 192.168.2.11
> > > >>> VM1-eth2: 192.168.3.11
> > > >>> VM2-eth0: 192.168.1.22
> > > >>> VM2-eth1: 192.168.2.22
> > > >>> VM3-eth2: 192.168.3.22
> > > >>>
> > > >>> And, yes, Host-eth0 and Host-eth2 are in the same subnet, with eth0
> > > >>> dedicated to the Host and eth2 dedicated to the VMs.
> > > >>>
> > > >>> In VMWare this was quite easy to setup (three bridged networks).
> > > >>
> > > >> Its easy with KVM too. You want 3 NICs per VM, so you need to pass
> > > >> the corresponding parameters(including qemu-ifup script) for 3 NICs
> > > >> to each VM.
> > > >> In the host you need to create 2 bridges: say br-eth1 and br-eth2.
> > > >> Make them as the interface on the host in place of the corresponding
> > > >> eth interfaces.(brct addbr br-eth1; ifcfg eth1 0.0.0.0 up; brctl
> > > >> addif br-eth eth1; assign eth1's ip and routes to breth1; same for
> > > >> eth2). In the corresponding qemu-ifup scripts of each interface use
> > > >> bridge=br-ethN (This basicaly translates to brctl addif br-ethN $1,
> > > >> where $ is the tap device created)
> > > >> This should work perfectly fine with your existing NW setup.
> > > >> For a quick reference use: http://www.linux-kvm.org/page/Networking
> > > >
> > > > Thanks for your help, but... I am still not able to get it to work
> > > > the way I want.
> > > > This is what I have don so far:
> > > > brctl addbr br-eth1
> > > > brctl addbr br-eth3
> > > >
> > > > ip link set eth1 up
> > > > ip link set eth3 up
> > > >
> > > > brctl addif br-eth1 eth1
> > > > brctl addif br-eth3 eth3
> > > >
> > > > tunctl -b -t qtap1
> > > > tunctl -b -t qtap3
> > > >
> > > > brctl addif br-eth1 qtap1
> > > > brctl addif br-eth3 qtap3
> > > >
> > > > ifconfig qtap1 up 0.0.0.0 promisc
> > > > ifconfig qtap3 up 0.0.0.0 promisc
> > > >
> > > > # ifconfig
> > > > eth0 Link encap:Ethernet HWaddr 00:0d:88:52:51:24
> > > > inet addr:192.168.1.3 Bcast:192.168.1.255
> >
> > Mask:255.255.255.0
> >
> > > > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > > > RX packets:443638 errors:0 dropped:0 overruns:0 frame:0
> > > > TX packets:758540 errors:0 dropped:0 overruns:0 carrier:0
> > > > collisions:0 txqueuelen:1000
> > > > RX bytes:47041686 (44.8 MiB) TX bytes:990115354 (944.2
> > > > MiB) Interrupt:19 Base address:0xec00
> > > >
> > > > eth1 Link encap:Ethernet HWaddr 00:0d:88:52:51:25
> > > > inet addr:192.168.4.1 Bcast:192.168.4.255
> >
> > Mask:255.255.255.0
> >
> > > > UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
> > > > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> > > > TX packets:6 errors:0 dropped:0 overruns:0 carrier:6
> > > > collisions:0 txqueuelen:1000
> > > > RX bytes:0 (0.0 B) TX bytes:360 (360.0 B)
> > > > Interrupt:18 Base address:0xe880
> > > >
> > > > eth3 Link encap:Ethernet HWaddr 00:0d:88:52:51:27
> > > > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > > > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> > > > TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
> > > > collisions:0 txqueuelen:1000
> > > > RX bytes:0 (0.0 B) TX bytes:240 (240.0 B)
> > > > Interrupt:16 Base address:0xe480
> > > >
> > > > qtap1 Link encap:Ethernet HWaddr 26:c0:de:df:c5:e4
> > > > UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
> > > > RX packets:351 errors:0 dropped:0 overruns:0 frame:0
> > > > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> > > > collisions:0 txqueuelen:500
> > > > RX bytes:14742 (14.3 KiB) TX bytes:0 (0.0 B)
> > > >
> > > > qtap3 Link encap:Ethernet HWaddr 26:3e:ba:2d:97:bc
> > > > UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
> > > > RX packets:6 errors:0 dropped:0 overruns:0 frame:0
> > > > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> > > > collisions:0 txqueuelen:500
> > > > RX bytes:252 (252.0 B) TX bytes:0 (0.0 B)
> > > >
> > > >
> > > > # brctl show
> > > > bridge name bridge id STP enabled interfaces
> > > > br-eth1 8000.000d88525125 no eth1
> > > > qtap1
> > > > br-eth3 8000.000d88525127 no eth3
> > > > qtap3
> > > >
> > > >
> > > > This is the way I start the guest:
> > > > kvm -net nic,vlan=1,model=rtl8139,macaddr=52:54:00:12:34:56 -net
> > > > tap,vlan=1,ifname=qtap1,script=no,downscript=no -net
> > > > nic,vlan=3,model=rtl8139,macaddr=52:54:00:12:34:58 -net
> > > > tap,vlan=3,ifname=qtap3,script=no,downscript=no Robbie.img -m 1024
> > > >
> > > >
> > > > The eth3/br-eth3/qtap3 looks OK (I can ping the "default-GW" on that
> > > > network from the guest) but the connection to the "shared" interface
> > > > (eth1/br- eth1/qtap1) does not work, I can not ping or ssh to/from
> > > > the guest from/to the host. Do not ask me if I can ping any other
> > > > host on that network - there are no other host on the network yet,
> > > > just the
> >
> > Host
> >
> > > > and the guest.
> > > >
> > > > Any suggestions?
> > >
> > > eth1 should not have an IP address:
> > > # ifconfig eth1 0.0.0.0
> > >
> > > br-eth1 is not activated (it's missing in `ifconfig`), and it needs an
> > > IP address:
> > > # ifconfig br-eth1 192.168.2.1/24
> > >
> > > Even if it works I would explicitly activate br-eth3 too:
> > > # ifconfig br-eth3 0.0.0.0 up
> > >
> > > Looking at the output of `ifconfig` shows that the IP-address of eth0
> > > (192.168.1.3) doesn't match 192.168.1.1 from your address list, and
> > > eth1 (192.168.4.1) is in a different network than the specified
> >
> > 192.168.2.1/24.
> >
> > OK, the 192.168.1.3 - 192.168.1.1 is my typo, 192.168.1.3 is the correct
> > IP for the Host on this interface.
> > I was first planning to use the 192.168.2.0/24 network here, but I had
> > forgotten that that was already used somewhere else in my network, so I
> > just
> > picked the next "free" network address. Sorry for the confusion.
> >
> > I have tried you suggestions above but I still have no network
> > connection.
> >
> > # ifconfig eth1
> > eth1 Link encap:Ethernet HWaddr 00:0d:88:52:51:25
> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> > TX packets:3 errors:0 dropped:0 overruns:0 carrier:3
> > collisions:0 txqueuelen:1000
> > RX bytes:0 (0.0 B) TX bytes:180 (180.0 B)
> > Interrupt:18 Base address:0xe880
> >
> >
> > # ifconfig br-eth1
> > br-eth1 Link encap:Ethernet HWaddr 00:0d:88:52:51:25
> > inet addr:192.168.4.1 Bcast:192.168.4.255 Mask:255.255.255.0
> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > RX packets:65 errors:0 dropped:0 overruns:0 frame:0
> > TX packets:62 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:0
> > RX bytes:3518 (3.4 KiB) TX bytes:4750 (4.6 KiB)
> >
> > # ifconfig br-eth3
> > br-eth3 Link encap:Ethernet HWaddr 00:0d:88:52:51:27
> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:0
> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
> >
> > Regards,
OK, lets simplify it a bit. This is now the layout that I am playing with and
trying to get KVM to work .
+-----------------------------------+
| Host |
| +----------+ eth0 |---- 192.168.1.0/24
| | VM eth1|------qtap1 |
| | | | |
| | eth0 | br-eth1---eth1 |---- 192.168.4.0/24
| +----------+ |
| |
+-----------------------------------+
Host-eth0 is only for the Host (no VM I/O)
Host-eth1 is shared between the Host and the VM (Host <--> VM, Host<--
>"external", VM <--> "external")
VM-eth0 is unused at the moment.
This is the commands I have been using to setup the Networking:
brctl addbr br-eth1
brctl setfd br-eth1 0
brctl sethello br-eth1 1
brctl stp br-eth1 off
brctl addif br-eth1 eth1
tunctl -b -t qtap1
brctl addif br-eth1 qtap1
ifconfig qtap1 up 0.0.0.0 promisc
ifconfig br-eth1 192.168.4.1/24 up
ifconfig eth1 0.0.0.0 up
And this is the way I start the VM:
kvm -net nic,vlan=1,model=rtl8139,macaddr=52:54:00:12:34:56 -net
tap,vlan=1,ifname=qtap1,script=no,downscript=no Robbie.img -m 1024 -k de-ch
On the Host I have the following:
# ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:0d:88:52:51:25
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:18 errors:0 dropped:0 overruns:0 carrier:18
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:1080 (1.0 KiB)
Interrupt:18 Base address:0xe880
# ifconfig br-eth1
br-eth1 Link encap:Ethernet HWaddr 00:0d:88:52:51:25
inet addr:192.168.4.1 Bcast:192.168.4.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:501 errors:0 dropped:0 overruns:0 frame:0
TX packets:479 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:26378 (25.7 KiB) TX bytes:35080 (34.2 KiB)
# ifconfig qtap1
qtap1 Link encap:Ethernet HWaddr da:77:3a:a9:40:23
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:503 errors:0 dropped:0 overruns:0 frame:0
TX packets:481 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:33524 (32.7 KiB) TX bytes:35240 (34.4 KiB)
# brctl show
bridge name bridge id STP enabled interfaces
br-eth1 8000.000d88525125 no eth1
qtap1
# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.4.0 0.0.0.0 255.255.255.0 U 0 0 0 br-
eth1
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
239.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 eth0
And in the VM I have:
# ifconfig
eth1 Link encap:Ethernet HWaddr 52:54:00:12:34:56
inet addr:192.168.4.4 Bcast:192.168.4.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12 errors:0 dropped:0 overruns:0 frame:0
TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1040 (1.0 KiB) TX bytes:1018 (1018.0 B)
Interrupt:11 Base address:0xc000
# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.4.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 192.168.4.1 0.0.0.0 UG 0 0 0 eth1
But Still I get the following:
>From the Host to the VM:
$ ssh root@xxxxxxxxxxx
ssh: connect to host 192.168.4.4 port 22: Connection refused
And from the VM to the Host:
# ssh root@xxxxxxxxxxx
ssh: connect to host 192.168.4.1 port 22: Connection refused
Any more suggestions?
--
Dan Johansson, <http://www.dmj.nu>
***************************************************
This message is printed on 100% recycled electrons!
***************************************************
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
