Hello Alex,
>From the results of the run you sent us, it appears that the IP of the bridge and the IP of the AGL guest are the same.
In this case, the ubuntu host tries to communicate with the bridge instead of the AGL guest.
For this reason, please change the IP of the AGL guest and try to communicate with it.
For your information, the results of my run are shown below.
=========================================
1. Start AGL using bridge.
-----------------------------------------
$ ip addr show eno1
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether f8:32:e4:6e:69:a1 brd ff:ff:ff:ff:ff:ff
inet 10.68.11.80/24 brd 10.68.11.255 scope global dynamic noprefixroute eno1
valid_lft 86090sec preferred_lft 86090sec
inet6 fe80::35ab:30bd:d7a4:d5b4/64 scope link noprefixroute
valid_lft forever preferred_lft forever
$ brctl show br-tmp
bridge name bridge id STP enabled interfaces
br-tmp can't get info No such device
$
$ sudo ip link add name br-tmp type bridge
$ sudo ip link set dev br-tmp up
$ sleep 1
$ ip addr show br-tmp
14: br-tmp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether 8e:5e:00:e6:5f:61 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.10/24 scope global br-tmp
valid_lft forever preferred_lft forever
inet6 fe80::8c5e:ff:fee6:5f61/64 scope link
valid_lft forever preferred_lft forever
$
$ cat /etc/qemu/bridge.conf
allow br-tmp
$ sudo qemu-system-x86_64 -enable-kvm -cpu kvm64 -cpu qemu64,+ssse3,+sse4.1,+sse4.2,+popcnt -m 2048 -machine q35 -netdev bridge,br=br-tmp,id=net0 -device virtio-net-pci,netdev=net0 -show-cursor -usb -usbdevice tablet -device virtio-rng-pci -vga virtio -vnc :0 -soundhw hda -serial mon:vc -serial mon:stdio -serial null -kernel bzImage -append 'root=/dev/vda rw console=tty0 mem=2048M oprofile.timer=1 console=ttyS0,115200n8 verbose fstab=no' -drive file=agl-demo-platform-crosssdk-qemux86-64.ext4,if=virtio,format=raw -snapshot
qemu-system-x86_64: -usbdevice tablet: '-usbdevice' is deprecated, please use '-device usb-...' instead
qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:ECX.svm [bit 2]
Home directory not accessible: Permission denied
pulseaudio: pa_context_connect() failed
pulseaudio: Reason: Connection refused
pulseaudio: Failed to initialize PA contextaudio: Could not init `pa' audio driver
Home directory not accessible: Permission denied
Automotive Grade Linux 13.90.0+snapshot-8c489602f218bcf21de0d3c9f8cf620ea5f06430 qemux86-64 ttyS1
qemux86-64 login: root
root@qemux86-64:~# ifconfig
enp0s3 Link encap:Ethernet HWaddr 52:54:00:12:91:69
inet6 addr: fe80::5054:ff:fe12:9169/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2 errors:0 dropped:0 overruns:0 frame:0
TX packets:20 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:388 (388.0 B) TX bytes:3500 (3.4 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:10 errors:0 dropped:0 overruns:0 frame:0
TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1594 (1.5 KiB) TX bytes:1594 (1.5 KiB)
root@qemux86-64:~# ifconfig enp0s3 192.168.1.20 netmask 255.255.255.0
root@qemux86-64:~#
=========================================
2. Ping from ubuntu host (using another terminal)
-----------------------------------------
$ ping 192.168.1.20 -c 3
PING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.
64 bytes from 192.168.1.20: icmp_seq=1 ttl=64 time=0.288 ms
64 bytes from 192.168.1.20: icmp_seq=2 ttl=64 time=0.288 ms
64 bytes from 192.168.1.20: icmp_seq=3 ttl=64 time=0.260 ms
--- 192.168.1.20 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2036ms
rtt min/avg/max/mdev = 0.260/0.278/0.288/0.023 ms
=========================================
One change to the command is to remove the option “ip=dhcp”.
If “ip=dhcp” is present, the console cannot be operated until the timeout is reached.
(The console can still be operated with the option if you wait long enough for the timeout).
The environment I have created is called a 'host only network'.
This means that AGL can communicate with the host, but not directly with the Internet on the WAN side.
[cid:image002.png@01D89203.7A919CB0]
I hope the above helps you.
Best Regard,
Kuzu,
Panasonic Automotive Systems Co., Ltd.
From: Alex Silva -T (alexsil - AEROTEK ULC at Cisco) <alexsil@xxxxxxxxx>
Sent: Thursday, July 7, 2022 12:17 AM
To: agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx; KUZU KAZUKI (葛生 一樹) <kuzu.kazuki@xxxxxxxxxxxxxxxx>; Zhao Jiancong(Jerry) (趙 健淙) <chou.kensou@xxxxxxxxxxxxxxxx>
Cc: SUDA SEIYA (須田 聖也) <suda.seiya@xxxxxxxxxxxxxxxx>; agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx
Subject: Re: Question about AGL hosted on QEMU on an Ubuntu VM (nested virtualization)
Hey Kuzu,
Failed to add: I tried moving files between AGL and Ubuntu as you mentioned, had “connection refused”, so definitely it’s not pinging/connecting to AGL, really hope the error is in how I’m running qemu…
Without bridge:
qemu-system-x86_64 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:35:02 -netdev user,id=net0,hostfwd=tcp::2222-:22 -drive file=agl-demo-platform-crosssdk-qemux86-64.ext4,if=virtio,format=raw -show-cursor -usb -usbdevice tablet -device virtio-rng-pci -vga virtio -vnc :0 -soundhw hda -machine q35 -cpu kvm64 -cpu qemu64,+ssse3,+sse4.1,+sse4.2,+popcnt -enable-kvm -m 2048 -serial mon:vc -serial mon:stdio -serial null -kernel bzImage -append 'root=/dev/vda rw console=tty0 mem=2048M ip=dhcp oprofile.timer=1 console=ttyS0,115200n8 verbose fstab=no'
With bridge (had to use sudo or bridge would fail, and would no longer accept hostfwd argument):
sudo qemu-system-x86_64 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:35:02 -netdev bridge,br=agl-bridge,id=net0 -drive file=agl-demo-platform-crosssdk-qemux86-64.ext4,if=virtio,format=raw -show-cursor -usb -usbdevice tablet -device virtio-rng-pci -vga virtio -vnc :0 -soundhw hda -machine q35 -cpu kvm64 -cpu qemu64,+ssse3,+sse4.1,+sse4.2,+popcnt -enable-kvm -m 2048 -serial mon:vc -serial mon:stdio -serial null -kernel bzImage -append 'root=/dev/vda rw console=tty0 mem=2048M ip=dhcp oprofile.timer=1 console=ttyS0,115200n8 verbose fstab=no'
Sincerely,
Alex Silva
From: agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx<mailto:agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx> <agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx<mailto:agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx>> on behalf of Alex Silva -T (alexsil - AEROTEK ULC at Cisco) via lists.automotivelinux.org <alexsil=cisco.com@xxxxxxxxxxxxxxxxxxxxxxxxx<mailto:alexsil=cisco.com@xxxxxxxxxxxxxxxxxxxxxxxxx>>
Date: Tuesday, July 5, 2022 at 11:36 AM
To: KUZU KAZUKI (葛生 一樹) <kuzu.kazuki@xxxxxxxxxxxxxxxx<mailto:kuzu.kazuki@xxxxxxxxxxxxxxxx>>, Zhao Jiancong(Jerry) (趙 健淙) <chou.kensou@xxxxxxxxxxxxxxxx<mailto:chou.kensou@xxxxxxxxxxxxxxxx>>
Cc: SUDA SEIYA (須田 聖也) <suda.seiya@xxxxxxxxxxxxxxxx<mailto:suda.seiya@xxxxxxxxxxxxxxxx>>, agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx<mailto:agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx> <agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx<mailto:agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx>>
Subject: Re: Question about AGL hosted on QEMU on an Ubuntu VM (nested virtualization)
Hello Kuzu,
No worries, you had indeed told m about how to start QEMU, I’m thinking maybe that’s part of the problem.
Look, I tried your instructions again on local Ubuntu (not VM), and this is what I got:
1. I was able to set up the bridge (specifying IP 192.168.1.10) without losing connectivity AFTER removing the previous one (virbr0) I had:
~/agl-demo$ sudo ip link add name agl-bridge type bridge
~/agl-demo$ sudo ip link set dev agl-bridge up
~/agl-demo$ sleep 1
~/agl-demo$ sudo ip addr add 192.168.1.10/24 dev agl-bridge
~/agl-demo$ sudo ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether fc:aa:14:52:c0:db brd ff:ff:ff:ff:ff:ff
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
link/ether 52:54:00:81:81:a7 brd ff:ff:ff:ff:ff:ff
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN mode DEFAULT group default qlen 1000
link/ether 52:54:00:81:81:a7 brd ff:ff:ff:ff:ff:ff
25: agl-bridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/ether d2:e6:4f:7a:79:4d brd ff:ff:ff:ff:ff:ff
~/agl-demo$ echo "allow agl-bridge" >> /etc/qemu/bridge.conf
~/agl-demo$ cat /etc/qemu/bridge.conf
allow agl-bridge
~/agl-demo$ ifconfig
agl-bridge: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.10 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::d8c1:75ff:fe85:d4aa prefixlen 64 scopeid 0x20<link>
ether da:c1:75:85:d4:aa txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 100 bytes 10188 (10.1 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp1s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.64 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::b15e:618d:6a9d:3abb prefixlen 64 scopeid 0x20<link>
ether fc:aa:14:52:c0:db txqueuelen 1000 (Ethernet)
RX packets 2626410 bytes 3352132863 (3.3 GB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 817090 bytes 149491929 (149.4 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 91312 bytes 8251360 (8.2 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 91312 bytes 8251360 (8.2 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 52:54:00:81:81:a7 txqueuelen 1000 (Ethernet)
RX packets 1153 bytes 382373 (382.3 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 371 bytes 57329 (57.3 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
~/agl-demo$ sudo nmcli conn show --active
NAME UUID TYPE DEVICE
Wired connection 1 6d8112e2-024f-3370-8ec5-4ea54021f108 ethernet enp1s0
agl-bridge 9b71d812-798d-4015-9b97-77a3fe8bdb4a bridge agl-bridge
virbr0 67904cdb-d46a-4cf6-b6c8-a69407b7d83a bridge virbr0
~/agl-demo$ sudo ip link set dev virbr0 down
~/agl-demo$ sudo brctl delbr virbr0
~/agl-demo$ sudo nmcli conn show --active
NAME UUID TYPE DEVICE
Wired connection 1 6d8112e2-024f-3370-8ec5-4ea54021f108 ethernet enp1s0
agl-bridge 9b71d812-798d-4015-9b97-77a3fe8bdb4a bridge agl-bridge
1. I can ping the IP address of the bridge just fine, but this happens with QEMU powered on or off as well (and even with QEMU launched without specifying the bridge), so how can I know I’m actually reaching AGL?
sudo qemu-system-x86_64 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:35:02 -netdev bridge,br=agl-bridge,id=net0 -drive file=agl-demo-platform-crosssdk-qemux86-64.ext4,if=virtio,format=raw -show-cursor -usb -usbdevice tablet -device virtio-rng-pci -vga virtio -vnc :0 -soundhw hda -machine q35 -cpu kvm64 -cpu qemu64,+ssse3,+sse4.1,+sse4.2,+popcnt -enable-kvm -m 2048 -serial mon:vc -serial mon:stdio -serial null -kernel bzImage -append 'root=/dev/vda rw console=tty0 mem=2048M ip=dhcp oprofile.timer=1 console=ttyS0,115200n8 verbose fstab=no'
…
qemux86-64 login: root
Last login: Tue Jul 5 16:31:48 UTC 2022 on ttyS1
root@qemux86-64:~# ifconfig
enp0s3 Link encap:Ethernet HWaddr 52:54:00:12:35:02
inet addr:169.254.52.77 Bcast:169.254.255.255 Mask:255.255.0.0
inet6 addr: fe80::5054:ff:fe12:3502/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:177 errors:0 dropped:0 overruns:0 frame:0
TX packets:85 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7951 (7.7 KiB) TX bytes:20676 (20.1 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:63 errors:0 dropped:0 overruns:0 frame:0
TX packets:63 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4218 (4.1 KiB) TX bytes:4218 (4.1 KiB)
root@qemux86-64:~# ifconfig enp0s3 192.168.1.10 netmask 255.255.255.0
root@qemux86-64:~# ip route add default via 192.168.1.10 dev enp0s3
root@qemux86-64:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.10 0.0.0.0 UG 0 0 0 enp0s3
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 enp0s3
root@qemux86-64:~# ifconfig
enp0s3 Link encap:Ethernet HWaddr 52:54:00:12:35:02
inet addr:192.168.1.10 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::5054:ff:fe12:3502/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:225 errors:0 dropped:0 overruns:0 frame:0
TX packets:105 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:9967 (9.7 KiB) TX bytes:28296 (27.6 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:143 errors:0 dropped:0 overruns:0 frame:0
TX packets:143 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:8138 (7.9 KiB) TX bytes:8138 (7.9 KiB)
second terminal (where I set up the bridge)
~/agl-demo$ ping 192.168.1.10
PING 192.168.1.10 (192.168.1.10) 56(84) bytes of data.
64 bytes from 192.168.1.10: icmp_seq=1 ttl=64 time=0.060 ms
64 bytes from 192.168.1.10: icmp_seq=2 ttl=64 time=0.064 ms
64 bytes from 192.168.1.10: icmp_seq=3 ttl=64 time=0.061 ms
64 bytes from 192.168.1.10: icmp_seq=4 ttl=64 time=0.062 ms
^C
--- 192.168.1.10 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3053ms
rtt min/avg/max/mdev = 0.060/0.061/0.064/0.001 ms
1. With QEMU specifying the bridge, I cannot ping either Ubuntu or reach the internet from AGL
root@qemux86-64:~# cat /etc/resolv.conf
# Generated by Connection Manager
nameserver 10.0.2.3
nameserver ::1
nameserver 127.0.0.1
root@qemux86-64:~# nslookup youtube.com
Server: 10.0.2.3
Address 1: 10.0.2.3
nslookup: can't resolve 'youtube.com'
root@qemux86-64:~# ping 192.168.1.64
PING 192.168.1.64 (192.168.1.64): 56 data bytes
^C
--- 192.168.1.64 ping statistics ---
27 packets transmitted, 0 packets received, 100% packet loss
1. Without the bridge, I can ping Ubuntu (192.168.1.64) and reach the internet from AGL
qemu-system-x86_64 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:35:02 -netdev user,id=net0,hostfwd=tcp::2222-:22 -drive file=agl-demo-platform-crosssdk-qemux86-64.ext4,if=virtio,format=raw -show-cursor -usb -usbdevice tablet -device virtio-rng-pci -vga virtio -vnc :0 -soundhw hda -machine q35 -cpu kvm64 -cpu qemu64,+ssse3,+sse4.1,+sse4.2,+popcnt -enable-kvm -m 2048 -serial mon:vc -serial mon:stdio -serial null -kernel bzImage -append 'root=/dev/vda rw console=tty0 mem=2048M ip=dhcp oprofile.timer=1 console=ttyS0,115200n8 verbose fstab=no'
…
root@qemux86-64:~# cat /etc/resolv.conf
# Generated by Connection Manager
nameserver 10.0.2.3
nameserver ::1
nameserver 127.0.0.1
root@qemux86-64:~# nslookup youtube.com
Server: 10.0.2.3
Address 1: 10.0.2.3
Name: youtube.com
Address 1: 142.251.46.238 sfo03s27-in-f14.1e100.net
Address 2: 2607:f8b0:4005:802::200e sfo07s26-in-x0e.1e100.net
root@qemux86-64:~# ping 192.168.1.64
PING 192.168.1.64 (192.168.1.64): 56 data bytes
64 bytes from 192.168.1.64: seq=0 ttl=255 time=0.274 ms
64 bytes from 192.168.1.64: seq=1 ttl=255 time=0.602 ms
^C
--- 192.168.1.64 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.274/0.438/0.602 ms
Could it be that I’m somehow specifying the parameters incorrectly?
Without bridge:
qemu-system-x86_64 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:35:02 -netdev user,id=net0,hostfwd=tcp::2222-:22 -drive file=agl-demo-platform-crosssdk-qemux86-64.ext4,if=virtio,format=raw -show-cursor -usb -usbdevice tablet -device virtio-rng-pci -vga virtio -vnc :0 -soundhw hda -machine q35 -cpu kvm64 -cpu qemu64,+ssse3,+sse4.1,+sse4.2,+popcnt -enable-kvm -m 2048 -serial mon:vc -serial mon:stdio -serial null -kernel bzImage -append 'root=/dev/vda rw console=tty0 mem=2048M ip=dhcp oprofile.timer=1 console=ttyS0,115200n8 verbose fstab=no'
With bridge (had to use sudo or bridge would fail, and would no longer accept hostfwd argument):
sudo qemu-system-x86_64 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:35:02 -netdev bridge,br=agl-bridge,id=net0 -drive file=agl-demo-platform-crosssdk-qemux86-64.ext4,if=virtio,format=raw -show-cursor -usb -usbdevice tablet -device virtio-rng-pci -vga virtio -vnc :0 -soundhw hda -machine q35 -cpu kvm64 -cpu qemu64,+ssse3,+sse4.1,+sse4.2,+popcnt -enable-kvm -m 2048 -serial mon:vc -serial mon:stdio -serial null -kernel bzImage -append 'root=/dev/vda rw console=tty0 mem=2048M ip=dhcp oprofile.timer=1 console=ttyS0,115200n8 verbose fstab=no'
I do appreciate your help, really do (and yeah, I’ll try to attend tomorrow’s meeting).
Sincerely,
Alex Silva
From: KUZU KAZUKI (葛生 一樹) <kuzu.kazuki@xxxxxxxxxxxxxxxx<mailto:kuzu.kazuki@xxxxxxxxxxxxxxxx>>
Date: Monday, July 4, 2022 at 9:59 PM
To: Alex Silva -T (alexsil - AEROTEK ULC at Cisco) <alexsil@xxxxxxxxx<mailto:alexsil@xxxxxxxxx>>, Zhao Jiancong(Jerry) (趙 健淙) <chou.kensou@xxxxxxxxxxxxxxxx<mailto:chou.kensou@xxxxxxxxxxxxxxxx>>
Cc: SUDA SEIYA (須田 聖也) <suda.seiya@xxxxxxxxxxxxxxxx<mailto:suda.seiya@xxxxxxxxxxxxxxxx>>, agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx<mailto:agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx> <agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx<mailto:agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx>>
Subject: RE: Question about AGL hosted on QEMU on an Ubuntu VM (nested virtualization)
Hello Alex,
I am so sorry to forgot to tell you to change the startup options for QEMU.
In order to use the created bridge with QEMU, I configured the network options as follows:
-netdev bridge,br=br-tmp,id=net0 -device virtio-net-pci,netdev=net0
After creating br-tmp according to the previously sent instructions, please change the QEMU network settings to the above and try again.
This should also work on a local workstation, so it may be a good idea to try it out before the cloud.
What about discussing directly in this week AGL Virtualization Expert Group meeting with following details.
- Location: Zoom (https://zoom.us/j/855006400?pwd=OER2bGVaVm91VjhKYm83Rk1hZTg2dz09)
- Time: 2022/7/6 12:00 (UTC)
Looking forward to seeing you in the Virt-EG!
Best Regard,
Kuzu,
Panasonic Automotive Systems Co., Ltd.
From: Alex Silva -T (alexsil - AEROTEK ULC at Cisco) <alexsil@xxxxxxxxx<mailto:alexsil@xxxxxxxxx>>
Sent: Tuesday, July 5, 2022 6:53 AM
To: KUZU KAZUKI (葛生 一樹) <kuzu.kazuki@xxxxxxxxxxxxxxxx<mailto:kuzu.kazuki@xxxxxxxxxxxxxxxx>>; Zhao Jiancong(Jerry) (趙 健淙) <chou.kensou@xxxxxxxxxxxxxxxx<mailto:chou.kensou@xxxxxxxxxxxxxxxx>>
Cc: SUDA SEIYA (須田 聖也) <suda.seiya@xxxxxxxxxxxxxxxx<mailto:suda.seiya@xxxxxxxxxxxxxxxx>>; agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx<mailto:agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Question about AGL hosted on QEMU on an Ubuntu VM (nested virtualization)
Hello Kuzu,
I followed these steps on my Ubuntu workstation (local machine, not a VM), and had the problem of every time I started QEMU, the ethernet connection would be lost and I had to delete the new bridge to recover (even restarting network manager would be of no use otherwise).
Why I tried it first on a local workstation: in case the network connection became affected, and it did.
I had to follow a different guide to get the bridge working (works perfectly with a local Ubuntu workstation).
Works there just fine:
root@qemux86-64:~# ifconfig
eth0 Link encap:Ethernet HWaddr 52:54:00:12:35:02
inet addr:192.168.122.64 Bcast:192.168.122.255 Mask:255.255.255.0
inet6 addr: fe80::5054:ff:fe12:3502/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:131 errors:0 dropped:0 overruns:0 frame:0
TX packets:90 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:10157 (9.9 KiB) TX bytes:12945 (12.6 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:11 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1670 (1.6 KiB) TX bytes:1670 (1.6 KiB)
root@qemux86-64:~# ls -lat /etc/resolv.conf
lrwxrwxrwx 1 root root 24 Jun 29 2020 /etc/resolv.conf -> /etc/resolv-conf.connman
root@qemux86-64:~# cat /etc/resolv.conf
# Generated by Connection Manager
nameserver 192.168.122.1
nameserver ::1
nameserver 127.0.0.1
root@qemux86-64:~#
I can indeed ping “192.168.122.64” from Ubuntu just fine.
However, I cannot do the same in my needed setup: where host Ubuntu is an AWS hosted VM itself.
This is what the bridge looks like in my local Ubuntu (my bridge is “virbr0”):
al666940@al666940-To-be-filled-by-O-E-M:~/agl-demo$ sudo nmcli connection show
NAME UUID TYPE DEVICE
Wired connection 1 6d8112e2-024f-3370-8ec5-4ea54021f108 ethernet enp1s0
virbr0 2658fd55-b813-4a02-9cd8-c769d68d047b bridge virbr0
And this is what it looks like on my AWS hosted VM Ubuntu:
alexsil@ip-172-31-36-149:~$ sudo nmcli connection show
[sudo] password for alexsil:
NAME UUID TYPE DEVICE
agl-bridge 343bf396-720d-42b4-a655-df5313980cec bridge --
bridge-agl-bridge 7e945a36-32ef-4676-a4fc-519468f04861 ethernet --
And of course, if I try to start it:
alexsil@ip-172-31-36-149:~$ sudo nmcli conn up agl-bridge
Error: Connection activation failed: Activation failed because the device is unmanaged
I think that’s the main roadblock.
I tried creating the interface using /etc/netplan/01-network-manager-all.yaml, but after applying it the networking failed and indeed I lost connectivity forever (have to recreate the whole thing again).
Sincerely,
Alex Silva
From: KUZU KAZUKI (葛生 一樹) <kuzu.kazuki@xxxxxxxxxxxxxxxx<mailto:kuzu.kazuki@xxxxxxxxxxxxxxxx>>
Date: Thursday, June 30, 2022 at 11:25 PM
To: Alex Silva -T (alexsil - AEROTEK ULC at Cisco) <alexsil@xxxxxxxxx<mailto:alexsil@xxxxxxxxx>>, Zhao Jiancong(Jerry) (趙 健淙) <chou.kensou@xxxxxxxxxxxxxxxx<mailto:chou.kensou@xxxxxxxxxxxxxxxx>>
Cc: SUDA SEIYA (須田 聖也) <suda.seiya@xxxxxxxxxxxxxxxx<mailto:suda.seiya@xxxxxxxxxxxxxxxx>>, agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx<mailto:agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx> <agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx<mailto:agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx>>
Subject: RE: Question about AGL hosted on QEMU on an Ubuntu VM (nested virtualization)
Hello Alex,
I could ping AGL VM from host Ubuntu usnig bridge.
You can reproduce the connection as follow:
1. Check the ubuntu host IP (in my environment, 10.0.0.201).
2. Configure a bridge on the ubuntu host and assign a different network than the above IP.
# ip link add name br-tmp type bridge
# ip link set dev br-tmp up
# sleep 1
# ip addr add 192.168.1.10/24 dev br-tmp
# ip link show br-tmp
3. Allow qemu to use br-tmp.
# echo "allow br-tmp" >> /etc/qemu/bridge.conf
4. Start an AGL VM with the following options instead of "user" network options:
-netdev bridge,br=br-tmp,id=net0 -device virtio-net-pci,netdev=net0
5. In the AGL VM, configure the ip address and default gateway as follows:
# ifconfig enp0s2 192.168.1.20 netmask 255.255.255.0
# ip route add default via 192.168.1.10 dev enp0s2
# route -n
6. You can now ping the AGL guest from the Ubuntu host.
# ping 192.168.1.20
# spc root@192.168.1.20:/xxxx<mailto:root@192.168.1.20:/xxxx> yyy
Please check if you can ping using the above way.
Best Regards,
Kuzu
From: Alex Silva -T (alexsil - AEROTEK ULC at Cisco) <alexsil@xxxxxxxxx<mailto:alexsil@xxxxxxxxx>>
Sent: Friday, July 1, 2022 6:37 AM
To: KUZU KAZUKI (葛生 一樹) <kuzu.kazuki@xxxxxxxxxxxxxxxx<mailto:kuzu.kazuki@xxxxxxxxxxxxxxxx>>; Zhao Jiancong(Jerry) (趙 健淙) <chou.kensou@xxxxxxxxxxxxxxxx<mailto:chou.kensou@xxxxxxxxxxxxxxxx>>
Cc: SUDA SEIYA (須田 聖也) <suda.seiya@xxxxxxxxxxxxxxxx<mailto:suda.seiya@xxxxxxxxxxxxxxxx>>; agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx<mailto:agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Question about AGL hosted on QEMU on an Ubuntu VM (nested virtualization)
Hello Kuzu,
I managed to share files using option 1, thanks a whole bunch!
As for making the AGL reachable (pingable) to other VM’s, that is indeed very important. Any chance you can share how the network bridge was set up? I’ll run with that and figure how if at all can be implemented on AS VM’s (hey, if AGL can be set up on AWS hosetd VM, then we can surely enable a bridge on that as well).
Sincerely,
Alex Silva
From: KUZU KAZUKI (葛生 一樹) <kuzu.kazuki@xxxxxxxxxxxxxxxx<mailto:kuzu.kazuki@xxxxxxxxxxxxxxxx>>
Date: Tuesday, June 28, 2022 at 8:32 PM
To: Alex Silva -T (alexsil - AEROTEK ULC at Cisco) <alexsil@xxxxxxxxx<mailto:alexsil@xxxxxxxxx>>, Zhao Jiancong(Jerry) (趙 健淙) <chou.kensou@xxxxxxxxxxxxxxxx<mailto:chou.kensou@xxxxxxxxxxxxxxxx>>
Cc: SUDA SEIYA (須田 聖也) <suda.seiya@xxxxxxxxxxxxxxxx<mailto:suda.seiya@xxxxxxxxxxxxxxxx>>, agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx<mailto:agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx> <agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx<mailto:agl-dev-community@xxxxxxxxxxxxxxxxxxxxxxxxx>>
Subject: RE: Question about AGL hosted on QEMU on an Ubuntu VM (nested virtualization)
Hello Alex,
Thank you for asking.
If you want to copy files between an AGL VM and an ubuntu host, you have two options.
- Start AGL VM without the snapshot option and copy files from the console of AGL VM using scp.
- Rewrite ext4 file using loop device as following:
$ lodev=$(losetup -f)
$ sudo losetup $lodev agl-demo-polatform-crosssdk-qemuarm64.ext4
$ sudo mount $lodev /mnt
$ cp filename /mnt/target_location
$ sudo umount /mnt
$ sudo losetup -d $lodev
About how to ping the AGL VM from other AWS hosted VM's, I continue to check that.
Currently, I assume that the qemu networking configuration option are not suitable for that purpose.
Specifying "user" in the net option sets up a NAT network in the QEMU network system.
Therefore, ping from the AGL VM to the host OS is possible, but not in the reverse direction.
A possible one way to ping the AGL VM from the host OS is to use a bridge.
I have confirmed that pinging from the host OS to the AGL VM can be done using bridge on local Raspberry Pi, but this has not been confirmed so far on AWS.
Also in case of not reponsing your email in time, let me CC the community group to see if any other member can help you also.
Best Regards,
Kuzu
From: Alex Silva -T (alexsil - AEROTEK ULC at Cisco) <alexsil@xxxxxxxxx<mailto:alexsil@xxxxxxxxx>>
Sent: Monday, June 27, 2022 11:33 PM
To: Zhao Jiancong(Jerry) (趙 健淙) <chou.kensou@xxxxxxxxxxxxxxxx<mailto:chou.kensou@xxxxxxxxxxxxxxxx>>; KUZU KAZUKI (葛生 一樹) <kuzu.kazuki@xxxxxxxxxxxxxxxx<mailto:kuzu.kazuki@xxxxxxxxxxxxxxxx>>
Cc: SUDA SEIYA (須田 聖也) <suda.seiya@xxxxxxxxxxxxxxxx<mailto:suda.seiya@xxxxxxxxxxxxxxxx>>
Subject: Re: Question about AGL hosted on QEMU on an Ubuntu VM (nested virtualization)
Hello Jerry,
Another question that I have is, well, how exactly do you enable file sharing between AGL and the hosting computer?
I ask because to do that, as far as I know, using QEMU and KVM, one has to create a VM first.
And since AGL doesn’t seem to come with an ISO, but a prebuilt image and kernel (for QEMU), can’t exactly create (or start) a VM out of those.
https://download.automotivelinux.org/AGL/snapshots/master/latest/qemux86-64/deploy/images/qemux86-64/agl-demo-platform-crosssdk-qemux86-64.ext4.xz (which when unpacked is ext4)
https://download.automotivelinux.org/AGL/snapshots/master/latest/qemux86-64/deploy/images/qemux86-64/bzImage
Thus, can’t seem to figure how to enable file sharing with a prebuilt image.
Sincerely,
Alex Silva
From: Alex Silva -T (alexsil - AEROTEK ULC at Cisco) <alexsil@xxxxxxxxx<mailto:alexsil@xxxxxxxxx>>
Date: Wednesday, June 22, 2022 at 7:54 AM
To: Zhao Jiancong(Jerry) (趙 健淙) <chou.kensou@xxxxxxxxxxxxxxxx<mailto:chou.kensou@xxxxxxxxxxxxxxxx>>, KUZU KAZUKI (葛生 一樹) <kuzu.kazuki@xxxxxxxxxxxxxxxx<mailto:kuzu.kazuki@xxxxxxxxxxxxxxxx>>
Cc: SUDA SEIYA (須田 聖也) <suda.seiya@xxxxxxxxxxxxxxxx<mailto:suda.seiya@xxxxxxxxxxxxxxxx>>
Subject: Re: Question about AGL hosted on QEMU on an Ubuntu VM (nested virtualization)
Hello Jerry,
Thanks for the reply!
1. I use a bare-metal instance type (deployed on a dedicated host), running Ubuntu server 22.04, to host AGL via QEMU
* For x86: i3.metal (ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-20220420)
* For 64-bit ARM: r6g.metal (ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-arm64-server-20220420)
2. The VM hosting AGL via QEMU (let’s call it “QEMU VM”) can ping other VM’s and other VM’s can ping this VM just fine
* Communication between AGKL hosting VM and other VM’s works both ways
3. The AGL instance itself can ping other VM’s just fine, BUT other VM’s cannot ping the AGL instance itself
* Communication works only one way (Ping from AGL instance to other cloud instance: OK)
I fear I may have to do some complex routing or network bridge setup, due to the private IP I see when running ifconfig within AGL:
root@qemux86-64:~# ifconfig
eth0 Link encap:Ethernet HWaddr 52:54:00:12:35:02
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fec0::5054:ff:fe12:3502/64 Scope:Site
inet6 addr: fe80::5054:ff:fe12:3502/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:186 errors:0 dropped:0 overruns:0 frame:0
TX packets:371 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:18723 (18.2 KiB) TX bytes:37344 (36.4 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:11 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1670 (1.6 KiB) TX bytes:1670 (1.6 KiB)
I appreciate the assistance really.
Sincerely,
Alex Silva
From: Zhao Jiancong(Jerry) (趙 健淙) <chou.kensou@xxxxxxxxxxxxxxxx<mailto:chou.kensou@xxxxxxxxxxxxxxxx>>
Date: Tuesday, June 21, 2022 at 6:41 PM
To: Alex Silva -T (alexsil - AEROTEK ULC at Cisco) <alexsil@xxxxxxxxx<mailto:alexsil@xxxxxxxxx>>, KUZU KAZUKI (葛生 一樹) <kuzu.kazuki@xxxxxxxxxxxxxxxx<mailto:kuzu.kazuki@xxxxxxxxxxxxxxxx>>
Cc: SUDA SEIYA (須田 聖也) <suda.seiya@xxxxxxxxxxxxxxxx<mailto:suda.seiya@xxxxxxxxxxxxxxxx>>
Subject: RE: Question about AGL hosted on QEMU on an Ubuntu VM (nested virtualization)
Hi Alex,
CC: @KUZU KAZUKI (葛生 一樹)<mailto:kuzu.kazuki@xxxxxxxxxxxxxxxx>
Nice to meeting you. Thanks for using virtual AGL.
Let me confirm several things.
* Which AWS server type are you using? Graviton or Non-graviton? Baremetal or normal EC2?
* Do you mean the communication is only in single direction?
* Ping from AGL instance to other cloud instance: OK
* Ping from other cloud instance to AGL instance: NOK
Sincerely,
Jerry
Panasonic Automotive
From: Alex Silva -T (alexsil - AEROTEK ULC at Cisco) <alexsil@xxxxxxxxx<mailto:alexsil@xxxxxxxxx>>
Sent: Wednesday, June 22, 2022 8:43 AM
To: Zhao Jiancong(Jerry) (趙 健淙) <chou.kensou@xxxxxxxxxxxxxxxx<mailto:chou.kensou@xxxxxxxxxxxxxxxx>>
Subject: Question about AGL hosted on QEMU on an Ubuntu VM (nested virtualization)
Hello Jerry,
I’m Alex Silva, working with Cisco, and was trying to use AGL (hosted on an Ubuntu VM in AWS, via QEMU).
Long story short: I need to be able to ping (reach) the AGL instance from other AWS hosted VM’s.
I followed the instructions here to deploy AGL Linux on Ubuntu via QEMU
https://docs.automotivelinux.org/en/master/#0_Getting_Started/1_Quickstart/Using_Ready_Made_Images/#_top
And I managed to have AGL up and running on both AMD and ARM architectures.
And AGL instance seems to work and connect to other VM’s just fine (it can ping the other Ubuntu VM’s o the network like for example “54.213.111.78”, NSLOOKUP works, WGET works, etc), I can even see the GUI if I use Vinagre (on gnome desktop).
Automotive Grade Linux 13.90.0+snapshot-8c489602f218bcf21de0d3c9f8cf620ea5f06430 qemux86-64 ttyS1
qemux86-64 login: root
root@qemux86-64:~# ifconfig
eth0 Link encap:Ethernet HWaddr 52:54:00:12:35:02
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::5054:ff:fe12:3502/64 Scope:Link
inet6 addr: fec0::5054:ff:fe12:3502/64 Scope:Site
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:142 errors:0 dropped:0 overruns:0 frame:0
TX packets:303 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:13701 (13.3 KiB) TX bytes:31065 (30.3 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:11 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1670 (1.6 KiB) TX bytes:1670 (1.6 KiB)
root@qemux86-64:~# ping 54.213.111.78
PING 54.213.111.78 (54.213.111.78): 56 data bytes
64 bytes from 54.213.111.78: seq=0 ttl=255 time=0.658 ms
64 bytes from 54.213.111.78: seq=1 ttl=255 time=0.656 ms
64 bytes from 54.213.111.78: seq=2 ttl=255 time=0.626 ms
64 bytes from 54.213.111.78: seq=3 ttl=255 time=0.736 ms
^C
--- 54.213.111.78 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.626/0.669/0.736 ms
HOWEVER, I can’t seem to reach the AGL instance (ping it’s IP, I can only see it’s private one which is “10.0.2.15”) from anywhere, not even from the VM hosting it.
So I’m guessing I’m missing something rather obvious here (obvious for you guys I hope) in my setup.
I was wondering if I could ask you about it (I’m guessing you guys must be able to connect to your AGL running instances somehow).
The Confluence page directs inquiries to you, hence my email asking about this.
[cid:image003.png@01D89203.7A919CB0]
Sincerely,
Alex Silva
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#9945): https://lists.automotivelinux.org/g/agl-dev-community/message/9945
Mute This Topic: https://lists.automotivelinux.org/mt/92058762/2167316
Group Owner: agl-dev-community+owner@xxxxxxxxxxxxxxxxxxxxxxxxx
Unsubscribe: https://lists.automotivelinux.org/g/agl-dev-community/leave/4543822/2167316/883735764/xyzzy [list-automotive-discussions82@xxxxxxxxxxx]
-=-=-=-=-=-=-=-=-=-=-=-
