Re: AGL / NFS

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Just a question:
i have to add uboot header with below command, other wise bootz says bad image!

mkimage -A arm -O linux -T ramdisk -d initramfs-icmm.ext4.gz uRamdisk 

is it normal or should init ramdisk image works out of the box?

regards
vahid
 

On Sat, Oct 12, 2019 at 3:08 PM Vahid Gharaee <vgharaee@xxxxxxxxx> wrote:
Dear Stephane,

netboot booted perfectly well.
the problem was due to a strange bug in my dt. (a clk setting) 
ping was working but none of the services were!!   (turned out i should not have been sure of net :)
which led to nbd-client getting time out. now it works fine.

thanks for all your help
vahid


On Thu, Oct 10, 2019 at 2:38 PM Vahid Gharaee <vgharaee@xxxxxxxxx> wrote:
Dear Stephane,
Thanks for the detailed guide.

I am sure of network because even during "nbd-client trying to connect..." I can ping the board from the host.
tried the dhcp instead of static ip and result was the same.
I will continue with nbd.debug.
it gives me a prompt. on that prompt i cansee that i have the /dev/nbd0
also i can ping the host. but nbd-client does not run (timeout)
strange is that when I boot from normal sdcart nbd-client and connecting to host through that is working.
the problem happens when booting from initramdisk.
I will debug further and tell you where did i get.

thanks again for the information.
Vahid

On Thu, Oct 10, 2019 at 1:09 PM Stephane Desneux <stephane.desneux@xxxxxxx> wrote:
Hi Vahid,

This seems to be an issue with the network setup.

1) check the kernel command line

Can you dump the kernel command line arguments? (these are logged by the kernel
at the beginning: search for "Kernel command line" in kernel log). On a Renesas
(H3+KF) setup, I have this command line:

console=ttySC0,115200 ignore_loglevel vmalloc=384M video=HDMI-A-1:1920x1080-32@6
0 root=/dev/ram0 ramdisk_size=16384 ip=dhcp rw rootfstype=ext4 rootwait
rootdelay=2 nbd.server=10.0.0.11

'ip=dhcp' is the important parameter for netboot :) Note that you can also put
an IP address (dhcp server is not mandatory, but just convenient)

2) check if the kernel has initialized the first network interface correctly

If the command line is ok, do you see IP initialization later in the kernel log,
like this:

[    5.129353] Micrel KSZ9031 Gigabit PHY e6800000.ethernet-ffffffff:00:
attached PHY driver [Micrel KSZ9031 Gigabit
 PHY] (mii_bus:phy_addr=e6800000.ethernet-ffffffff:00, irq=236)
[    8.331681] ravb e6800000.ethernet eth0: Link is Up - 1Gbps/Full - flow
control rx/tx
[    8.354037] Sending DHCP requests ., OK
[    8.373881] IP-Config: Got DHCP answer from 10.0.0.1, my address is 10.0.0.30
[    8.381017] IP-Config: Complete:
[    8.384246]      device=eth0, hwaddr=2e:0a:0a:00:87:be, ipaddr=10.0.0.30,
mask=255.255.255.0, gw=10.0.0.1
[    8.393811]      host=h3ulcb, domain=homelan.sdx, nis-domain=(none)
[    8.400076]      bootserver=10.0.0.5, rootserver=10.0.0.5, rootpath=
nameserver0=10.0.0.1

3) investigate manually by using nbd.debug

If you want to debug the initramfs sequence (mounting nbd device etc.), you can
add nbd.debug=1 in the kernel command line (in uboot AFAICS). This will set some
breakpoints in the initramfs script and will give you a shell prompt: you can
then investigate further manually.

Hope this helps!
---
Stephane Desneux - CTO - IoT.bzh
stephane.desneux@xxxxxxx - www.iot.bzh

On 10/10/2019 09:08, Vahid Gharaee wrote:
> UPDATE:
>
> when i boot up the board with normal sdcard and then run nbd-client:
> *nbd-client 192.168.10.1 10809 /dev/nbd0*
> It starts fine and connects to nbd-server.
> but with initramdisk i keep getting this:
> ----------------------------------------------------------------------------
>          ___   ________                _   __     __  __                __
>         /   | / ____/ /               / | / /__  / /_/ /_  ____  ____  / /_
>        / /| |/ / __/ /      ______   /  |/ / _ \/ __/ __ \/ __ \/ __ \/ __/
>       / ___ / /_/ / /___   /_____/  / /|  /  __/ /_/ /_/ / /_/ / /_/ / /_
>      /_/  |_\____/_____/           /_/ |_/\___/\__/_.___/\____/\____/\__/
> ________________________________________________________________________________
> /sbin/init[1]: starting initrd script
> /sbin/init[1]: mounting FS: proc /proc
> /sbin/init[1]: mounting FS: sysfs /sys
> /sbin/init[1]: mounting FS: devtmpfs /dev
> /sbin/init[1]: /dev (devtmpfs) already mounted
> /sbin/init[1]: mounting FS: devpts /dev/pts
> /sbin/init[1]: mounting FS: tmpfs /dev/shm
> /sbin/init[1]: mounting FS: tmpfs /tmp
> /sbin/init[1]: mounting FS: tmpfs /run
> /sbin/init[1]: NBD parameters: device /dev/nbd0, server 192.168.10.1:10809
> <http://192.168.10.1:10809>
> /sbin/init[1]: mounting FS: smackfs /sys/fs/smackfs
> /sbin/init[1]: Starting NBD client
> nbd-client: can't connect to remote host (192.168.10.1): Connection timed out
> /sbin/init[1]: NBD client failed
> /sbin/init[1]: Retrying (5 trie(s) left)...
> /sbin/init[1]: Starting NBD client
> random: crng init done
> nbd-client: can't connect to remote host (192.168.10.1): Connection timed out
> /sbin/init[1]: NBD client failed
> /sbin/init[1]: Retrying (4 trie(s) left)...
> /sbin/init[1]: Starting NBD client
> nbd-client: can't connect to remote host (192.168.10.1): Connection timed out
> /sbin/init[1]: NBD client failed
> /sbin/init[1]: Retrying (3 trie(s) left)...
> /sbin/init[1]: Starting NBD client
> nbd-client: can't connect to remote host (192.168.10.1): Connection timed out
> /sbin/init[1]: NBD client failed
> /sbin/init[1]: Retrying (2 trie(s) left)...
> /sbin/init[1]: Starting NBD client
> nbd-client: can't connect to remote host (192.168.10.1): Connection timed out
> /sbin/init[1]: NBD client failed
> /sbin/init[1]: Retrying (1 trie(s) left)...
> /sbin/init[1]: Starting NBD client
> nbd-client: can't connect to remote host (192.168.10.1): Connection timed out
> /sbin/init[1]: NBD client failed
> /sbin/init[1]: ERROR Unable to mount NBD device /dev/nbd0 using server
> 192.168.10.1:10809 <http://192.168.10.1:10809>
> /sbin/init[1]: Rebooting...
> ci_hdrc ci_hdrc.0: remove, state 4
> usb usb1: USB disconnect, device number 1
> ci_hdrc ci_hdrc.0: USB bus 1 deregistered
> reboot: Restarting system
> ------------------------------------------------
>
>
> could anyone help me?
> thanks
> vahid
>
> On Tue, Oct 8, 2019 at 7:04 PM Vahid Gharaee <vgharaee@xxxxxxxxx
> <mailto:vgharaee@xxxxxxxxx>> wrote:
>
>     UPDATE:
>     checked the kernel config:
>     CONFIG_BLK_DEV_NBD=y
>
>     appreciate your help.
>     Vahid
>
>
>     On Mon, Oct 7, 2019 at 4:16 PM Vahid Gharaee <vgharaee@xxxxxxxxx
>     <mailto:vgharaee@xxxxxxxxx>> wrote:
>
>         UPDATE:
>         now that netboot is up, it can not connect to xnbd on the host.
>
>
>         command and log on host:
>         ------------------------------------------
>         *sudo xnbd-server --target --lport 10809
>         agl-demo-platform-imx6qdlsabreauto-20190804063210.rootfs.ext4*
>         *
>         *
>         xnbd-server(23532) msg: listen port 10809
>         xnbd-server(23532) msg: cmd target mode
>         xnbd-server(23532) WARN: disksize 1762785280 is not a multiple of a page
>         size (4096)
>         xnbd-server(23532) WARN: disksize 1762785280 is not a multiple of 4096
>         (xnbd's cache block size)
>         xnbd-server(23532) msg: disk
>         /home/vahid/YOCTO/rootfs_images/agl-demo-platform-imx6solo-icmm1010-20190908052044.rootfs.ext4
>         size 1762785280 B (1681 MB)
>         xnbd-server(23532) WARN: disksize is not a multiple of CBLOCKSIZE
>         xnbd-server(23532) msg: xnbd master initialization done
>         xnbd-server(23532) msg: server 0.0.0.0:10809
>         <http://0.0.0.0:10809>,TCP,fd=4,reuseaddr,nodelay
>         xnbd-server(23532) msg: server [::]:10809,TCP,fd=5,reuseaddr,nodelay
>         xnbd-server(23532) msg: start polling
>         -----------------------------------------------------
>
>
>         log on target:
>         -----------------------------------------------------
>         Run /sbin/init as init process
>         ________________________________________________________________________________
>                  ___   ________                _   __     __  __                __
>                 /   | / ____/ /               / | / /__  / /_/ /_  ____  ____  / /_
>                / /| |/ / __/ /      ______   /  |/ / _ \/ __/ __ \/ __ \/ __ \/ __/
>               / ___ / /_/ / /___   /_____/  / /|  /  __/ /_/ /_/ / /_/ / /_/ / /_
>              /_/  |_\____/_____/           /_/ |_/\___/\__/_.___/\____/\____/\__/
>         ________________________________________________________________________________
>         /sbin/init[1]: starting initrd script
>         /sbin/init[1]: mounting FS: proc /proc
>         /sbin/init[1]: mounting FS: sysfs /sys
>         /sbin/init[1]: mounting FS: devtmpfs /dev
>         /sbin/init[1]: /dev (devtmpfs) already mounted
>         /sbin/init[1]: mounting FS: devpts /dev/pts
>         /sbin/init[1]: mounting FS: tmpfs /dev/shm
>         /sbin/init[1]: mounting FS: tmpfs /tmp
>         /sbin/init[1]: mounting FS: tmpfs /run
>         /sbin/init[1]: NBD parameters: device /dev/nbd0, server :10809
>         /sbin/init[1]: mounting FS: smackfs /sys/fs/smackfs
>         /sbin/init[1]: Starting NBD client
>         BusyBox v1.29.3 (2019-10-02 15:12:01 UTC) multi-call binary.
>
>         Usage: nbd-client HOST PORT BLOCKDEV
>         /sbin/init[1]: NBD client failed
>         /sbin/init[1]: Retrying (5 trie(s) left)...
>         /sbin/init[1]: Starting NBD client
>         BusyBox v1.29.3 (2019-10-02 15:12:01 UTC) multi-call binary.
>
>         Usage: nbd-client HOST PORT BLOCKDEV
>         /sbin/init[1]: NBD client failed
>         /sbin/init[1]: Retrying (4 trie(s) left)...
>         /sbin/init[1]: Starting NBD client
>         BusyBox v1.29.3 (2019-10-02 15:12:01 UTC) multi-call binary.
>
>         Usage: nbd-client HOST PORT BLOCKDEV
>         /sbin/init[1]: NBD client failed
>         /sbin/init[1]: Retrying (3 trie(s) left)...
>         /sbin/init[1]: Starting NBD client
>         BusyBox v1.29.3 (2019-10-02 15:12:01 UTC) multi-call binary.
>
>         Usage: nbd-client HOST PORT BLOCKDEV
>         /sbin/init[1]: NBD client failed
>         /sbin/init[1]: Retrying (2 trie(s) left)...
>         /sbin/init[1]: Starting NBD client
>         BusyBox v1.29.3 (2019-10-02 15:12:01 UTC) multi-call binary.
>
>         Usage: nbd-client HOST PORT BLOCKDEV
>         /sbin/init[1]: NBD client failed
>         /sbin/init[1]: Retrying (1 trie(s) left)...
>         /sbin/init[1]: Starting NBD client
>         BusyBox v1.29.3 (2019-10-02 15:12:01 UTC) multi-call binary.
>
>         Usage: nbd-client HOST PORT BLOCKDEV
>         /sbin/init[1]: NBD client failed
>         /sbin/init[1]: ERROR Unable to mount NBD device /dev/nbd0 using server
>         :10809
>         /sbin/init[1]: Rebooting...
>         ci_hdrc ci_hdrc.0: remove, state 4
>         usb usb1: USB disconnect, device number 1
>         ci_hdrc ci_hdrc.0: USB bus 1 deregistered
>         reboot: Restarting system
>         -----------------------------------------------------------------------
>         *
>         *
>         appriciate any help.
>         kind regards
>         Vahid
>         *
>         *
>
>         On Mon, Oct 7, 2019 at 3:21 PM Vahid Gharaee <vgharaee@xxxxxxxxx
>         <mailto:vgharaee@xxxxxxxxx>> wrote:
>
>             UPDATE:
>
>             when I ran below command it worked to netboot successfully:
>             *
>             *
>             *$ mkimage -A arm -O linux -T ramdisk -d initramfs-icmm.ext4.gz
>             uRamdisk*
>             (then i used uRamdisk file with bootz)
>
>             this just add uboot header.(if im not mistaken)
>             please enlighten me why the output of agl build did not work out of
>             the bux?
>             or sth. is wrong with my way of using it.
>
>             kind regards
>             Vahid
>
>
>
>             On Mon, Oct 7, 2019 at 12:45 PM Vahid Gharaee <vgharaee@xxxxxxxxx
>             <mailto:vgharaee@xxxxxxxxx>> wrote:
>
>                 Dear Jan,
>                 Dear Stephane,
>                 sorry for the late reply.
>
>                 /*This is my u-boot env:*/
>                 ---------------------
>                 fdt_addr=0x18000000
>                 fdt_file=imx6solo-icmm1010.dtb
>                 fileaddr=12c00000
>                 image_name=zImage
>                 initrd_addr=0x12c00000
>                 initrd_file=initramfs-icmm.ext4.gz
>                 ipaddr=192.168.10.2
>                 loadaddr=0x12000000
>
>                 netboot_bootargs=setenv bootargs console=ttymxc2,115200
>                 root=/dev/ram0 rootwait=0 rw
>
>                 netboot_loadbootscript=echo Booting from netboot ...; run
>                 netboot_bootargs; tftp ${loadaddr} ${image_name};tftp
>                 ${fdt_addr} ${fdt_file};tftp ${initrd_addr} ${initrd_file};bootz
>                 ${loadaddr} ${initrd_addr} ${fdt_addr};
>                 --------------------------------
>
>                 */The U-boot log:/*
>                 -----------------------------------
>                 => boot
>                 Booting from netboot ...
>                 Using FEC device
>                 TFTP from server 192.168.10.1; our IP address is 192.168.10.2
>                 Filename 'zImage'.
>                 Load address: 0x12000000
>                 Loading:
>                 #################################################################
>                 #################################################################
>                 #################################################################
>                 #################################################################
>                 #################################################################
>                 #################################################################
>                 #################################################################
>                 ######################################################
>                 4.1 MiB/s
>                 done
>                 Bytes transferred = 7468976 (71f7b0 hex)
>                 Using FEC device
>                 TFTP from server 192.168.10.1; our IP address is 192.168.10.2
>                 Filename 'imx6solo-icmm1010.dtb'.
>                 Load address: 0x18000000
>                 Loading: ###
>                 3.2 MiB/s
>                 done
>                 Bytes transferred = 37028 (90a4 hex)
>                 Using FEC device
>                 TFTP from server 192.168.10.1; our IP address is 192.168.10.2
>                 Filename 'initramfs-icmm.ext4.gz'.
>                 Load address: 0x12c00000
>                 Loading:
>                 #################################################################
>                 ###################################################
>                 4.2 MiB/s
>                 done
>                 Bytes transferred = 1702876 (19fbdc hex)
>                 Kernel image @ 0x12000000 [ 0x000000 - 0x71f7b0 ]
>                 Wrong Ramdisk Image Format
>                 Ramdisk image is corrupt or invalid
>                 ------------------------------------------------------------------------------------------------------
>
>                 all 3 files (kernel, ramdisk, dt) came from agl successful
>                 build. and my board
>                 boots normal with sd card.
>                 but with ramdisk... it fails.
>                 (also nbd runs with no error.)
>
>                 Where did I make mistake?
>
>                 appreciate your help.
>                 Vahid
>
>
>
>
>
>                 On Tue, Oct 1, 2019 at 9:17 AM Jan-Simon Moeller
>                 <jsmoeller@xxxxxxxxxxxxxxxxxxx
>                 <mailto:jsmoeller@xxxxxxxxxxxxxxxxxxx>> wrote:
>
>                     Paste the full boot log after bootz, pls.
>                     wrong image means usually kernel 'image' format invalid.
>
>                     Usual issues:
>                     Used correct addresses ??
>
>                     bootz 0x00kernel 0x00initrd 0x00fdt
>
>                     Can you exlude any overlap of the ranges due to size?
>
>
>                     Best,
>                     js
>
>
>
>
>                     Vahid Gharaee <vgharaee@xxxxxxxxx
>                     <mailto:vgharaee@xxxxxxxxx>> schrieb am Di., 1. Okt. 2019,
>                     06:49:
>
>                         sorry for typo:
>                         [..*bootz* says ..]
>                         [..app with *qt* or ...]
>
>                         On Tue, Oct 1, 2019 at 6:10 AM Vahid Gharaee
>                         <vgharaee@xxxxxxxxx <mailto:vgharaee@xxxxxxxxx>> wrote:
>
>                             Thanks for the info.
>                             I have tftp and tried netboot.
>                             After uploading kernel, dt and init-ram-disk into
>                             ram in boot stage boots says "wrong image". I
>                             thought maybe init ram disk image is not ready for
>                             imx yet. Now that you're saying it should work, I'll
>                             investigate further to find problem.
>
>                             Another question is: when NBD push the ext4 rootfs,
>                             supposed everything working, how can we debug? I
>                             mean with the nfs we actually had a folder on our
>                             host and we could install and debug app with at
>                             or.... , But now what could we do with just a *.ext4
>                             that NBD is serving to Target ?
>
>                             Thank you for your support
>                             Vahid
>
>                             On Tue, Oct 1, 2019, 00:26 Stephane Desneux
>                             <stephane.desneux@xxxxxxx> wrote:
>
>                                 Exactly.
>
>                                 And even if there's no TFTP (or no ethernet
>                                 driver in uboot), it's still
>                                 possible to cheat: for example, you can build
>                                 with agl-netboot feature and
>                                 you'll get a kernel image, a dtb, an initramfs
>                                 and a rootfs. You can put
>                                 everything but the rootfs on a sdcard and boot
>                                 from it 'as usual' then let the
>                                 initramfs mount the rootfs through NBD: set the
>                                 nbd.xxx variables accordingly in
>                                 the kernel command line and don't forget to
>                                 serve your rootfs ext4 file with a
>                                 nbd server).
>
>                                 Please keep us posted on your progress: I'm sure
>                                 many people would be interested
>                                 with this netboot feature :)
>
>                                 HTH
>                                 ---
>                                 Stephane Desneux - CTO - IoT.bzh
>                                 stephane.desneux@xxxxxxx - www.iot.bzh
>
>                                 On 30/09/2019 22:20, Jan-Simon Moeller wrote:
>                                 > Does the u-boot of your board do tftp? If yes,
>                                 the answer is yes.
>                                 >
>                                 > best,
>                                 > js
>                                 >
>                                 > Vahid Gharaee <vgharaee@xxxxxxxxx
>                                 <mailto:vgharaee@xxxxxxxxx>
>                                 <mailto:vgharaee@xxxxxxxxx
>                                 <mailto:vgharaee@xxxxxxxxx>>> schrieb am Mo.,
>                                 > 30. Sep. 2019, 11:21:
>                                 >
>                                 >     Dear Stephane, Community,
>                                 >
>                                 >     does the netboot works for imx6 ?
>                                 >
>                                 >     kind regards
>                                 >     vahid
>                                 >
>                                 >
>                                 >     On Sat, Aug 3, 2019 at 12:32 PM Vahid
>                                 Gharaee <vgharaee@xxxxxxxxx
>                                 <mailto:vgharaee@xxxxxxxxx>
>                                 >     <mailto:vgharaee@xxxxxxxxx
>                                 <mailto:vgharaee@xxxxxxxxx>>> wrote:
>                                 >
>                                 >
>                                 >         Thank you very much for your prompt reply.
>                                 >         i'm gonna try netboot
>                                 >         vahid
>                                 >
>                                 >         On Sat, Aug 3, 2019 at 12:18 PM
>                                 Stephane Desneux
>                                 >         <stephane.desneux@xxxxxxx> wrote:
>                                 >
>                                 >             Hi Vahid,
>                                 >
>                                 >             No, you can't use NFS because
>                                 extended attributes are not handled at
>                                 >             filesystem
>                                 >             level but they are required to use
>                                 SMACK.
>                                 >
>                                 >             That's why we added support for
>                                 netboot through NBD (network block
>                                 >             device at
>                                 >             kernel level, filesystem agnostic).
>                                 >
>                                 >             The goal is to have the rootfs
>                                 mounted on a device named /dev/nbd0.
>                                 >             On server
>                                 >             side, a daemon is responsible for
>                                 pushing the required blocks.
>                                 >
>                                 >             The chain is something like this:
>                                 >
>                                 >                 [--------- target ----------] 
>                                                       [-----
>                                 >             server ------]
>                                 >
>                                 >             ext4 FS -> block device (nbd0) ->
>                                 nbd-client --(tcp)--> nbd-server
>                                 >             -> ext4 image
>                                 >
>                                 >
>                                 >             To enable the NBD feature, you'll
>                                 have to add the 'agl-netboot'
>                                 >             feature at
>                                 >             aglsetup time.
>                                 >
>                                 >             Check this README:
>                                 >
>                                 >           
>                                  https://git.automotivelinux.org/AGL/meta-agl/tree/meta-netboot/README
>                                 >
>                                 >             You can also find an example
>                                 script to be run on server side (it's
>                                 >             suited for
>                                 >             the AGL docker container which is
>                                 a debian):
>                                 >
>                                 >           
>                                  https://git.automotivelinux.org/AGL/docker-worker-generator/tree/INSTALL/tools/scripts_netboot/run_xnbd_server
>                                 >
>                                 >             Hope this helps.
>                                 >             ---
>                                 >             Stephane Desneux - CTO - IoT.bzh
>                                 >             stephane.desneux@xxxxxxx - www.iot.bzh
>                                 >
>                                 >             On 03/08/2019 09:16, Vahid Gharaee
>                                 wrote:
>                                 >             > Dear all,
>                                 >             >
>                                 >             > Could anyone tell me if it's
>                                 possible to run AGL over NFS?
>                                 >             > I mean, because SMACK is used, I
>                                 could not use NFS.
>                                 >             > I need NFS to debug more easily.
>                                 >             >
>                                 >             > Thank you in advanced.
>                                 >             > Vahid 
>                                 >             >
>                                 >             >
>                                 _______________________________________________
>                                 >             > automotive-discussions mailing list
>                                 >             >
>                                 automotive-discussions@xxxxxxxxxxxxxxxxxxxxxxxxx
>                                 <mailto:automotive-discussions@xxxxxxxxxxxxxxxxxxxxxxxxx>
>                                 >           
>                                  <mailto:automotive-discussions@xxxxxxxxxxxxxxxxxxxxxxxxx
>                                 <mailto:automotive-discussions@xxxxxxxxxxxxxxxxxxxxxxxxx>>
>                                 >             >
>                                 >           
>                                  https://lists.linuxfoundation.org/mailman/listinfo/automotive-discussions
>                                 >             >
>                                 >
>                                 >   
>                                  _______________________________________________
>                                 >     automotive-discussions mailing list
>                                 >   
>                                  automotive-discussions@xxxxxxxxxxxxxxxxxxxxxxxxx <mailto:automotive-discussions@xxxxxxxxxxxxxxxxxxxxxxxxx>
>                                 >   
>                                  <mailto:automotive-discussions@xxxxxxxxxxxxxxxxxxxxxxxxx
>                                 <mailto:automotive-discussions@xxxxxxxxxxxxxxxxxxxxxxxxx>>
>                                 >   
>                                  https://lists.linuxfoundation.org/mailman/listinfo/automotive-discussions
>                                 >
>
>                         _______________________________________________
>                         automotive-discussions mailing list
>                         automotive-discussions@xxxxxxxxxxxxxxxxxxxxxxxxx
>                         <mailto:automotive-discussions@xxxxxxxxxxxxxxxxxxxxxxxxx>
>                         https://lists.linuxfoundation.org/mailman/listinfo/automotive-discussions
>
_______________________________________________
automotive-discussions mailing list
automotive-discussions@xxxxxxxxxxxxxxxxxxxxxxxxx
https://lists.linuxfoundation.org/mailman/listinfo/automotive-discussions

[Index of Archives]     [LARTC]     [Bugtraq]     [Yosemite Forum]     [Photo]

  Powered by Linux