Am Freitag, den 28.07.2017, 10:56 -0600 schrieb Jason Gunthorpe: > On Fri, Jul 28, 2017 at 06:41:05PM +0200, Benjamin Drung wrote: > > Am Freitag, den 28.07.2017, 09:47 -0600 schrieb Jason Gunthorpe: > > > On Fri, Jul 28, 2017 at 05:42:05PM +0200, Benjamin Drung wrote: > > > > > > > $ journalctl | grep mlx > > > > mlx4_core: Mellanox ConnectX core driver v3.3-1.0.4 (03 Jul > > > > 2016) > > > > mlx4_core: Initializing 0000:02:00.0 > > > > mlx4_core: device is working in RoCE mode: Roce V1 > > > > > > So you are running purely in roce mode? > > > > I want to use mlx4_ib and IPoIB. The output is the same for a > > properly > > working system, but these system will load mlx4_ib and then more > > logs > > will appear. > > This approach relies on this patch in upstream: > > $ git describe --contains b046ffe54dc13ff8ae918c83bedb71aa7919d63b > v3.13-rc1~105^2~189^2 > > And it works fine on my 4.13 stock systems with mlx4. Yes, that commit is part of our 4.4 kernel. > Are you using an initrd? If you have mlx4_core in the initrd then you > must also have mlx4_ib and mlx4_en as well, otherwise the kernel > triggered autoload will fail to find the module in the initrd and it > will not be retried. Yes. That was the issue. I am using an initrd (from initramfs-tools). The mlx4_core module was included in the initrd, but mlx4_ib was missing. I tweaked initramfs-tools to include mlx4_ib and then it was autoloaded. Thanks. Remaining problem: The IPoIB InfiniBand partition does not come up correctly: $ ip a [...] 9: ib1.d00b@ib1: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 65520 qdisc mq state UP group default qlen 1024 [...] The corresponding config in /etc/network/interfaces: auto ib0.d00b iface ib0.d00b inet6 static address <redacted> netmask 64 pre-up echo connected > /sys/class/net/$IFACE/mode dad-attempts 300 I tried bring up the device manually: $ sudo ip link set dev ib0.d00b down $ sudo ifup ib0.d00b $ sudo ifdown ib0.d00b $ sudo ifup ib0.d00b But the result is still the same. It still has the M-DOWN state and pinging other host does not work. I haven't found useful information on the web regarding the M-DOWN state. -- Benjamin Drung System Developer Debian & Ubuntu Developer ProfitBricks GmbH Greifswalder Str. 207 D - 10405 Berlin Email: benjamin.drung@xxxxxxxxxxxxxxxx Web: https://www.profitbricks.com Sitz der Gesellschaft: Berlin. Registergericht: Amtsgericht Charlottenburg, HRB 125506B. Geschäftsführer: Achim Weiss. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
