On 2/2/2017 1:07 AM, feu tche wrote:
hi Moni, guys,
Sure yes I could ping, yes, no firewall or anything, in all cases that
was clean minimal install of Ubuntu 16.04.1 with updates and 4.9.6
kernel.
In my previous tests that was all on vmware vms on virtual network,
but as you asked I also decided to check on real hardware.
I've got same results (below) between two machines with dual Intel
I350 and I also checked same both directions against a machine with
dual Mellanox ConnectX-3 Pro.
=====================================================================================
Client:
root@rdma-client:~/rdma-core-master# rxe_cfg status
Argument "" isn't numeric in numeric eq (==) at
/root/rdma-core-master/providers/rxe/rxe_cfg line 140.
Name Link Driver Speed NMTU IPv4_addr RDEV RMTU
bond0 yes bonding 1500 147.75.100.109 rxe0 1024 (3)
p1p1 yes igb 1500
p1p2 yes igb 1500
root@rdma-client:~/rdma-core-master# ibv_devinfo -d rxe0
hca_id: rxe0
transport: InfiniBand (0)
fw_ver: 0.0.0
node_guid: 0ec4:7aff:fe80:5d8e
sys_image_guid: 0000:0000:0000:0000
vendor_id: 0x0000
vendor_part_id: 0
hw_ver: 0x0
phys_port_cnt: 1
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 1024 (3)
sm_lid: 0
port_lid: 0
port_lmc: 0x00
link_layer: Ethernet
root@rdma-client:~/rdma-core-master# ping 147.75.100.177
PING 147.75.100.177 (147.75.100.177) 56(84) bytes of data.
64 bytes from 147.75.100.177: icmp_seq=1 ttl=63 time=0.451 ms
64 bytes from 147.75.100.177: icmp_seq=2 ttl=63 time=0.548 ms
64 bytes from 147.75.100.177: icmp_seq=3 ttl=63 time=0.548 ms
64 bytes from 147.75.100.177: icmp_seq=4 ttl=63 time=0.552 ms
root@rdma-client:~/rdma-core-master# ibv_rc_pingpong -d rxe0 -g 1 -i 1
-n 1 147.75.100.177
local address: LID 0x0000, QPN 0x000011, PSN 0x4af006, GID
::ffff:147.75.100.109
remote address: LID 0x0000, QPN 0x000011, PSN 0xf304fb, GID
::ffff:147.75.100.177
^C
root@rdma-client:~/rdma-core-master# ibv_rc_pingpong -d rxe0 -g 0 -i 1
-n 1 147.75.100.177
local address: LID 0x0000, QPN 0x000011, PSN 0xe483d9, GID
fe80::ec4:7aff:fe80:5d8e
remote address: LID 0x0000, QPN 0x000011, PSN 0xda937e, GID
fe80::ec4:7aff:fe80:5c2e
^C
====================================================================================
Server:
H/W path Device Class Description
======================================================
system SYS-5039MS-H12TRF (To be
filled by O.E.M.)
/0 bus X11SSE-F
/0/100 bridge Sky Lake Host Bridge/DRAM
Registers
/0/100/1 bridge Sky Lake PCIe Controller
(x16)
/0/100/1/0 p1p1 network I350 Gigabit Network
Connection
/0/100/1/0.1 p1p2 network I350 Gigabit Network Connection
root@rdma-server:~/rdma-core-master# rxe_cfg status
Argument "" isn't numeric in numeric eq (==) at
/root/rdma-core-master/providers/rxe/rxe_cfg line 140.
Name Link Driver Speed NMTU IPv4_addr RDEV RMTU
bond0 yes bonding 1500 147.75.100.177 rxe0 1024 (3)
p1p1 yes igb 1500
p1p2 yes igb 1500
we never tried coupling rxe with a bond/bridge
please retry with a real device. like the p1p1 you have there.
thanks.
root@rdma-server:~/rdma-core-master# ibv_devinfo -d rxe0
hca_id: rxe0
transport: InfiniBand (0)
fw_ver: 0.0.0
node_guid: 0ec4:7aff:fe80:5c2e
sys_image_guid: 0000:0000:0000:0000
vendor_id: 0x0000
vendor_part_id: 0
hw_ver: 0x0
phys_port_cnt: 1
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 1024 (3)
sm_lid: 0
port_lid: 0
port_lmc: 0x00
link_layer: Ethernet
root@rdma-server:~/rdma-core-master# ibv_rc_pingpong -d rxe0 -g 1 -i 1
local address: LID 0x0000, QPN 0x000011, PSN 0xf304fb, GID
::ffff:147.75.100.177
remote address: LID 0x0000, QPN 0x000011, PSN 0x4af006, GID
::ffff:147.75.100.109
Completion for unknown wr_id 0
parse WC failed 2
root@rdma-server:~/rdma-core-master# ibv_rc_pingpong -d rxe0 -g 0 -i 1
local address: LID 0x0000, QPN 0x000011, PSN 0xddb87c, GID
fe80::ec4:7aff:fe80:5c2e
remote address: LID 0x0000, QPN 0x000011, PSN 0x415a75, GID
fe80::ec4:7aff:fe80:5d8e
^C
Trace is attached.
Thanks!
Fedor
On 1 Feb 2017 09:35, "Moni Shoua" <monis@xxxxxxxxxxxx> wrote:
Hi
1. Can you ping from 192.168.237.132 to 192.168.237.133?
2. Which Ethernet NICs are you using (vendor)?
3. Can you please run the test with tracing on the server side (the
side you sent the log for) and attach the results
(/sys/kernel/debug/tracing/trace)
thanks
I forgot to mention that in the trace filter I'd like to see functions
from ib_core, rdma_cm, ib_cm, ib_rxe
Also, please check if you have a firewall (iptables) that block UDP/4791
thanks
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
