Does it mean that RoCE always requires the network interface to have an IP address as it cannot use the GID to establish connections? On Tue, Mar 8, 2022 at 11:49 AM Mark Zhang <markzhang@xxxxxxxxxx> wrote: > > On 3/7/2022 9:57 PM, Sylvain Didelot wrote: > > External email: Use caution opening links or attachments > > > > > > Hi, > > > > I have configured one of my Mellanox network adapters for RoCE: > > --- > > CA 'roceP1p1s0f1' > > CA type: MT4123 > > Number of ports: 1 > > Firmware version: 20.32.1010 > > Hardware version: 0 > > Node GUID: 0xb8cef603002d1707 > > System image GUID: 0xb8cef603002d1706 > > Port 1: > > State: Active > > Physical state: LinkUp > > Rate: 100 > > Base lid: 0 > > LMC: 0 > > SM lid: 0 > > Capability mask: 0x00010000 > > Port GUID: 0xbacef6fffe2d1707 > > Link layer: Ethernet > > --- > > > > The Infiniband stack was installed from the official Ubuntu repository > > (20.04.4 LTS): > > --- > > $ apt search ibverbs > > Sorting... Done > > Full Text Search... Done > > ibverbs-providers/focal,now 28.0-1ubuntu1 arm64 [installed] > > User space provider drivers for libibverbs > > > > ibverbs-utils/focal,now 28.0-1ubuntu1 arm64 [installed] > > Examples for the libibverbs library > > > > libibverbs-dev/focal,now 28.0-1ubuntu1 arm64 [installed] > > Development files for the libibverbs library > > > > libibverbs1/focal,now 28.0-1ubuntu1 arm64 [installed] > > Library for direct userspace use of RDMA (InfiniBand/iWARP) > > > > librdmacm-dev/focal,now 28.0-1ubuntu1 arm64 [installed] > > Development files for the librdmacm library > > > > librdmacm1/focal,now 28.0-1ubuntu1 arm64 [installed] > > Library for managing RDMA connections > > > > rdmacm-utils/focal,now 28.0-1ubuntu1 arm64 [installed] > > Examples for the librdmacm library > > --- > > > > When I start the ucmatose server with the address format "gid", the > > tool fails binding with the error "No such device" > > > > Here is an example of the output: > > --- > > $ cat /sys/class/infiniband/roceP1p1s0f1/ports/1/gids/0 > > fe80:0000:0000:0000:bace:f6ff:fe2d:1707 > > > > $ ucmatose -b fe80:0000:0000:0000:bace:f6ff:fe2d:1707 -P ib -f gid > > cmatose: starting server > > cmatose: bind address failed: No such device > > test complete > > return status -1 > > --- > > > > Does rdmacm support connection establishment using GID with RoCE? Or > > Is it a known limitation for RoCE? > > FYI, the same experiment without RoCE (Link layer: Infiniband) works perfectly. > > > > Thanks for your help and your feedback. > > > > Sylvain Didelot > > I think ucmatose doesn't support RoCE when using "-f gid", as in this > case ai_family is set to AF_IB.
