Hi Lijun, > From: linux-rdma-owner@xxxxxxxxxxxxxxx <linux-rdma- > owner@xxxxxxxxxxxxxxx> On Behalf Of oulijun > Sent: Wednesday, November 13, 2019 6:36 AM > 在 2019/11/2 0:00, Parav Pandit 写道: > > Hi Lijun, > > > >> -----Original Message----- > >> From: Jason Gunthorpe <jgg@xxxxxxxx> > >> Sent: Friday, November 1, 2019 8:06 AM > >> To: oulijun <oulijun@xxxxxxxxxx>; Parav Pandit <parav@xxxxxxxxxxxx> > >> Cc: Doug Ledford <dledford@xxxxxxxxxx>; linux-rdma <linux- > >> rdma@xxxxxxxxxxxxxxx> > >> Subject: Re: 【Ask for help】 A question for __ib_cache_gid_add() > >> > >> On Fri, Nov 01, 2019 at 05:36:36PM +0800, oulijun wrote: > >>> Hi > >>> I am using the ubuntu system(5.0.0 kernel) to test the hip08 NIC > >>> port,. When I modify the perr mac1 to mac2,then restore to mac1, it > >>> will > >> cause the gid0 and gid 1 of the roce to be unavailable, and check > >> that the > >> /sys/class/infiniband/hns_0/ports/1/gid_attrs/ndevs/0 is show invalid. > >>> the protocol stack print will appear. > >>> > >>> Oct 16 17:59:36 ubuntu kernel: [200635.496317] __ib_cache_gid_add: > >>> unable to add gid fe80:0000:0000:0000:4600:4dff:fea7:9599 error=-28 > >>> Oct 16 17:59:37 ubuntu kernel: [200636.705848] 8021q: adding VLAN 0 > >>> to HW filter on device enp189s0f0 Oct 16 17:59:37 ubuntu kernel: > >>> [200636.705854] __ib_cache_gid_add: unable to add gid > >>> fe80:0000:0000:0000:4600:4dff:fea7:9599 error=-28 Oct 16 17:59:39 > >>> ubuntu kernel: [200638.755828] hns3 0000:bd:00.0 enp189s0f0: link up > >>> Oct 16 17:59:39 ubuntu kernel: [200638.755847] IPv6: > >>> ADDRCONF(NETDEV_CHANGE): enp189s0f0: link becomes ready Oct 16 > >>> 18:00:56 ubuntu kernel: [200715.699961] hns3 0000:bd:00.0 enp189s0f0: > >>> link down Oct 16 18:00:56 ubuntu kernel: [200716.016142] > >>> __ib_cache_gid_add: unable to add gid > >>> fe80:0000:0000:0000:4600:4dff:fea7:95f4 error=-28 Oct 16 18:00:58 > >>> ubuntu kernel: [200717.229857] 8021q: adding VLAN 0 to HW filter on > >>> device enp189s0f0 Oct 16 18:00:58 ubuntu kernel: [200717.229863] > >>> __ib_cache_gid_add: unable to add gid > >>> fe80:0000:0000:0000:4600:4dff:fea7:95f4 error=-28 > >>> > >>> Has anyone else encounterd a similar problem ? I wonder if the > >> _ib_cache_add_gid() is defective in 5.0 kernel? > >> > >> Maybe Parav knows? > > I used the kernel from [1], which seems to be fine; it has the required > commits [2], [3], [4]. > > > > Are you running RDMA traffic/applications which are using GID 0 and 1 when > changing MAC? > > If so, administrative operation such as MAC address change during active > RDMA traffic is unsupported, which can lead to this error. > > Can you please confirm? > Hi, parav Pandit > if running RDMA traffic/application which are using vlan gid when vconfig > rem the vlan, it will happen the following error? Yes, that is correct. Application is bound the rdma device and its associated vlan netdevice, holding reference to it while QPs are active. Once application releases the QPs, unregistration should progress. Supporting dynamic detachment of netdevices from the RoCE device was added using below 7 patches, those are not present in the Ubuntu 5.0.0. kernel that I looked at. 3bf3e2b881c1 RDMA/rxe: Consider skb reserve space based on netdev of GID 8f9748602491 IB/cm: Reduce dependency on gid attribute ndev check a70c07397fd8 RDMA: Introduce and use GID attr helper to read RoCE L2 fields adb4a57a7a1d RDMA/cma: Use rdma_read_gid_attr_ndev_rcu to access netdev dab2175800ef RDMA/rxe: Use rdma_read_gid_attr_ndev_rcu to access netdev 5102eca9039b net/smc: Use rdma_read_gid_l2_fields to L2 fields 943bd984b108 RDMA/core: Allow detaching gid attribute netdevice for RoCE
