On Thu, Aug 05, 2021 at 10:38:42AM -0400, Olga Kornievskaia wrote: > Hi folks, > > Can somebody help me understand how RoCE (this is probably RDMA core > and not specific to RoCE but I'm not sure) manages destination MAC > addresses for its connection? > > Specifically the problem being observed is a server initiates an RDMA > CM disconnect (client replies), client tries to reconnect. Server > sends an ARP advertising a different MAC for the IP that the RDMA > connection was using. RDMA code keeps sending the RDMA CM connect > message to the old MAC for a certains period of time (90-100sec) then > it finally sends it to the new MAC address. > > Question: how does the core RDMA layer manage the MAC address for the > connection. Why does it seem like it ignores the ARP updates? RDMA objects acquire a MAC adress when they are created and do not synchronize with the neighbor cache after. What you are seeing is that the CM_ID object holds the bad mac until it is destoroyed and likely a new CM_ID object gets created that holds the updated MAC Jason
