On Wed, Sep 23, 2020 at 07:50:11PM +0300, Leon Romanovsky wrote: > When an application is not using RDMA CM and if it is using multiple RDMA > devices with one or more RoCE ports, finding the right GID table entry is > a long process. > > For example, with two RoCE dual-port devices in a system, when IP > failover is used between two RoCE ports, searching a suitable GID > entry for a given source IP, matching netdevice of given RoCEv1/v2 type > requires iterating over all 4 ports * 256 entry GID table. > > Even though the best first match GID table for given criteria is used, > when the matching entry is on the 4th port, it requires reading > 3 ports * 256 entries * 3 files (GID, netdev, type) = 2304 files. > > The GID table needs to be referred on every QP creation during IP > failover on other netdevice of an RDMA device. > > In an alternative approach, a GID cache may be maintained and updated on > GID change event was reported by the kernel. However, it comes with below > two limitations: > (a) Maintain a thread per application process instance to listen and update > the cache. > (b) Without the thread, on cache miss event, query the GID table. Even in > this approach, if multiple processes are used, a GID cache needs to be > maintained on a per-process basis. With a large number of processes, > this method doesn't scale. > > Hence, we introduce this series of patches, which introduces an API to > query the complete GID tables of an RDMA device, that returns all valid > GID table entries. > > This is done through single ioctl, eliminating 2304 read, 2304 open and > 2304 close system calls to just a total of 2 calls (one for each device). > > While at it, we also introduce an API to query an individual GID entry > over ioctl interface, which provides all GID attributes information. > > Thanks > > Avihai Horon (4): > RDMA/core: Change rdma_get_gid_attr returned error code > RDMA/core: Modify enum ib_gid_type and enum rdma_network_type > RDMA/core: Introduce new GID table query API > RDMA/uverbs: Expose the new GID query API to user space I made the edit to fix the locking, please check it Applied to for-next Thanks, Jason
