> On 3 Jun 2021, at 14:10, Mark Zhang <markzhang@xxxxxxxxxx> wrote:
>
> On 6/3/2021 2:50 PM, Anand Khoje wrote:
>> External email: Use caution opening links or attachments
>> ib_query_port() calls device->ops.query_port() to get the port
>> attributes. The method of querying is device driver specific.
>> The same function calls device->ops.query_gid() to get the GID and
>> extract the subnet_prefix (gid_prefix).
>> The GID and subnet_prefix are stored in a cache. But they do not get
>> read from the cache if the device is an Infiniband device. The
>> following change takes advantage of the cached subnet_prefix.
>> Testing with RDBMS has shown a significant improvement in performance
>> with this change.
>> The function ib_cache_is_initialised() is introduced because
>> ib_query_port() gets called early in the stage when the cache is not
>> built while reading port immutable property.
>> In that case, the default GID still gets read from HCA for IB link-
>> layer devices.
>> Fixes: fad61ad ("IB/core: Add subnet prefix to port info")
>> Signed-off-by: Anand Khoje <anand.a.khoje@xxxxxxxxxx>
>> Signed-off-by: Haakon Bugge <haakon.bugge@xxxxxxxxxx>
>> ---
>> drivers/infiniband/core/cache.c | 7 ++++++-
>> drivers/infiniband/core/device.c | 9 +++++++++
>> include/rdma/ib_cache.h | 6 ++++++
>> include/rdma/ib_verbs.h | 6 ++++++
>> 4 files changed, 27 insertions(+), 1 deletion(-)
>> diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c
>> index b6700ad..724ac0e 100644
>> --- a/drivers/infiniband/core/cache.c
>> +++ b/drivers/infiniband/core/cache.c
>> @@ -1624,6 +1624,8 @@ int ib_cache_setup_one(struct ib_device *device)
>> err = ib_cache_update(device, p, true);
>> if (err)
>> return err;
>> + set_bit(IB_PORT_CACHE_INITIALIZED,
>> + &device->port_data[p].flags);
>> }
>> return 0;
>> @@ -1639,8 +1641,11 @@ void ib_cache_release_one(struct ib_device *device)
>> * all the device's resources when the cache could no
>> * longer be accessed.
>> */
>> - rdma_for_each_port (device, p)
>> + rdma_for_each_port (device, p) {
>> + clear_bit(IB_PORT_CACHE_INITIALIZED,
>> + &device->port_data[p].flags);
>> kfree(device->port_data[p].cache.pkey);
>> + }
>> gid_table_release_one(device);
>> }
>
> Do we need to clear it in gid_table_cleanup_one()?
Good point. Is it feasible that ib_query_port() can be called on a device that has been removed? If yes, we need it in gid_table_cleanup_one() as well.
Thxs, Håkon
