From: Jason Gunthorpe <jgg@xxxxxxxxxxxx>
The implementation of disassociation has had a long standing and serious
flaw, userspace cannot recover from it. Since all commands fail in the
kernel with EIO an application is unable to call ibv_destroy_XX, which
means the app cannot cleanup the memory. Ultimately this makes the feature
useless for a continually available system.
A simple solution of having userspace understand that EIO means the kernel
is gone unfortunately does not work for several of the objects (such as CQ
and QP) that return data necessary for userspace to complete destruction
in a multi-threaded environment.
Thus, this series completes the re-arrangement of the core code to permit
an uboject to exist on the IDR list without an associated HW object. When
this happens the destroy method can work as normal even after
disassociation.
Overall this approach makes the code generally more uniform and typical as
several of the locking flows are made more explicit. We no longer rely on
an implicit SRCU based check of ib_dev in the command caller for
correctness, everything is contained within some kind of lock.
This series does not remove the last roadblock for ioctl methods as the
ioctl methods can rely on data held inside the driver module that
registered the device. A followup series will be required to disconnect
the ioctl specs from the device data upon removal.
Jason Gunthorpe (11):
IB/uverbs: Remove rdma_explicit_destroy() from the ioctl methods
IB/uverbs: Make the write path destroy methods use the same flow as
ioctl
IB/uverbs: Consolidate uobject destruction
IB/uverbs: Convert 'bool exclusive' into an enum
IB/uverbs: Allow RDMA_REMOVE_DESTROY to work concurrently with
disassociate
IB/uverbs: Allow uobject allocation to work concurrently with
disassociate
IB/uverbs: Lower the test for ongoing disassociation
IB/uverbs: Do not pass struct ib_device to the write based methods
IB/uverbs: Do not pass struct ib_device to the ioctl methods
IB/uverbs: Do not block disassociate during write()
IB/uverbs: Allow all DESTROY commands to succeed after disassociate
drivers/infiniband/core/rdma_core.c | 516 +++++++++++-------
drivers/infiniband/core/rdma_core.h | 5 +
drivers/infiniband/core/uverbs.h | 5 +-
drivers/infiniband/core/uverbs_cmd.c | 232 ++++----
drivers/infiniband/core/uverbs_ioctl.c | 33 +-
drivers/infiniband/core/uverbs_main.c | 33 +-
drivers/infiniband/core/uverbs_std_types.c | 3 +-
.../core/uverbs_std_types_counters.c | 21 +-
drivers/infiniband/core/uverbs_std_types_cq.c | 39 +-
drivers/infiniband/core/uverbs_std_types_dm.c | 13 +-
.../core/uverbs_std_types_flow_action.c | 35 +-
drivers/infiniband/core/uverbs_std_types_mr.c | 22 +-
drivers/infiniband/hw/mlx5/devx.c | 43 +-
drivers/infiniband/hw/mlx5/flow.c | 8 +-
include/rdma/ib_verbs.h | 7 +-
include/rdma/uverbs_ioctl.h | 4 +-
include/rdma/uverbs_std_types.h | 39 +-
include/rdma/uverbs_types.h | 98 ++--
18 files changed, 622 insertions(+), 534 deletions(-)
--
2.18.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
