From: Parav Pandit <parav@xxxxxxxxxxxx> Describe ib_core_device, ib_device association and their existence in net namespaces for backward compatibility. Signed-off-by: Parav Pandit <parav@xxxxxxxxxxxx> Signed-off-by: Leon Romanovsky <leonro@xxxxxxxxxxxx> --- Documentation/infiniband/core_devices.txt | 101 ++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 Documentation/infiniband/core_devices.txt diff --git a/Documentation/infiniband/core_devices.txt b/Documentation/infiniband/core_devices.txt new file mode 100644 index 000000000000..ff28def49ff4 --- /dev/null +++ b/Documentation/infiniband/core_devices.txt @@ -0,0 +1,101 @@ +Linux RDMA devices and their sysfs entries +------------------------------------------ + +1. Background +-------------- +RDMA networking devices have at least 3 link or transport layers. +(a) InfiniBand +(b) RoCE +(c) iWarp + +These networking devices provide kernel bypass for sending/receiving +data to/from the network. + +There are various modes in which these devices are used along with +other protocols for connection establishment and/or for data transfer. +Such as, +(a) rdmacm for connection establishement and verbs for data transfer. +(b) tcp/ip for connection establishment and verbs for data transfer. + +Additionally rdma devices can be shared among multiple net namespaces. + +It is also desired to have per net namespace rdma devices as the +stack matures. + +sysfs entries are heavily used for device discovery, statistics and network +addresses in rdma stack. + +Therefore, to have minimal impact on backward compatibility for these 3 +transports and to provide forward looking method, the following sysfs +isolation approach is taken. + +2. Design +---------- + +For every rdma ib_device, core code creates an ib_core_device in every +net namespace to give the appearance that the rdma device is present +in all net namespaces. +Each ib_core_device owns the sysfs entries in their net namespace. + +All ib_core_device(s) points to one owner ib_device using driver_data. + +2.1 Shared rdma ib_device view in different net namespaces +----------------------------------------------------------- + + ib_core_device (net_ns_1) + +--------------+ + | | + | device | + | +----------+ | + | | | | + | | | | + | | | | + | +----------+ | (init_net) + | | ib_device + | *owner-------------------------+------>+--------------------+<--+ + +--------------+ | | | | + | | ib_core_device | | + | | +--------------+ | | + | | | | | | + | | | device | | | + | | | +----------+ | | | + ib_core_device (net_ns_2) | | | | | | | | + +--------------+ | | | | | | | | + | | | | | | | | | | + | device | | | | +----------+ | | | + | +----------+ | | | | | | | + | | | | | | | *owner--------------+ + | | | | | | +--------------+ | + | | | | | +--------------------+ + | +----------+ | | + | | | + | *owner------------------------+ + +--------------+ + +2.2 rdma ib_device bound to a net namespace (in future) +-------------------------------------------------------- + +In this mode, when an rdma device is bound to a net namespace, all compat +sysfs entries will be terminated. sysfs entries will reside in single +net namespace which device is bound to. +Thereby having one-to-one mapping and providing isolation of devices +to their owning net namespace. + +(net_ns_1) +ib_device ++--------------------+ +| | +| | +| ib_core_device | +| +--------------+ | +| | | | +| | device | | +| | +----------+ | | +| | | | | | +| | | | | | +| | | | | | +| | +----------+ | | +| | | | +| | *owner | | +| +--------------+ | ++--------------------+ -- 2.19.1
