Hello RDMAers,
First of all, we would like to say thank you for everyone who made the RDMA
workshop and summit happened. This two day event was held at KS/LPC 2016 venue
and offsite, respectively [1].
It wasn’t happened without you, Thank you.
The below summary for the first day came from many attendees and doesn’t try to
cover everything discussed there. We invite you to share your notes with us and
mailing list.
The following list mostly consists from future plans and TODOs.
· Implementing a new Linux RDMA provider by Knut Omang
o Weak documentation for driver developers.
o Unclear development guides, no best practices.
o Need general RDMA test suite.
· New IOCTL ABI by Matan Barak
o Agreed on acceptance track. All core changes in bisectable way and ioctl
parts under EXPERIMENTAL Kconfig option and after that patch-per-command
transition.
o Extensive review was done and it will be addressed in the actual patches.
o This new code is valid candidate to introduce tracepoints into RDMA
subsystem.
· Consolidation of RDMA User-space code by Jason Gunthorpe
o Ongoing work on code quality/modernization, there is need to continue with
sparse annotations, coverity, MMIO accessor.
o There is need for proper systemd integration to autoload modules and proper
vendor upgrade sequence.
o 7 licenses is too much for 76K LOC code, there is clear benefit in
decreasing number of copyrights.
o Ensure that distribution packages are built from rdma-core. RedHat is
already doing it, there is need to continue to work with other distributions.
o Reuse kernel uAPI headers as a source base for libibverbs and providers.
· Debuggability and Tracing by Leon Romanovsky
o Extend standard tools as much as possible, like ss and ethtool.
o Use tracepoints for MAD snooping, new ABI work and memory registrations
flows.
o Reduce redundant printks.
· Integration with other subsystems by Liran Liss
o Challenges to integrate complex RDMA schemes (GPU-to-HCA-to-GPU) where CPU
is not involved. Multiple solutions were sent to mailing lists, but not
accepted yet.
o Best candidate seems to be DMA-BUFs, which don’t map peer BARs to the CPU.
o Associating SELinux security contexts with partition keys and subnet IDs is
an effective policy for isolating users in an IB fabric.
· Containers and RDMA by Liran Liss
o Supporting RoCE UD QPs is subtle because these QPs may only send and
receive from a subset of the GIDs – those that correspond to netdevs within the
process's namespace. Providers must enforce this implicitly, or fail UD QP
creation.
o Consider delivering RoCE CM packets through the network stack for better
enforcement.
o For IB, a process must be constrained to the partitions associated with the
netdev within the process's network namespace.
o RDMA cgroups will expose two types of objects only (instead of exposing
verbs objects): HCA objects and HCA handles, which will be presented in
absolute values and can be limited by the drivers. It will allow native support
of future vendor specific objects.
· New Fabrics by Ira Weiny
o It was open discussion about the scope of RDMA subsystem and the old
requirement to support verbs for every driver in driver/infiniband/, which
doesn’t hold for usnic and PSM.
· Future and Roadmap for RDMA for Doug Ledford
o The interoperability between IB vendors is less important now, since
Mellanox is the only one active vendor and Oracle will be in the future.
o RoCE is the main place for interoperability.
o Revealed UNH testing lab initiative to test and perform debug hackatons for
vendors, in similar way to NFS-RDMA.
o OFED software stack is rising, while upstream continues to shine :)
· User Mode Ethernet Verbs by Tzahi Oved
o Verbs objects suite well OS bypass Eth programming with generic object model
to expend to new I/O offloads. These APIs can support both Eth and IPoIB
offloads.
o Developers can use common API for OS bypass and share objects like memory
registration, CQs, SRQs for both RDMA and user mode Eth.
o New introduced RSS objects present generic approach of breaking QP into
transport and work queues allowing flexible future expansion.
o Object oriented approach taken with CQ can be taken with posting work queue.
o Future vendor specific capabilities can be implemented using vendor
specific new kernel ABI support and user mode vendor API query through
libibverbs.
· Dual Licensing by Susan Coulter
o OFA can’t and won’t enforce dual-license on kernel space.
o User space will continue to be under dual-license.
It was great event and we are looking forward to see you next year.
Christoph and Leon.
[1] http://marc.info/?l=linux-rdma&m=147767504329945&w=2
Attachment:
signature.asc
Description: PGP signature
