And drop bits that are outdated or replaced by top-level Documentation. Signed-off-by: Christoph Hellwig <hch@xxxxxx> --- Documentation/libibverbs.md | 58 ++++++++++++++++ libibverbs/README | 164 -------------------------------------------- 2 files changed, 58 insertions(+), 164 deletions(-) create mode 100644 Documentation/libibverbs.md delete mode 100644 libibverbs/README diff --git a/Documentation/libibverbs.md b/Documentation/libibverbs.md new file mode 100644 index 0000000..cbe076e --- /dev/null +++ b/Documentation/libibverbs.md @@ -0,0 +1,58 @@ +# Introduction + +libibverbs is a library that allows programs to use RDMA "verbs" for +direct access to RDMA (currently InfiniBand and iWARP) hardware from +userspace. For more information on RDMA verbs, see the InfiniBand +Architecture Specification vol. 1, especially chapter 11, and the RDMA +Consortium's RDMA Protocol Verbs Specification. + +# Using libibverbs + +### Device nodes + +The verbs library expects special character device files named +/dev/infiniband/uverbsN to be created. When you load the kernel +modules, including both the low-level driver for your IB hardware as +well as the ib_uverbs module, you should see one or more uverbsN +entries in /sys/class/infiniband_verbs in addition to the +/dev/infiniband/uverbsN character device files. + +To create the appropriate character device files automatically with +udev, a rule like + + KERNEL="uverbs*", NAME="infiniband/%k" + +can be used. This will create device nodes named + + /dev/infiniband/uverbs0 + +and so on. Since the RDMA userspace verbs should be safe for use by +non-privileged users, you may want to add an appropriate MODE or GROUP +to your udev rule. + +### Permissions + +To use IB verbs from userspace, a process must be able to access the +appropriate /dev/infiniband/uverbsN special device file. You can +check the permissions on this file with the command + + ls -l /dev/infiniband/uverbs* + +Make sure that the permissions on these files are such that the +user/group that your verbs program runs as can access the device file. + +To use IB verbs from userspace, a process must also have permission to +tell the kernel to lock sufficient memory for all of your registered +memory regions as well as the memory used internally by IB resources +such as queue pairs (QPs) and completion queues (CQs). To check your +resource limits, use the command + + ulimit -l + +(or "limit memorylocked" for csh-like shells). + +If you see a small number such as 32 (the units are KB) then you will +need to increase this limit. This is usually done for ordinary users +via the file /etc/security/limits.conf. More configuration may be +necessary if you are logging in via OpenSSH and your sshd is +configured to use privilege separation. diff --git a/libibverbs/README b/libibverbs/README deleted file mode 100644 index 848eb05..0000000 --- a/libibverbs/README +++ /dev/null @@ -1,164 +0,0 @@ -Introduction -============ - -libibverbs is a library that allows programs to use RDMA "verbs" for -direct access to RDMA (currently InfiniBand and iWARP) hardware from -userspace. For more information on RDMA verbs, see the InfiniBand -Architecture Specification vol. 1, especially chapter 11, and the RDMA -Consortium's RDMA Protocol Verbs Specification. - -Using libibverbs -================ - -Device nodes ------------- - -The verbs library expects special character device files named -/dev/infiniband/uverbsN to be created. When you load the kernel -modules, including both the low-level driver for your IB hardware as -well as the ib_uverbs module, you should see one or more uverbsN -entries in /sys/class/infiniband_verbs in addition to the -/dev/infiniband/uverbsN character device files. - -To create the appropriate character device files automatically with -udev, a rule like - - KERNEL="uverbs*", NAME="infiniband/%k" - -can be used. This will create device nodes named - - /dev/infiniband/uverbs0 - -and so on. Since the RDMA userspace verbs should be safe for use by -non-privileged users, you may want to add an appropriate MODE or GROUP -to your udev rule. - -Permissions ------------ - -To use IB verbs from userspace, a process must be able to access the -appropriate /dev/infiniband/uverbsN special device file. You can -check the permissions on this file with the command - - ls -l /dev/infiniband/uverbs* - -Make sure that the permissions on these files are such that the -user/group that your verbs program runs as can access the device file. - -To use IB verbs from userspace, a process must also have permission to -tell the kernel to lock sufficient memory for all of your registered -memory regions as well as the memory used internally by IB resources -such as queue pairs (QPs) and completion queues (CQs). To check your -resource limits, use the command - - ulimit -l - -(or "limit memorylocked" for csh-like shells). - -If you see a small number such as 32 (the units are KB) then you will -need to increase this limit. This is usually done for ordinary users -via the file /etc/security/limits.conf. More configuration may be -necessary if you are logging in via OpenSSH and your sshd is -configured to use privilege separation. - -Valgrind support ----------------- - -When running applications that use libibverbs under the Valgrind -memory-checking debugger, Valgrind will falsely report "read from -uninitialized" for memory that was initialized by the kernel drivers. -Specifically, Valgrind cannot see when kernel drivers write to -userspace memory, so when the process reads from that memory, Valgrind -incorrectly assumes that the memory contents are uninitialized, and -therefore raises a warning. - -libibverbs can be built with specific support for the Valgrind -memory-checking debugger by specifying the --with-valgrind command -line argument to configure. This flag enables code in libibverbs to -tell Valgrind "this memory may look uninitialized, but it's really -OK," which therefore suppresses the incorrect "read from -uninitialized" warnings. This code adds trivial overhead to the -critical performance path, so it is disabled by default. The intent -is that production users can use a "normal" build of libibverbs and -developers can use the "valgrind debug" build by simply switching -their LD_LIBRARY_PATH environment variables. - -Libibverbs needs some header files from Valgrind in order to compile -this support; it is important to use the header files from the same -version of Valgrind that will be used at run time. You may need to -specify the directory where Valgrind's header files are installed as -an argument to --with-valgrind. For example - - ./configure --with-valgrind=/opt/valgrind - -will make the libibverbs build look for valgrind headers in -/opt/valgrind/include - -Reporting bugs -============== - -Bugs should be reported to the OpenFabrics mailing list -<general@xxxxxxxxxxxxxxxxxxxxx>. In your bug report, please include: - - * Information about your system: - - Linux distribution and version - - Linux kernel and version - - InfiniBand/iWARP hardware and firmware version - - ... any other relevant information - - * How to reproduce the bug. Command line arguments for a libibverbs - example program or source code that other developers can - compile and run is most convenient. - - * If the bug is a crash, the exact output printed out when the crash - occurred, including any kernel messages produced. - - * If a verbs call is mysteriously returning an error or failing, the - output of "strace -ewrite -ewrite=all <command>". - -Submitting patches -================== - -Patches should also be submitted to the OpenFabrics mailing list -<general@xxxxxxxxxxxxxxxxxxxxx>. Please use unified diff form (the -u -option to GNU diff), and include a good description of what your patch -does and why it should be applied. If your patch fixes a bug, please -make sure to describe the bug and how your fix works. - -Please include a change to the ChangeLog file (in standard GNU -changelog format) as part of your patch. - -Make sure that your contribution can be licensed under the same -license as the original code you are patching, and that you have all -necessary permissions to release your work. - -TODO -==== - -1.1 series ----------- - -The libibverbs API and ABI are frozen for all releases in the 1.1 -series. Methods were added to struct ibv_context to implement the -following features, so it should be possible to add them in a future -release in the 1.1 series: - - * Memory window (MW) support. - - * Implement the reregister memory region (MR) verb. We will add an - extension to the IB spec to allow the application to indicate that - the region is only being extended, and that operations in progress - should _not_ fail (contrary to the IB spec, which states that - reregister must be implemented so that it behaves equivalently to a - deregister followed by a register). - -Other possibilities -------------------- - -There are no plans to implement the following features, which would be -needed for completeness but don't seem particularly useful. However, -if there is demand from application developers or an implementation is -contributed, then the feature may be added. - - * Implement the query address handle (AH) verb. - * Implement the query memory region (MR) verb. -- 2.1.4 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
