With the latest perftest tool, I'm seeing something similar (failed status 4 which is the local protection fault) but with the ib_write_bw if I use the --use_cuda. server: ./ib_write_bw -d mlx5_0 -i 1 -F --report_gbits -R --use_cuda client: ./ib_write_bw -d mlx5_0 -i 1 -F --report_gbitgs 15.15.15.5 -R --use_cuda It works without the "--use_cuda". The ib_read_bw fails in the same manner. Any hints? >Sent: Wednesday, July 17, 2019 11:14 AM >To: linux-rdma@xxxxxxxxxxxxxxx >Subject: Does GPUDirect support IBV_WR_RDMA_WRITE_WITH_IMM? >Does GPUDirect support IBV_WR_RDMA_WRITE_WITH_IMM? >I can manage a host memory to gpu memory transfer, but not a gpu memory to gpu memory, nor a gpu memory to host memory transfer using it. >When I try it, I am getting a local protection fault to the WR. >The perftest appears to only use IBV_WR_RDMA_WRITE and IBV_WR_RDMA_READ. >It may be my use case is faulty and there is a better way, but is IBV_WR_RDMA_WRITE_WITH_IMM supported by GPUDirect? >Apologies if this isn’t the correct list. >Thanks >Andy CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient and may contain material that is proprietary, confidential, privileged or otherwise legally protected or restricted under applicable government laws. Any review, disclosure, distributing or other use without expressed permission of the sender is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies without reading, printing, or saving.
