Hi, Is it possible to transfer a large file, say 25GB, between two computers using RDMA UD, and have an exact copy of the original file on the receiving side? My understanding is that the order of the messages is not guaranteed with UD. But I thought that if I only use one QP I could ensure that the ordering of the data will be predictable. This is a proof of concept at this time. My thought was to load the file into memory on the client side and advance the address pointer after each transfer. The same scheme would be used on the receiving side. That is, allocate a large region of memory and increment the address pointer after each message is received. I have been using the program ib_send_bw from perftest as a starting point. After increasing the buffer size on the client side and advancing sg_list->addr on the client side, I seem to be sending what I intend to send. At the present time I am not yet advancing sg_list->addr on the server side. When I print out the first 100B of data after each transfer, I often see the data I expect, but not always. The commands I've been using to launch my modified version of ib_send_bw are client: ./ib_send_bw -R -d mlx5_0 -c UD -i 1 -F -q 1 --mmap=myInputfile.dat --report_gbits -n 1000 10.10.10.3 server: ./ib_send_bw -R -r 50 -d mlx5_0 -c UD -i 1 -q 1 -F -n 1000 --report_gbits The "-n 1000" is just for debugging. It will be increased later. Am I going about this in the correct way? Should I be able to copy a file using RDMA UD? Any thoughts or suggestions would be most appreciated. Thanks, Terry
