On Feb 20, 2008 8:34 AM, Erez Zilber <erezz@xxxxxxxxxxxx> wrote: > Bart Van Assche wrote: > > Or: data sent during the first burst is not transferred via one-sided > > remote memory reads or writes but via two-sided send/receive > > operations. At least on my setup, these operations are as fast as > > one-sided remote memory reads or writes. As an example, I obtained the > > following numbers on my setup (SDR 4x network); > > ib_write_bw: 933 MB/s. > > ib_read_bw: 905 MB/s. > > ib_send_bw: 931 MB/s. > > According to these numbers one can think that you don't need RDMA at > all, just send iSCSI PDUs over IB. Sorry, but you are misinterpreting what I wrote. > The benchmarks that you use are > synthetic IB benchmarks that are not equivalent to iSCSI over iSER. They > just send IB packets. I'm not surprised that you got more or less the > same performance because, AFAIK, ib_send_bw doesn't copy data (unlike > iSCSI that has to copy data that is sent/received without RDMA). I agree that ib_write_bw / ib_read_bw / ib_send_bw performance results are not equivalent to iSCSI over iSER. The reason that I included these performance results was to illustrate that two-sided data transfers over IB are about as fast as one-sided data transfers. > When you use RDMA with iSCSI (i.e. iSER), you don't need to create iSCSI > PDUs and process them. The CPU is not busy as it is with iSCSI over TCP > because no data copies are required. Another advantage is that you don't > need header/data digest because the IB HW does that. As far as I know, when using iSER, the FirstBurstLength bytes of data are sent via two-sided data transfers, and there is no CPU intervention required to transfer the data itself over the IB network. Bart Van Assche. - To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
