I've recently uncovered a bug in RXE that causes iCRC errors when running between RXE and a correct RoCE implementation. The bug is that RXE is not including pad bytes in its iCRC calculations. So if the application payload is not 4B aligned then you'll hit this bug. You can see this by running ib_write_bw, for example, between mlnx_ib and rxe. works: ib_write_bw -s 32 -n 5 fails: ib_write_bw -s 33 -n 5 I'll post a patch this coming weekend hopefully. Steve. On Wed, Nov 27, 2019 at 5:10 AM Leon Romanovsky <leon@xxxxxxxxxx> wrote: > > On Tue, Nov 26, 2019 at 04:53:14PM -0800, Bart Van Assche wrote: > > On 11/21/19 11:19 PM, wangqi wrote: > > > Do you know how to make soft-roce (on server) can send message > > > to the hard-roce (like Mellanox cx4 card) on a client? We tried rdma-core > > > 25.0 and 26.0. The rdma-core can support both soft-roce and hard-roce. > > > > > > But it seems that the soft-roce (server) and hard-roce (client) can not > > > communicate via "ib_send_bw", "ib_read_bw" and so on, but can > > > communicate via "rping". > > > > > > Do you ever try to use soft-roce and hard-roce together? > > > Do they work well? I really wonder why they can not communicate with > > > each other. Best wishes, > > > > I think this should be possible. The diagram on the following web page shows > > a RoCE NIC and softROCE connected to each other: > > > > http://www.roceinitiative.org/software-based-roce-a-new-way-to-experience-rdma/ > > It should work, but it didn't work for me now :) > > Thanks > > > > > Bart.
