I don't have any specific feedback on the issue you've observed, but I have a question about your config... On 18 September 2017 at 16:09, Jin Cai <caijin.laurence@xxxxxxxxx> wrote: > ms_async_rdma_roce_ver = 1 > ms_async_rdma_sl = 5 > ms_async_rdma_dscp = 136 Doesn't the DSCP field only apply to IP traffic and therefore require a "trust L3" QoS config, which would imply RoCE version 2? The setting is probably irrelevant if you have configured the network for L2 flow-control and Ceph is using the correct RoCE GID, but I was curious. -- Cheers, ~Blairo -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html