> From what I see srp_sq_size is controlled via > configfs. Can you set it to 2048 just for the sake of > confirmation this is indeed the issue? Now that we have confirmed that this works, what is the proper way to fix the problem? Is the core issue with mlx5 or srp? Thanks, Mark On Tue, Aug 19, 2014 at 11:01 AM, Mark Lehrer <lehrer@xxxxxxxxx> wrote: >> From what I see srp_sq_size is controlled via >> configfs. Can you set it to 2048 just for the sake of >> confirmation this is indeed the issue? > > Yes! This setting allowed the two machines to establish an SRP session. > > I'll try some I/O tests to see how well it works. > > Thanks, > Mark > > > On Tue, Aug 19, 2014 at 8:50 AM, Sagi Grimberg <sagig@xxxxxxxxxxxxxxxxxx> wrote: >> On 8/19/2014 2:20 AM, Mark Lehrer wrote: >>> >>> I have a client machine that is trying to establish an SRP connection, >>> and it is failing due to an ENOMEM memory allocation error. I traced >>> it down to the max_qp_sz field -- the mlx5 limits this to 16384 but >>> the request wants 32768. >>> >>> I spent some time trying to figure out how this limit is set, but it >>> isn't quite obvious. Is there a driver parameter I can set, or a hard >>> coded limit somewhere? >>> >>> I'm using Ubuntu 14.04 and targetcli on the target side, Windows >>> 2008r2 and WinOFED on the client side. >>> >> >> Hi Mark, >> >> I think the issue here is that the SRP target asks for srp_sq_size (default >> 4096) to allocate room for send WRs, but it also asks for >> SRPT_DEF_SG_PER_WQE(16) to allocate room for max_send_sge which >> generally makes the work queue entries bigger as they come inline. That >> probably exceeds the mlx5 max send queue size supported... >> >> It is strange that mlx5 driver is not able to fit same send queue >> lengths as mlx4... Probably work queue entries are slightly bigger (I >> can check that - and I will). >> >> I just wander how can the ULP requesting the space reservations in the >> send queue know that, and if it should know that at all... >> >> From what I see srp_sq_size is controlled via configfs. Can you set >> it to 2048 just for the sake of confirmation this is indeed the issue? >> >> Thanks, >> Sagi. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
