Hello,
I'm running into an issue where rdma_create_qp_ex returns EINVAL and I was hoping that someone could help me understand what is going on here.
The function that is actually throwing the EINVAL error is the write() call in rdma_init_qp_attr (which is being called by rdma_create_qp_ex):
...
ret = write(id->channel->fd, &cmd, sizeof cmd);
...
It returns -1 and sets errno to 22.
Note, this is an intermittent error and not always reproducible.
The setup and scenario is as follows:
- SPDK NVMF target on Debian 11.3 with top of tree rdma-core libs
- NVMe-oF kernel initiator, Debain 11.5 (no change in rdma-core libs)
- There is a switch between initiator and SPDK NVMF targets
- The kernel initiator is taking to 2 SPDK NVMF targets via DM and round-robin (I don't think this matters)
- On the initiator system there is a 512k block size fio load against 48 NMF subsystems (2 target apps with 24 subsystems)
- When I kill the SPDK target and restart it, then I occasionally get this EINVAL on one of the queue pairs
It's unclear to me why the write call is retuning EINVAL. The file descriptor should be valid since I see the same fd in later qpair creation requests.
Any insights are appreciated.
-- Michael
