Hi,
If I log in too many iser targets (above 250?), logging in fails with
FMR allocation error. Then I'm unable to log any target in or out.
Bug is similar to
https://www.spinics.net/lists/linux-rdma/msg51639.html, but in my case,
I'm using Mellanox OFED ver. 4.2-1.2.0.0 on XenServer 7.5 (kernel
4.4.52) and most of configuration files are missing on my system.
* 3 iSCSI/iSER initiators
* each providing multiple (300? 500?) separate targets
[ 697.303260] scsi host73: iSCSI Initiator over iSER
[ 697.327380] fmr_pool: fmr_create failed for FMR 168
[ 697.335201] iser: iser_alloc_fmr_pool: FMR allocation failed, err -12
[ 697.335216] iser: iser_alloc_rx_descriptors: failed allocating rx
descriptors / data buffers
[ 727.121794] iser: iser_disconnected_handler: iscsi_iser connection
isn't bound
I'm unable to verify this issue on newer kernels as there aren't any
available. Dom0 memory has almost direct impact to this bug - at current
setting (10GB) I'm able to log in ~250 targets via iSER. On 4GB, the
number was significantly lower (~80 - 90 targets).
I can't increase dom0 memory indefinitely as there has to be free RAM
for VMs, unless there will be possibility to accurately calculate needed
memory for iSER.
Regards,
Michał
