On 3/6/2014 12:04 AM, Nicholas A. Bellinger wrote:
On Wed, 2014-03-05 at 14:12 +0200, Sagi Grimberg wrote:
On 3/5/2014 2:06 AM, Nicholas A. Bellinger wrote:
On Tue, 2014-03-04 at 17:17 +0200, Sagi Grimberg wrote:
On 3/4/2014 2:00 AM, Nicholas A. Bellinger wrote:
From: Nicholas Bellinger <nab@xxxxxxxxxxxxxxx>
<SNIP>
More on cleanup flow. isert_cma_handler does not handle
RDMA_CM_EVENT_TIMEWAIT_EXIT.
To be more specific, according to IB spec, when initiating disconnect
(rdma_disconnect/ib_send_cm_dreq),
one should not destroy a used qp until getting TIMEWAIT_EXIT CM event.
We are working on this in iSER initiator.
It might lead to "stale connection" CM rejects on future connections
(SRP also does not do that).
<nod>, I noticed that as well during recent debugging.
However, AFAICT the RDMA_CM_EVENT_TIMEWAIT_EVENT doesn't (always) occur
on the target side after a RDMA_CM_EVENT_DISCONNECTED, and thus far I've
not been able to ascertain what's different about the shutdown sequence
that would make this happen, or not happen..
Any ideas..?
That's probably because the cm_id is destroyed before you get the event.
There is a specific
timout computation to get this event (see IB spec). If you will attempt
to disconnect while
the link is down (initiator won't receive it and send you disconnect
back), you should be able
to see this event. As I understand, in order to comply the spec, the QP
(and the cm_id afterwards)
should be destroyed only when getting this event and not before.
<nod>, thanks for the additional background.
So currently rdma_destroy_qp() + rdma_destroy_id() is being done via
isert_connect_release(), which occurs after the final isert_put_conn()
happens from either the RDMA_CM_EVENT_DISCONNECTED handler, or within
isert_free_conn() in one of the per connection kernel thread contexts
via iscsit_close_connection().
If I understand the above correctly, the isert_put_conn() should move
from the RDMA_CM_EVENT_DISCONNECTED handler into the TIMEWAIT_EVENT
handler, yes..?
Yes.
And it's safe to assume that DISCONNECTED will always occur before
TIMEWAIT_EVENT, right..?
DISCONNECTED event may not even come at all (in case the initiator
didn't call rdma_disconnect). no guarantees here..
But, if once we get the TIMEWAIT event, we destroy the qp and the
*cm_id*, we won't get any CM events at all.
As I understand, we don't even need to explicitly destroy the cm_id, we
can just return a non-zero return from cma_handler
for TIMEWAIT events which will cause rdma_cm to implicitly destroy the
cm_id.
Hope this helps,
Sagi.
--nab
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
