On Thu, 2017-12-14 at 14:22 -0500, Chuck Lever wrote:
> > On Dec 14, 2017, at 2:03 PM, Trond Myklebust <trondmy@primarydata.c
> > om> wrote:
> >
> > Does the RDMA code update the connect cookie when the connection
> > breaks? It looks to me as if it only does that when the connection
> > is
> > re-established. We really want both.
> >
> > > I imagine this issue could also impact write buffer exhaustion
> > > on TCP.
> >
> > See net/sunrpc/xprtsock.c:xs_tcp_state_change()
>
> xprtrdma manipulates the connect_cookie in its connect worker,
> see rpcrdma_connect_worker. This was added by:
>
> commit 575448bd36208f99fe0dd554a43518d798966740
> Author: Tom Talpey <talpey@xxxxxxxxxx>
> AuthorDate: Thu Oct 9 15:00:40 2008 -0400
> Commit: Trond Myklebust <Trond.Myklebust@xxxxxxxxxx>
> CommitDate: Fri Oct 10 15:10:36 2008 -0400
>
> RPC/RDMA: suppress retransmit on RPC/RDMA clients.
>
> Would it be more correct to bump the cookie in rpcrdma_conn_upcall,
> which is the equivalent to xs_tcp_state_change? (if so, why, so
> I can compose a reasonable patch description)
>
> It could be bumped in the RDMA_CM_EVENT_ESTABLISHED and the
> RDMA_CM_EVENT_DISCONNECTED cases, for example. I'm not sure
> RDMA provides a distinction between "server disconnected"
> and "client disconnected" although that probably does not
> matter for this purpose.
>
> But, why would the additional cookie update help? The transport
> is not disconnecting before the deadlock.
>
The connection cookie's purpose is twofold:
1) It tracks whether or not a request has been transmitted on the
current connection or not.
2) It ensures that when several requests with the same connection
cookie all call xprt_conditional_disconnect(), then that results in a
single disconnection event. To do so, it assumes that xprt_autoclose()
will change the cookie if the disconnection attempt is successful.
In TCP we do so in the xs_tcp_state_change(). If the RDMA transport can
guarantee that the call to xprt->ops->close(xprt) is always successful,
then you could do so there.
--
Trond Myklebust
Linux NFS client maintainer, PrimaryData
trond.myklebust@xxxxxxxxxxxxxxx
��.n��������+%������w��{.n�����{��w��� jg��� �����ݢj����G���� ���j:+v���w�m������w�������h�����٥
