On Thu, 2020-04-23 at 17:33 -0400, Olga Kornievskaia wrote: > On Tue, Apr 21, 2020 at 5:20 PM Trond Myklebust < > trondmy@xxxxxxxxxxxxxxx> wrote: > > On Tue, 2020-04-21 at 15:47 -0400, Olga Kornievskaia wrote: > > > On Mon, Apr 20, 2020 at 5:20 PM Trond Myklebust < > > > trondmy@xxxxxxxxxxxxxxx> wrote: > > > > On Mon, 2020-04-20 at 15:35 -0400, Olga Kornievskaia wrote: > > > > > On Mon, Apr 20, 2020 at 3:02 PM Trond Myklebust < > > > > > trondmy@xxxxxxxxxxxxxxx> wrote: > > > > > > On Mon, 2020-04-20 at 10:59 -0400, Olga Kornievskaia wrote: > > > > > > > On Mon, Apr 20, 2020 at 10:53 AM Olga Kornievskaia < > > > > > > > olga.kornievskaia@xxxxxxxxx> wrote: > > > > > > > > Yes we are consistent in requesting to same connection > > > > > > > > to > > > > > > > > with > > > > > > > > the > > > > > > > > same channel binding, but we don't send > > > > > > > > BIND_CONN_TO_SESSION as > > > > > > > > the > > > > > > > > first thing on the "main" connection (ie connection > > > > > > > > that > > > > > > > > cared > > > > > > > > the > > > > > > > > CREATE_SESSION and was bound to fore and back channel > > > > > > > > by > > > > > > > > default). > > > > > > > > When that connection is reset, the first thing that > > > > > > > > happens > > > > > > > > is > > > > > > > > the > > > > > > > > client re-sends the operation that was not replied to. > > > > > > > > That > > > > > > > > has > > > > > > > > a > > > > > > > > SEQUENCE and by the rule the server binds that > > > > > > > > connection > > > > > > > > to > > > > > > > > the > > > > > > > > fore channel only (and sets the callback being down). > > > > > > > > We > > > > > > > > then > > > > > > > > send > > > > > > > > BIND_CONN_TO_SESSION and request FORE_OR_BOTH where > > > > > > > > this > > > > > > > > has > > > > > > > > already been bound to FORE only. > > > > > > > > > > > > > > > > > > > > > > How about this: before we send BIND_CONN_TO_SESSION with > > > > > > > fore_or_both, we somehow always reset the connection > > > > > > > (maybe > > > > > > > you > > > > > > > were > > > > > > > suggestion that already and i wasn't following). > > > > > > > > > > > > No. I didn't realise that we were being automatically set > > > > > > to > > > > > > just > > > > > > the > > > > > > fore channel. However as I said earlier, the spec says that > > > > > > the > > > > > > server > > > > > > MUST reply with NFS4ERR_INVAL in this case. It is not > > > > > > allowed > > > > > > to > > > > > > just > > > > > > return NFS4_OK and silently set the wrong channel binding. > > > > > > > > > > How's this: > > > > > client sends BIND_CONN_TO_SESSION with FORE_OR_BOTH, server > > > > > select > > > > > FORE channel. connection breaks before the reply gets to the > > > > > client. > > > > > Client resends. Is the server suppose to now fail with > > > > > ERR_INVAL? > > > > > > > > > > There actually is such a thing between demand and request. > > > > > FORE > > > > > and > > > > > BACK are demands and FORE_OR_BOTH and BACK_OR_BOTH are > > > > > requests. > > > > > The > > > > > spec writes only about demands and not the requests. I > > > > > believe > > > > > that's > > > > > intentional because BIND_CONN_TO_SESSIOn doesn't have a > > > > > sequence > > > > > and > > > > > not protected by reply session semantics. > > > > > > > > OK. However if we take that interpretation, then the question > > > > is > > > > why > > > > would the server downgrade our FORE_OR_BOTH to FORE and what is > > > > the > > > > point of the client even retrying at all in that case? > > > > > > As far as I can tell, the client behaves improperly. It shouldn't > > > have > > > sent an operation on a new connection before sending > > > BIND_CONN_TO_SESSION. > > > > > > > The server can always reject the client's back channel creation > > > > attempt, but doing so has consequences: it means there is no > > > > way to > > > > hand out delegations or layouts. So I'm confused by the concept > > > > of > > > > a > > > > server that sets SEQ4_STATUS_CB_PATH_DOWN or > > > > SEQ4_STATUS_CB_PATH_DOWN_SESSION, but then doesn't allow the > > > > client > > > > to > > > > set a back channel. > > > > > > Because we can't guarantee that BIND_CONN_TO_SESSION happens as > > > the > > > first operation, I think the solution is for the transport that > > > will > > > do FORE_OR_BOTH request, the client must reset the connection > > > first? > > > Would that be acceptable? > > > > > > > It sounds like we have no choice. However in that case, we should > > not > > try to recover in the case where the attempt to bind the > > backchannel > > fails. > > I can't get the client to reset the connection. Is calling > xprt_force_disconnect() suppose to do that? If so, then that doesn't > happen. I have tried calling into xprt->ops->close(xprt) but that > leads to badness as well... Any suggestions of how to get connection > reset? Is xs_reset_transport() it? It's hard to figure out how all > these functions are different when their names imply connection > resetting... xprt_force_disconnect() should be the right function. Note, though, that the function is asynchronous (it queues up the call to xprt_autoclose() on the xprtiod workqueue), so it won't necessarily immediately show the connection as being broken. However it should ensure that no further requests are transmitted on that xprt until after the connection breaks. -- Trond Myklebust Linux NFS client maintainer, Hammerspace trond.myklebust@xxxxxxxxxxxxxxx
