Hi Trond- I'm seeing some interesting client hangs that arise from a well- timed server crash or network partition. The easiest to see is gss_destroy() on an Kerberized NFSv4 mount. NFSv4 asserts the RPC_TASK_NO_RETRANS_TIMEOUT flag (hereafter I'll refer to it as NORTO) when creating a new rpc_clnt. The initial rpc_ping() for that rpc_clnt is done before the logic that sets cl_noretranstimeo, thus that ping works as expected (SOFT | SOFTCONN) and can time out properly if the server isn't responsive. However, once that ping succeeds, cl_noretranstimeo is asserted, and all subsequent RPC requests on that rpc_clnt are with NORTO semantics. When it comes time to destroy the GSS context for that rpc_clnt, the NULL procedure with the GSS decorations is sent with SOFT | SOFTCONN | NORTO. If the server isn't responding at that point, the client continues to retransmit the GSS context destruction request forever, and the xprt and possibly the nfs_client are pinned. The problem also arises for lease management operations such as singleton SEQUENCE or RENEW requests. These are also done with SOFT, as I recall they need to time out properly. But with NORTO + SOFT, they will be retried until a connection loss that might never come. I've thought of some ways to modify the cl_noretranstimeo logic such that it can be disabled for particular RPC tasks, though none is really striking me as exceptionally clever: - Add a field to struct rpc_procinfo that contains a mask of RPC_TASK flags to clear for each procedure. - Add logic to rpc_task_set_client() that clears NORTO in some special cases. - Reverse the meaning of NORTO (e.g., make it RPC_TASK_RETRANS_TIMEOUT) so that it can be set by a caller for particular RPC tasks if the rpc_clnt-default behavior is NORTO. Any thoughts? -- Chuck Lever
