On Wed, Sep 05 2018, Trond Myklebust wrote: > On Thu, 2018-09-06 at 07:12 +1000, NeilBrown wrote: >> On Wed, Sep 05 2018, Olga Kornievskaia wrote: >> >> > On Tue, Sep 4, 2018 at 8:04 PM NeilBrown <neilb@xxxxxxxx> wrote: >> > > >> > > On Tue, Sep 04 2018, Trond Myklebust wrote: >> > > >> > > > On Wed, 2018-09-05 at 08:47 +1000, NeilBrown wrote: >> > > > > With NFSv4.1, the server specifies max_rqst_sz and >> > > > > max_resp_sz in the >> > > > > reply to CREATE session. >> > > > > >> > > > > If the client finds it needs to call nfs4_reset_session(), it >> > > > > might >> > > > > get >> > > > > smaller sizes back, so any pending read/writes would need to >> > > > > be >> > > > > resized. >> > > > > >> > > > > However, I cannot see how the retry handling for reads/writes >> > > > > has any >> > > > > chance to change the size. It looks like a request is broken >> > > > > up to >> > > > > match the original ->rsize and ->wsize, then those individual >> > > > > IO >> > > > > requests can be retried, but the higher level request is >> > > > > never >> > > > > re-evaluated in light of a new size. >> > > > > >> > > > > Am I missing something, or is this not supported at present? >> > > > > If it isn't supported, any suggestions on how best to handle >> > > > > a >> > > > > reduction of the rsize/wsize ?? >> > > > > >> > > > >> > > > Why would a sane server want to do this? >> > > >> > > Why would a sane protocol support it? :-) >> > > >> > > I have a network trace of SLE11-SP4 (3.0 based) talking to "a >> > > NetApp >> > > appliance". >> > > It sends a 64K write and gets NFS4ERR_REQ_TOO_BIG. >> > > It then closes the file (getting NFS4ERR_SEQ_MISORDERED even >> > > though it >> > > used a seq number 1 more than the WRITE request), and then >> > > DESTROY_SESSION and CREATE_SESSION. >> > > The CREATE_SESSION gets "max req size" of 33812 and "max resp >> > > size" of >> > > 33672. >> > > It then opens the file again and retries the 64K write.... >> > > >> > > I have a separate trace showing the initial mount where the sizes >> > > are 71680 >> > > and 81920. >> > > >> > > I don't have a trace where it stops working, but reportedly >> > > writes work >> > > smoothly for some hours after a mount, but then suddenly stop >> > > working. >> > > >> > > The CREATE_SESSION *call* requests I see have the small (32K) >> > > sizes, but >> > > presumably they are the result of a previous CREATE_SESSION reply >> > > giving >> > > a small value. >> > > >> > > I just had a thought. >> > > If one session is shared by two "struct nfs_server" with >> > > different >> > > ->rsize or ->wsize, then the session might get set up with the >> > > smaller >> > > size, and the mount using the larger size will get confused. >> > > In 3.0 (and even 3.10) nfs4_init_session() limits the requested >> > > session >> > > parameters to ->rsize and ->wsize. >> > > That changed in 18aad3d552c7. >> > > >> > > Maybe I just need to remove that code from nfs4_init_session(). >> > > I'll give it a try. >> > > >> > >> > Neil, does the code have this commit? >> > >> > commit 033853325fe3bdc70819a8b97915bd3bca41d3af >> > Author: Olga Kornievskaia <kolga@xxxxxxxxxx> >> > Date: Wed Mar 8 14:39:15 2017 -0500 >> > >> > NFSv4.1 respect server's max size in CREATE_SESSION >> > >> > Currently client doesn't respect max sizes server returns in >> > CREATE_SESSION. >> > nfs4_session_set_rwsize() gets called and server->rsize, >> > server->wsize are 0 >> > so they never get set to the sizes returned by the server. >> > >> > Signed-off-by: Olga Kornievskaia <kolga@xxxxxxxxxx> >> > Signed-off-by: Anna Schumaker <Anna.Schumaker@xxxxxxxxxx> >> > >> > > Thanks, >> > > NeilBrown >> >> Thanks for the suggestion. >> The kernel doesn't have that patch, but I don't think it is relevant. >> The ->rsize does have a suitable value - it isn't zero. >> The problem is that the session limit appears to change, and the >> client >> doesn't adjust to the change. >> >> My current theory is that the client actually requested the change, >> though on behalf of a different filesystem using the same session. >> > > So perhaps the right thing to do then, is to advertise a session max > response/reply size of NFS_MAX_FILE_IO_SIZE + > max(nfs41_maxread_overhead,nfs41_maxwrite_overhead) and just expect the > server negotiate that value down if it needs to? Agreed - and that is exactly what Linux has done since about 3.11. My customer is still on 3.0 which advertises ->[wr]size, and this change wasn't obvious from the change logs so I didn't find it during my initial investigation. I've provided a patch and hope to get feedback in about a week. Thanks, NeilBrown
Attachment:
signature.asc
Description: PGP signature
