On Wed, 2022-06-01 at 09:27 +0200, Mkrtchyan, Tigran wrote: > Hi Olga, > > ----- Original Message ----- > > From: "Olga Kornievskaia" <olga.kornievskaia@xxxxxxxxx> > > To: "trondmy" <trondmy@xxxxxxxxxxxxxxx> > > Cc: "Anna Schumaker" <anna.schumaker@xxxxxxxxxx>, "linux-nfs" > > <linux-nfs@xxxxxxxxxxxxxxx> > > Sent: Tuesday, 31 May, 2022 18:03:34 > > Subject: Re: [PATCH] pNFS: fix IO thread starvation problem during > > LAYOUTUNAVAILABLE error > > <snip> > > To clarify, can you confirm that LAYOUTUNAVAILABLE would only turn > > off > > the inode permanently but for a period of time? > > > > It looks to me that for the LAYOUTTRYLATER, the client would face > > the > > same starvation problem in the same situation. I don't see anything > > marking the segment failed for such error? I believe the client > > returns nolayout for that error, falls back to MDS but allows > > asking > > for the layout for a period of time, having again the queue of > > waiters > > Your assumption doesn't match to our observation. For files that off- > line > (DS down or file is on tape) we return LAYOUTTRYLATER. Usually, > client keep > re-trying LAYOUTGET until file is available again. We use flexfile > layout > as nfs4_file has less predictable behavior. For files that should be > served > by MDS we return LAYOUTUNAVAILABLE. Typically, those files are quite > small > and served with a single READ request, so I haven't observe > repetitive LAYOUTGET > calls. Right. If you're only doing READ, and this is a small file, then you are unlikely to see any repetition of the LAYOUTGET calls like Olga describes, because the client page cache will take care of serialising the initial I/O (by means of the folio lock/page lock) and will cache the results so that no further I/O is needed. The main problems with NFS4ERR_LAYOUTUNAVAILABLE in current pNFS implementation will occur when reading large files and/or when writing to the file. In those cases, we will send a LAYOUTGET each time we need to schedule more I/O because we don't cache the negative result of the previous attempt. -- Trond Myklebust Linux NFS client maintainer, Hammerspace trond.myklebust@xxxxxxxxxxxxxxx
