On Fri, May 24, 2019 at 11:31:55AM -0600, Alan Post wrote: > On Tue, May 21, 2019 at 03:46:03PM +0000, Trond Myklebust wrote: > > Have you tried upgrading to 4.19.44? There is a fix that went in not > > too long ago that deals with a request leak that can cause stack traces > > like the above that wait forever. > > > > Following up on this. I have set aside a rack of machines and put > Linux 4.19.44 on them. They ran jobs overnight and will do the > same over the long weekend (Memorial day in the US). Given the > error rate (both over time and over submitted jobs) we see across > the cluster this well be enough time to draw a conclusion as to > whether 4.19.44 exhibits this hang. > In the six days I've run Linux 4.19.44 on a single rack, I've seen no occurrences of this hang. Given the incident rate for this issue across the cluster over the same period of time, I would have expected to see one on two incidents on the rack running 4.19.44. This is promising--I'm going to deploy 4.19.44 to another rack by the end of the day Friday May 31st and hope for more of the same. I wondered upthread whether the following commits were what you had in mind when you asked about 4.19.44: 63b0ee126f7e: NFS: Fix an I/O request leakage in nfs_do_recoalesce be74fddc976e: NFS: Fix I/O request leakages Confirming that it is these patches and no others has become topical for me: my upstream is now providing a 4.19.37 build, and I note these two patches are included since 4.19.31 and so are presumably in my now-available upstream 4.19.37 build. If I could trouble you to confirm whether or not this is the complete set of patches you had in mind for the 4.19 branch after 4.19.28 when you recommended I try 4.19.44 I would appreciate it. Lurking on the list for the past week or two and watching everyone's work has been inspiring. Thank you again. I'll report back no later than next week. -A -- Alan Post | Xen VPS hosting for the technically adept PO Box 61688 | Sunnyvale, CA 94088-1681 | https://prgmr.com/ email: adp@xxxxxxxxx