On Thu, May 05, 2011 at 10:21:26AM +1000, Dave Chinner wrote: > On Wed, May 04, 2011 at 12:57:36AM +0000, Jamie Heilman wrote: > > Dave Chinner wrote: > > > OK, so the common elements here appears to be root filesystems > > > with small log sizes, which means they are tail pushing all the > > > time metadata operations are in progress. Definitely seems like a > > > race in the AIL workqueue trigger mechanism. I'll see if I can > > > reproduce this and cook up a patch to fix it. > > > > Is there value in continuing to post sysrq-w, sysrq-l, xfs_info, and > > other assorted feedback wrt this issue? I've had it happen twice now > > myself in the past week or so, though I have no reliable reproduction > > technique. Just wondering if more data points will help isolate the > > cause, and if so, how to be prepared to get them. > > > > For whatever its worth, my last lockup was while running > > 2.6.39-rc5-00127-g1be6a1f with a preempt config without cgroups. > > Can you all try the patch below? I've managed to trigger a couple of > xlog_wait() lockups in some controlled load tests. The lockups don't > appear to occur with the following patch to he race condition in > the AIL workqueue trigger. They are still there, just harder to hit. FWIW, I've also discovered that "echo 2 > /proc/sys/vm/drop_caches" gets the system moving again because that changes the push target. I've found two more bugs, and now my test case is now reliably reproducably a 5-10s pause at ~1M created 1byte files and then hanging at about 1.25M files. So there's yet another problem lurking that I need to get to the bottom of. Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
