On Sun, Dec 05, 2010 at 10:04:17PM +0800, Rik van Riel wrote: > On 12/05/2010 01:44 AM, Wu Fengguang wrote: > > I noticed that my NFSROOT test system goes slow responding when there > > is heavy dd to a local disk. Traces show that the NFSROOT's bdi_limit > > is near 0 and many tasks in the system are repeatedly stuck in > > balance_dirty_pages(). > > > > There are two related problems: > > > > - light dirtiers at one device (more often than not the rootfs) get > > heavily impacted by heavy dirtiers on another independent device > > > > - the light dirtied device does heavy throttling because bdi_limit=0, > > and the heavy throttling may in turn withhold its bdi_limit in 0 as > > it cannot dirty fast enough to grow up the bdi's proportional weight. > > > > Fix it by introducing some "low pass" gate, which is a small (<=8MB) > > value reserved by others and can be safely "stole" from the current > > global dirty margin. It does not need to be big to help the bdi gain > > its initial weight. > > Makes a lot of sense to me. > > Acked-by: Rik van Riel <riel@xxxxxxxxxx> Thanks. I find the problem when testing the IO-less balance_dirty_pages(). The old kernel may behave a bit better, but should still benefit from the patch. Now I find one more problem..with a fix. --- Subject: writeback: safety margin for bdi stat error Date: Tue Dec 07 20:38:28 CST 2010 In a simple dd test on a 8p system with "mem=256M", I find the light dirtier tasks on the root fs are all heavily throttled. That happens because the global limit is exceeded. It's unbelievable at first sight, because the test fs doing the heavy dd is under its bdi limit. After doing some tracing, it's discovered that bdi_dirty < bdi_limit < global_limit < nr_dirty So the root cause is, the bdi_dirty is well under nr_dirty due to accounting errors. They should be very close because there is only one heavy dirtied bdi in the system. This can be fixed by using bdi_stat_sum(), however that's costly on large NUMA machines. So do a less costly fix of lowering the bdi limit, so that the accounting errors won't lead to the absurd situation "global limit exceeded but bdi limit not exceeded". CC: Rik van Riel <riel@xxxxxxxxxx> CC: Peter Zijlstra <a.p.zijlstra@xxxxxxxxx> Signed-off-by: Wu Fengguang <fengguang.wu@xxxxxxxxx> --- mm/page-writeback.c | 5 +++++ 1 file changed, 5 insertions(+) --- linux-next.orig/mm/page-writeback.c 2010-12-07 20:35:00.000000000 +0800 +++ linux-next/mm/page-writeback.c 2010-12-07 20:37:34.000000000 +0800 @@ -451,6 +451,11 @@ unsigned long bdi_dirty_limit(struct bac u64 bdi_dirty; long numerator, denominator; + if (likely(dirty > bdi_stat_error(bdi))) + dirty -= bdi_stat_error(bdi); + else + return 0; + /* * Calculate this BDI's share of the dirty ratio. */ -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
