On Thu, Aug 25, 2011 at 02:00:58AM +0800, Vivek Goyal wrote: > On Wed, Aug 24, 2011 at 08:12:58AM +0800, Wu Fengguang wrote: > > > You somehow directly jump to > > > > > > balanced_rate = task_ratelimit_200ms * write_bw / dirty_rate > > > > > > without explaining why following will not work. > > > > > > balanced_rate_(i+1) = balance_rate(i) * write_bw / dirty_rate > > > > Thanks for asking that, it's probably the root of confusions, so let > > me answer it standalone. > > > > It's actually pretty simple to explain this equation: > > > > write_bw > > balanced_rate = task_ratelimit_200ms * ---------- (1) > > dirty_rate > > > > If there are N dd tasks, each task is throttled at task_ratelimit_200ms > > for the past 200ms, we are going to measure the overall bdi dirty rate > > > > dirty_rate = N * task_ratelimit_200ms (2) > > > > put (2) into (1) we get > > > > balanced_rate = write_bw / N (3) > > > > So equation (1) is the right estimation to get the desired target (3). > > > > > > As for > > > > write_bw > > balanced_rate_(i+1) = balanced_rate_(i) * ---------- (4) > > dirty_rate > > > > Let's compare it with the "expanded" form of (1): > > > > write_bw > > balanced_rate_(i+1) = balanced_rate_(i) * pos_ratio * ---------- (5) > > dirty_rate > > > > So the difference lies in pos_ratio. > > > > Believe it or not, it's exactly the seemingly use of pos_ratio that > > makes (5) independent(*) of the position control. > > > > Why? Look at (4), assume the system is in a state > > > > - dirty rate is already balanced, ie. balanced_rate_(i) = write_bw / N > > - dirty position is not balanced, for example pos_ratio = 0.5 > > > > balance_dirty_pages() will be rate limiting each tasks at half the > > balanced dirty rate, yielding a measured > > > > dirty_rate = write_bw / 2 (6) > > > > Put (6) into (4), we get > > > > balanced_rate_(i+1) = balanced_rate_(i) * 2 > > = (write_bw / N) * 2 > > > > That means, any position imbalance will lead to balanced_rate > > estimation errors if we follow (4). Whereas if (1)/(5) is used, we > > always get the right balanced dirty ratelimit value whether or not > > (pos_ratio == 1.0), hence make the rate estimation independent(*) of > > dirty position control. > > > > (*) independent as in real values, not the seemingly relations in equation > > Ok, I think I am beginning to see your point. Let me just elaborate on > the example you gave. Thank you very much :) > Assume a system is completely balanced and a task is writing at 100MB/s > rate. > > write_bw = dirty_rate = 100MB/s, pos_ratio = 1; N=1 > > bdi->dirty_ratelimit = 100MB/s > > Now another tasks starts dirtying the page cache on same bdi. Number of > dirty pages should go up pretty fast and likely position ratio feedback > will kick in to reduce the dirtying rate. (rate based feedback does not > kick in till next 200ms) and pos_ratio feedback seems to be instantaneous. That's right. There must be some instantaneous feedback to react to fast workload changes. With pos_ratio providing this capability, the estimated balanced rate can take time to follow. Note that pos_ratio by itself is enough to limit dirty pages within the [freerun, limit] control scope. The cost of (temporarily) large error in balanced rate is, task_ratelimit will be fluctuating much more, due to the fact pos_ratio will depart from 1.0 (to the point it can fully compensate for the rate errors) and dirty pages approaching @freerun or @limit where the slope of pos_ratio goes sharp. The correct estimation of balanced rate serves to drive pos_ratio back to 1.0, where it has the most flat slope. > Assume new pos_ratio is .5 > > So new throttle rate for both the tasks is 50MB/s. > > bdi->dirty_ratelimit = 100MB/s (a feedback has not kicked in yet) > task_ratelimit = bdi->dirty_ratelimit * pos_ratio = 100 *.5 = 50MB/s > > Now lets say 200ms have passed and rate base feedback is reevaluated. > > write_bw > bdi->dirty_ratelimit_(i+1) = bdi->dirty_ratelimit_i * --------- > dirty_bw > > bdi->dirty_ratelimit_(i+1) = 100 * 100/100 = 100MB/s > > Ideally bdi->dirty_ratelimit should have now become 50MB/s as N=2 but > that did not happen. And reason being that there are two feedback control > loops and pos_ratio loops reacts to imbalances much more quickly. Because > previous loop has already reacted to the imbalance and reduced the > dirtying rate of task, rate based loop does not try to adjust anything > and thinks everything is just fine. That's right. > Things are fine in the sense that still dirty_rate == write_bw but > system is not balanced in terms of number of dirty pages and pos_ratio=.5 Yes. The bad thing is, if the above equation (of pure rate feedback) is used, the system is going to remain in that position-imbalanced state forever, which is bad for the smoothness of task_ratelimit. > So you are trying to make one feedback loop aware of second loop so that > if second loop is unbalanced, first loop reacts to that as well and not > just look at dirty_rate and write_bw. So refining new balanced rate by > pos_ratio helps. > write_bw > bdi->dirty_ratelimit_(i+1) = bdi->dirty_ratelimit_i * --------- * pos_ratio > dirty_bw > > Now if global dirty pages are imbalanced, balanced rate will still go > down despite the fact that dirty_bw == write_bw. This will lead to > further reduction in task dirty rate. Which in turn will lead to reduced > number of dirty rate and should eventually lead to pos_ratio=1. Right, that's a good alternative viewpoint to the below one. write_bw bdi->dirty_ratelimit_(i+1) = task_ratelimit_i * --------- dirty_bw (1) the periodic rate estimation uses that to refresh the balanced rate on every 200ms (2) as long as the rate estimation is correct, pos_ratio is able to drive itself to 1.0 > A related question though I should have asked you this long back. How does > throttling based on rate helps. Why we could not just work with two > pos_ratios. One is gloabl postion ratio and other is bdi position ratio. > And then throttle task gradually to achieve smooth throttling behavior. > IOW, what property does rate provide which is not available just by > looking at per bdi dirty pages. Can't we come up with bdi setpoint and > limit the way you have done for gloabl setpoint and throttle tasks > accordingly? Good question. If we have no idea of the balanced rate at all, but still want to limit dirty pages within the range [freerun, limit], all we can do is to throttle the task at eg. 1TB/s at @freerun and 0 at @limit. Then you get a really sharp control line which will make task_ratelimit fluctuate like mad... So the balanced rate estimation is the key to get smooth task_ratelimit, while pos_ratio is the ultimate guarantee for the dirty pages range. Thanks, Fengguang -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
