On Wed 09-01-13 17:26:36, Namjae Jeon wrote: <snip> > But in one normal scenario, the changes actually results in > performance degradation. > > Results for ‘dd’ thread on two devices: > Before applying Patch: > #> dd if=/dev/zero of=/mnt/sdb2/file1 bs=1048576 count=800 & > #> dd if=/dev/zero of=/mnt/sda6/file2 bs=1048576 count=2000 & > #> > #> 2000+0 records in > 2000+0 records out > 2097152000 bytes (2.0GB) copied, 77.205276 seconds, 25.9MB/s -> USB > HDD WRITE Speed > > [2]+ Done dd if=/dev/zero of=/mnt/sda6/file2 bs=1048576 count=2000 > #> > #> > #> 800+0 records in > 800+0 records out > 838860800 bytes (800.0MB) copied, 154.528362 seconds, 5.2MB/s -> USB > Flash WRITE Speed > > After applying patch: > #> dd if=/dev/zero of=/mnt/sdb2/file1 bs=1048576 count=800 & > dd if=/ > #> dd if=/dev/zero of=/mnt/sda6/file2 bs=1048576 count=2000 & > #> > #> 2000+0 records in > 2000+0 records out > 2097152000 bytes (2.0GB) copied, 123.844770 seconds, 16.1MB/s ->USB > HDD WRITE Speed > 800+0 records in > 800+0 records out > 838860800 bytes (800.0MB) copied, 141.352945 seconds, 5.7MB/s -> USB > Flash WRITE Speed > > [2]+ Done dd if=/dev/zero of=/mnt/sda6/file2 bs=1048576 count=2000 > [1]+ Done dd if=/dev/zero of=/mnt/sdb2/file1 bs=1048576 count=800 > > So, after applying our changes: > 1) USB HDD Write speed dropped from 25.9 -> 16.1 MB/s > 2) USB Flash Write speed increased marginally from 5.2 -> 5.7 MB/s > > Normally if we have a USB Flash and HDD plugged in system. And if we > initiate the ‘dd’ on both the devices. Once dirty memory is more than > the background threshold, flushing starts for all BDI (The write-back > for the devices will be kicked by the condition): > If (global_page_state(NR_FILE_DIRTY) + > global_page_state(NR_UNSTABLE_NFS) > background_thresh)) > return true; > As the slow device and the fast device always make sure that there is > enough DIRTY data in memory to kick write-back. > Since, USB Flash is slow, the DIRTY pages corresponding to this device > is much higher, resulting in returning ‘true’ everytime from > over_bground_thresh. So, even though HDD might have only few KB of > dirty data, it is also flushed immediately. > This frequent flushing of HDD data results in gradually increasing the > bdi_dirty_limit() for HDD. Interesting. Thanks for testing! So is this just a problem with initial writeout fraction estimation. I.e. if you first let dd to USB HDD run for a couple of seconds to ramp up its fraction and only then start writeout to USB flash, is there still a problem with USB HDD throughput with the changed over_bground_thresh() function? > But, when we introduce the change to control per BDI i.e., > if (global_page_state(NR_FILE_DIRTY) + > global_page_state(NR_UNSTABLE_NFS) > background_thresh && > reclaimable * 2 + bdi_stat_error(bdi) * 2 > bdi_bground_thresh) > > Now, in this case, when we consider the same scenario, writeback for > HDD will only be kicked only if ‘reclaimable * 2 + bdi_stat_error(bdi) > * 2 > bdi_bground_thresh’ > But this condition is not true a lot many number of times, so > resulting in false. I'm surprised it's not true so often... dd(1) should easily fill the caches. But maybe we are oscilating between below-background-threshold and at-dirty-limit situations rather quickly. Do you have recordings of BDI_RECLAIMABLE and BDI_DIRTY from the problematic run? > This continuous failure to start write-back for HDD actually results > in lowering the bdi_dirty_limit for HDD, in a way PAUSING the writer > thread for HDD. > This is actually resulting in less number of WRITE operations per > second for HDD. As, the ‘dd’ on USB HDD will be put to long sleep(MAX > PAUSE) in balance_dirty_pages. > > While for USB Flash, its bdi_dirty_limit is kept on increasing as it > is getting more chance to flush dirty data in over_bground_thresh. As, > bdi_reclaimable > bdi_dirty_limit is true. So, resulting more number > of WRITE operation per second for USB Flash. > From these observations, we feel that these changes might not be > needed. Please let us know in case we are missing on any point here, > we can further check more on this. Well, at least we know changing the condition has unexpected side effects. I'd like to understand those before discarding the idea - because in your setup flusher thread must end up writing rather small amount of pages in each run when it's running continuously and that's not too good either... Honza -- Jan Kara <jack@xxxxxxx> SUSE Labs, CR -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>