On Thu 08-09-22 13:47:56, Ojaswin Mujoo wrote: > On Tue, Sep 06, 2022 at 05:29:06PM +0200, Jan Kara wrote: > > Hello, > > > > Here is a second version of my mballoc improvements to avoid spreading > > allocations with mb_optimize_scan=1. The patches fix the performance > > regression I was able to reproduce with reaim on my test machine: > > > > mb_optimize_scan=0 mb_optimize_scan=1 patched > > Hmean disk-1 2076.12 ( 0.00%) 2099.37 ( 1.12%) 2032.52 ( -2.10%) > > Hmean disk-41 92481.20 ( 0.00%) 83787.47 * -9.40%* 90308.37 ( -2.35%) > > Hmean disk-81 155073.39 ( 0.00%) 135527.05 * -12.60%* 154285.71 ( -0.51%) > > Hmean disk-121 185109.64 ( 0.00%) 166284.93 * -10.17%* 185298.62 ( 0.10%) > > Hmean disk-161 229890.53 ( 0.00%) 207563.39 * -9.71%* 232883.32 * 1.30%* > > Hmean disk-201 223333.33 ( 0.00%) 203235.59 * -9.00%* 221446.93 ( -0.84%) > > Hmean disk-241 235735.25 ( 0.00%) 217705.51 * -7.65%* 239483.27 * 1.59%* > > Hmean disk-281 266772.15 ( 0.00%) 241132.72 * -9.61%* 263108.62 ( -1.37%) > > Hmean disk-321 265435.50 ( 0.00%) 245412.84 * -7.54%* 267277.27 ( 0.69%) > > > > The changes also significanly reduce spreading of allocations for small / > > moderately sized files. I'm not able to measure a performance difference > > resulting from this but on eMMC storage this seems to be the main culprit > > of reduced performance. Untarring of raspberry-pi archive touches following > > numbers of groups: > > > > mb_optimize_scan=0 mb_optimize_scan=1 patched > > groups 4 22 7 > > > > To achieve this I have added two more changes on top of v1 - patches 4 and 5. > > Patch 4 makes sure we use locality group preallocation even for files that are > > not likely to grow anymore (previously we have disabled all preallocations for > > such files, however locality group preallocation still makes a lot of sense for > > such files). This patch reduced spread of a small file allocations but larger > > file allocations were still spread significantly because they avoid locality > > group preallocation and as they are not power-of-two in size, they also > > immediately start with cr=1 scan. To address that I've changed the data > > structure for looking up the best block group to allocate from (see patch 5 > > for details). > > > > Stefan, can you please test whether these patches fix the problem for you as > > well? Comments & review welcome. > > > > Honza > > Previous versions: > > Link: http://lore.kernel.org/r/20220823134508.27854-1-jack@xxxxxxx # v1 > > Hi Jan, > > Thanks for the patch. I tested this series on my raspberry pi and I can > confirm that the regression is no longer present with both > mb_optimize_scan=0 and =1 taking similar amount of time to untar. The > allocation spread I'm seeing is as follows: > mb_optimize_scan=0: 10 > mb_optimize_scan=1: 17 (Check graphs for more details) > > For mb_optimize_scan=1, I also compared the spread of locality group PA > eligible files (<64KB) and inode pa files. The results can be found > here: > > mb_optimize_scan=0: > https://github.com/OjaswinM/mbopt-bug/blob/master/grpahs/patchv2-mbopt0.png > mb_optimize_scan=1: > https://github.com/OjaswinM/mbopt-bug/blob/master/grpahs/patchv2.png > mb_optimize_scan=1 (lg pa only): > https://github.com/OjaswinM/mbopt-bug/blob/master/grpahs/patchv2-lgs.png > mb_optimize_scan=1 (inode pa only): > https://github.com/OjaswinM/mbopt-bug/blob/master/grpahs/patchv2-i.png > > The smaller files are now closer together due to the changes to > locality group pa logic. Most of the spread is now coming from mid to > large files. > > To test this further, I created a tar of 2000 100KB files to see if > there is any performance drop due to higher spread of these files and > notcied that although the spread is slightly higher(5BGs vs 9), we don't > see a performance drop while untarring with mb_optimize_scan=1. > > Although we still have some spread, I think we have brought it down to a > much more manageable level and that combined with improvements to CR1 > allocation have given us a good performance improvement. > > Feel free to add: > Tested-by: Ojaswin Mujoo <ojaswin@xxxxxxxxxxxxx> Thanks a lot for the throughout testing! Honza -- Jan Kara <jack@xxxxxxxx> SUSE Labs, CR
