Dave Chinner <david@xxxxxxxxxxxxx> 于2024年11月26日周二 08:51写道： > > is simply restating what you said in the previous email that I > explicitly told you didn't answer the question I was asking you. > > Please listen to what I'm asking you to do. You don't need to > explain anything to me, I just want you to run an experiment and > report the results. > > This isn't a hard thing to do: the inode32 filesystem should fill to > roughly 50% before it really starts to spill to the lower AGs. > Record and paste the 'xfs_spaceman -c "freesp -a X"' histograms for > each AG when the filesystem is a little over half full. > > That's it. I don't need you to explain anything to me, I simply want > to know if the inode32 allocation policy does, in fact, work the way > it is expected to under your problematic workload. > > -Dave. > -- > Dave Chinner > david@xxxxxxxxxxxxx Hi, sorry for the delay. Seeing that others(adding them to CC list) have also encountered this issue: https://lore.kernel.org/all/20241216130551.811305-1-txpeng@xxxxxxxxxxx/ As an reference, maybe we should give out the result we got so far: +---------------+--------+--------+--------+ | Space Used (%)| Normal | inode32| AF | +---------------+--------+--------+--------+ | 30 | 35.11 | 35.25 | 35.11 | | 41 | 57.35 | 57.58 | 55.96 | | 46 | 71.48 | 71.74 | 54.04 | | 51 | 88.40 | 88.68 | 49.49 | | 56 | 100.00 | 100.00 | 43.91 | | 62 | | | 37.00 | | 67 | | | 28.12 | | 72 | | | 16.32 | | 77 | | | 19.51 | +---------------+--------+--------+--------+ The raw data will be attached in the tail of the mail. The first column represents the percentage of the space used. The rest three columns represents the fragmentation of the free space, which is the percentage of free extent in range [1,1] from the output of "xfs_db -c 'freesp' $test_dev". How to test the Normal vs AF yourself? Apply the patches and follow the commands in: https://lore.kernel.org/linux-xfs/20241104014439.3786609-1-zhangshida@xxxxxxxxxx/ How to test the inode32 yourself? 1. we need to do some hack to the kernel at first: ============ diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c index 09dc44480d16..69fa9f8867df 100644 --- a/fs/xfs/xfs_super.c +++ b/fs/xfs/xfs_super.c @@ -253,9 +253,10 @@ xfs_set_inode_alloc_perag( } set_bit(XFS_AGSTATE_ALLOWS_INODES, &pag->pag_opstate); - if (pag->pag_agno < max_metadata) + if (pag->pag_agno < max_metadata) { + pr_info("%s===agno:%d\n", __func__, pag->pag_agno); set_bit(XFS_AGSTATE_PREFERS_METADATA, &pag->pag_opstate); - else + } else clear_bit(XFS_AGSTATE_PREFERS_METADATA, &pag->pag_opstate); return true; } @@ -312,7 +313,7 @@ xfs_set_inode_alloc( * sufficiently large, set XFS_OPSTATE_INODE32 if we must alter * the allocator to accommodate the request. */ - if (xfs_has_small_inums(mp) && ino > XFS_MAXINUMBER_32) + if (xfs_has_small_inums(mp)) xfs_set_inode32(mp); else xfs_clear_inode32(mp); ========== so that we can test inode32 in a small disk img and observe it in a controllable way. 2. Do the same test as the method we used to test Normal vs AF, but with a little change. 2.1. Create an 1g sized img file and format it as xfs: dd if=/dev/zero of=test.img bs=1M count=1024 mkfs.xfs -f test.img sync 2.2. Make a mount directory: mkdir mnt 2.3. Run the auto_frag.sh script, which will call another scripts To enable the inode32, you should change the mount option in frag.sh: ========== - mount -o af1=1 $test_dev $test_mnt + mount -o inode32 $test_dev $test_mnt ========== run: ./auto_frag.sh 1 And we are still hesitant about whether we should report these results since: 1. it's tested with the assumption that the hack that we did to the inode32 will have no impact on the estimation of the metadata preference method. 2. it's tested under an alternate-punching script instead of some real MySQL workload. And I am afraid that Dave will blame us for not doing exactly what you told us to test. Sorry.:p Maybe we should port the algorithm to a release version and do a few months test with some users or database guys for the inode32 or the new algorithm in a whole. We should reply back at that time maybe. And Tianxiang, would you mind working with us on the problem? Teamwork will be quite efficient. We'll try our best to figure out a way to see how to let everyone play an important role in this work. Cheers, Shida ===============Attachment 1: Normal===================== test_dev:test.img test_mnt:mnt/ fize_size:512000KB mount test.img mnt/ file:mnt//frag size:500MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 285M 676M 30% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 63630 63630 35.11 2048 4095 1 2923 1.61 32768 65536 2 114672 63.28 test_dev:test.img test_mnt:mnt/ fize_size:204800KB mount test.img mnt/ file:mnt//frag2 size:200MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 386M 575M 41% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 89127 89127 57.35 2048 4095 1 2923 1.88 8192 16383 1 14226 9.15 32768 65536 1 49144 31.62 test_dev:test.img test_mnt:mnt/ fize_size:102400KB mount test.img mnt/ file:mnt//frag3 size:100MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 436M 525M 46% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 101877 101877 71.48 2048 4095 1 2923 2.05 8192 16383 1 14226 9.98 16384 32767 1 23492 16.48 test_dev:test.img test_mnt:mnt/ fize_size:102400KB mount test.img mnt/ file:mnt//frag4 size:100MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 486M 475M 51% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 114579 114579 88.40 512 1023 1 811 0.63 8192 16383 1 14226 10.98 test_dev:test.img test_mnt:mnt/ fize_size:102400KB mount test.img mnt/ file:mnt//frag5 size:100MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 537M 424M 56% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 116730 116730 100.00 ===============Attachment 2: inode 32===================== test_dev:test.img test_mnt:mnt/ fize_size:512000KB mount -o af1=1 test.img mnt/ file:mnt//frag size:500MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 285M 676M 30% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 63887 63887 35.25 2048 4095 1 2931 1.62 32768 65536 2 114407 63.13 test_dev:test.img test_mnt:mnt/ fize_size:204800KB mount -o af1=1 test.img mnt/ file:mnt//frag2 size:200MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 386M 575M 41% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 89487 89487 57.58 2048 4095 1 2931 1.89 8192 16383 1 13858 8.92 32768 65536 1 49144 31.62 test_dev:test.img test_mnt:mnt/ fize_size:102400KB mount -o af1=1 test.img mnt/ file:mnt//frag3 size:100MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 435M 526M 46% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 102235 102235 71.74 2048 4095 1 2931 2.06 8192 16383 1 13858 9.72 16384 32767 1 23492 16.48 test_dev:test.img test_mnt:mnt/ fize_size:102400KB mount -o af1=1 test.img mnt/ file:mnt//frag4 size:100MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 486M 475M 51% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 114937 114937 88.68 512 1023 1 819 0.63 8192 16383 1 13858 10.69 test_dev:test.img test_mnt:mnt/ fize_size:102400KB mount -o af1=1 test.img mnt/ file:mnt//frag5 size:100MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 537M 424M 56% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 116730 116730 100.00 ===============Attachment 3: AF===================== test_dev:test.img test_mnt:mnt/ fize_size:512000KB mount -o af1=1 test.img mnt/ file:mnt//frag size:500MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 285M 676M 30% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 63630 63630 35.11 2048 4095 1 2923 1.61 32768 65536 2 114672 63.28 test_dev:test.img test_mnt:mnt/ fize_size:204800KB mount -o af1=1 test.img mnt/ file:mnt//frag2 size:200MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 385M 576M 41% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 86974 86974 55.96 2048 4095 1 2923 1.88 32768 65536 1 65528 42.16 test_dev:test.img test_mnt:mnt/ fize_size:102400KB mount -o af1=1 test.img mnt/ file:mnt//frag3 size:100MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 436M 525M 46% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 77038 77038 54.04 32768 65536 1 65528 45.96 test_dev:test.img test_mnt:mnt/ fize_size:102400KB mount -o af1=1 test.img mnt/ file:mnt//frag4 size:100MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 486M 475M 51% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 64186 64186 49.48 32768 65536 1 65528 50.52 test_dev:test.img test_mnt:mnt/ fize_size:102400KB mount -o af1=1 test.img mnt/ file:mnt//frag5 size:100MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 536M 425M 56% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 51312 51312 43.91 2 3 11 22 0.02 32768 65536 1 65528 56.07 test_dev:test.img test_mnt:mnt/ fize_size:102400KB mount -o af1=1 test.img mnt/ file:mnt//frag6 size:100MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 586M 375M 62% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 38486 38486 37.00 32768 65536 1 65528 63.00 test_dev:test.img test_mnt:mnt/ fize_size:102400KB mount -o af1=1 test.img mnt/ file:mnt//frag7 size:100MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 637M 324M 67% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 25633 25633 28.12 32768 65536 1 65528 71.88 test_dev:test.img test_mnt:mnt/ fize_size:102400KB mount -o af1=1 test.img mnt/ file:mnt//frag8 size:100MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 687M 274M 72% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 12783 12783 16.32 32768 65536 1 65528 83.68 test_dev:test.img test_mnt:mnt/ fize_size:102400KB mount -o af1=1 test.img mnt/ file:mnt//frag9 size:100MB Filesystem Type Size Used Avail Use% Mounted on /dev/loop0 xfs 960M 737M 224M 77% /data/proj/frag_test/mnt umount test.img from to extents blocks pct 1 1 12768 12768 19.51 8192 16383 1 16370 25.02 32768 65536 1 36295 55.47
