On Sep 25, 2023, at 12:08 AM, Li Dongyang <dongyangli@xxxxxxx> wrote:
>
> This patch prepares the expansion of GDT blocks beyond a
> single group, by make mke2fs to not set BLOCK_UNINIT on
> groups with GDT blocks, block/inode bitmaps, or inode table
> blocks allocated.
>
> Otherwise, we still rely on kernel side to initialize the
> block bitmap if the groups has BLOCK_UNINIT set, and the
> kernel doesn't know a group could have GDT blocks allocated,
> so it would make an bad block bitmap.
>
> As a result, expect output of several tests needs to be changed,
> especially if the test uses dumpe2fs to print the group summary.
>
> Signed-off-by: Li Dongyang <dongyangli@xxxxxxx>
AFAIK, this patch is totally fine to use even in the absence
of the large GDT functionality. It just avoids setting the
BLOCK_UNINIT on any groups that have GDT blocks allocated, and
is making life a bit simpler for the kernel online resize.
At most this is 45 groups' block bitmaps, even for a 576PB
filesystem that is maxing out the 32-bit group descriptor limit.
Reviewed-by: Andreas Dilger <adilger@xxxxxxxxx>
> ---
> lib/ext2fs/initialize.c | 2 +-
> tests/j_ext_long_trans/expect | 2 +-
> tests/j_long_trans/expect | 2 +-
> tests/j_long_trans_mcsum_32bit/expect | 2 +-
> tests/j_long_trans_mcsum_64bit/expect | 2 +-
> tests/j_short_trans_mcsum_64bit/expect | 2 +-
> tests/j_short_trans_recover_mcsum_64bit/expect | 2 +-
> tests/m_bigjournal/expect.1 | 16 ++++++++--------
> tests/m_resize_inode_meta_bg/expect.1 | 6 +++---
> tests/m_uninit/expect.1 | 10 +++++-----
> 10 files changed, 23 insertions(+), 23 deletions(-)
>
> diff --git a/lib/ext2fs/initialize.c b/lib/ext2fs/initialize.c
> index 90012f732..5560cd3f8 100644
> --- a/lib/ext2fs/initialize.c
> +++ b/lib/ext2fs/initialize.c
> @@ -535,7 +535,7 @@ ipg_retry:
> * because the block bitmap needs to be padded.
> */
> if (csum_flag) {
> - if (i != fs->group_desc_count - 1)
> + if (i != fs->group_desc_count - 1 && numblocks == 0)
> ext2fs_bg_flags_set(fs, i,
> EXT2_BG_BLOCK_UNINIT);
> ext2fs_bg_flags_set(fs, i, EXT2_BG_INODE_UNINIT);
> diff --git a/tests/j_ext_long_trans/expect b/tests/j_ext_long_trans/expect
> index ea3c87fcb..b95aa9bc2 100644
> --- a/tests/j_ext_long_trans/expect
> +++ b/tests/j_ext_long_trans/expect
> @@ -77,7 +77,7 @@ Root inode not allocated. Allocate? yes
>
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> -Block bitmap differences: +(1--259) +275 +(291--418) +2341
> +Block bitmap differences: +(1--260) +262 +264 +266 +268 +(275--276) +278 +280 +282 +284 +(291--546) +(675--802) +(931--1058) +(1187--1314) +(1443--1570) +2341 +(8193--8450) +(24577--24834) +(40961--41218) +(57345--57602) +(73729--73986)
> Fix? yes
>
> Free blocks count wrong for group #0 (5838, counted=5851).
> diff --git a/tests/j_long_trans/expect b/tests/j_long_trans/expect
> index 82b3caf17..ee7af96ab 100644
> --- a/tests/j_long_trans/expect
> +++ b/tests/j_long_trans/expect
> @@ -72,7 +72,7 @@ Root inode not allocated. Allocate? yes
>
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> -Block bitmap differences: +(1--259) +273 +275 +289 +(291--418) +(2083--2210) +2341
> +Block bitmap differences: +(1--260) +262 +264 +266 +268 +273 +(275--276) +278 +280 +282 +284 +289 +(291--546) +(675--802) +(931--1058) +(1187--1314) +(1443--1570) +(2083--2210) +2341 +(8193--8450) +(24577--24834) +(40961--41218) +(57345--57602) +(73729--73986)
> Fix? yes
>
> Free blocks count wrong for group #0 (5838, counted=5851).
> diff --git a/tests/j_long_trans_mcsum_32bit/expect b/tests/j_long_trans_mcsum_32bit/expect
> index ffae07a69..0b6cf499c 100644
> --- a/tests/j_long_trans_mcsum_32bit/expect
> +++ b/tests/j_long_trans_mcsum_32bit/expect
> @@ -108,7 +108,7 @@ Root inode not allocated. Allocate? yes
>
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> -Block bitmap differences: +(1--260) +276 +(292--419) +2342 -(139265--155648)
> +Block bitmap differences: +(1--261) +263 +265 +267 +269 +(276--277) +279 +281 +283 +285 +(292--547) +(676--803) +(932--1059) +(1188--1315) +(1444--1571) +2342 +(8193--8451) +(24577--24835) +(40961--41219) +(57345--57603) +(73729--73987) -(139265--155648)
> Fix? yes
>
> Free blocks count wrong for group #0 (5837, counted=5850).
> diff --git a/tests/j_long_trans_mcsum_64bit/expect b/tests/j_long_trans_mcsum_64bit/expect
> index e891def16..b520c91bc 100644
> --- a/tests/j_long_trans_mcsum_64bit/expect
> +++ b/tests/j_long_trans_mcsum_64bit/expect
> @@ -107,7 +107,7 @@ Root inode not allocated. Allocate? yes
>
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> -Block bitmap differences: +(1--262) +278 +(294--421) +2344 -(139265--155648)
> +Block bitmap differences: +(1--263) +265 +267 +269 +271 +(278--279) +281 +283 +285 +287 +(294--549) +(678--805) +(934--1061) +(1190--1317) +(1446--1573) +2344 +(8193--8453) +(24577--24837) +(40961--41221) +(57345--57605) +(73729--73989) -(139265--155648)
> Fix? yes
>
> Free blocks count wrong for group #0 (5835, counted=5848).
> diff --git a/tests/j_short_trans_mcsum_64bit/expect b/tests/j_short_trans_mcsum_64bit/expect
> index d73e28297..5a4f5b94b 100644
> --- a/tests/j_short_trans_mcsum_64bit/expect
> +++ b/tests/j_short_trans_mcsum_64bit/expect
> @@ -28,7 +28,7 @@ Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> -Block bitmap differences: +(0--65) +(67--69) +(71--584) +(1097--2126) +(65536--69631) +(98304--98368)
> +Block bitmap differences: +(0--2126) +(32768--32832) +(65536--69631) +(98304--98368)
> Fix? yes
>
> Inode bitmap differences: +(1--11)
> diff --git a/tests/j_short_trans_recover_mcsum_64bit/expect b/tests/j_short_trans_recover_mcsum_64bit/expect
> index 8c637f122..7139fd80a 100644
> --- a/tests/j_short_trans_recover_mcsum_64bit/expect
> +++ b/tests/j_short_trans_recover_mcsum_64bit/expect
> @@ -30,7 +30,7 @@ Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> -Block bitmap differences: +(0--65) +(67--69) +(71--584) +(1097--2126) +(65536--69631) +(98304--98368)
> +Block bitmap differences: +(0--2126) +(32768--32832) +(65536--69631) +(98304--98368)
> Fix? yes
>
> Inode bitmap differences: +(1--11)
> diff --git a/tests/m_bigjournal/expect.1 b/tests/m_bigjournal/expect.1
> index eb0e3bc38..4e6674665 100644
> --- a/tests/m_bigjournal/expect.1
> +++ b/tests/m_bigjournal/expect.1
> @@ -58,7 +58,7 @@ Group 0: (Blocks 0-32767)
> 31837 free blocks, 5 free inodes, 2 directories, 5 unused inodes
> Free blocks: 931-32767
> Free inodes: 12-16
> -Group 1: (Blocks 32768-65535) [INODE_UNINIT, BLOCK_UNINIT]
> +Group 1: (Blocks 32768-65535) [INODE_UNINIT]
> Backup superblock at 32768, Group descriptors at 32769-32769
> Reserved GDT blocks at 32770-33440
> Block bitmap at 674 (bg #0 + 674), Inode bitmap at 758 (bg #0 + 758)
> @@ -72,7 +72,7 @@ Group 2: (Blocks 65536-98303) [INODE_UNINIT, BLOCK_UNINIT]
> 32768 free blocks, 16 free inodes, 0 directories, 16 unused inodes
> Free blocks: 65536-98303
> Free inodes: 33-48
> -Group 3: (Blocks 98304-131071) [INODE_UNINIT, BLOCK_UNINIT]
> +Group 3: (Blocks 98304-131071) [INODE_UNINIT]
> Backup superblock at 98304, Group descriptors at 98305-98305
> Reserved GDT blocks at 98306-98976
> Block bitmap at 676 (bg #0 + 676), Inode bitmap at 760 (bg #0 + 760)
> @@ -86,7 +86,7 @@ Group 4: (Blocks 131072-163839) [INODE_UNINIT, BLOCK_UNINIT]
> 32768 free blocks, 16 free inodes, 0 directories, 16 unused inodes
> Free blocks: 131072-163839
> Free inodes: 65-80
> -Group 5: (Blocks 163840-196607) [INODE_UNINIT, BLOCK_UNINIT]
> +Group 5: (Blocks 163840-196607) [INODE_UNINIT]
> Backup superblock at 163840, Group descriptors at 163841-163841
> Reserved GDT blocks at 163842-164512
> Block bitmap at 678 (bg #0 + 678), Inode bitmap at 762 (bg #0 + 762)
> @@ -100,7 +100,7 @@ Group 6: (Blocks 196608-229375) [INODE_UNINIT, BLOCK_UNINIT]
> 32768 free blocks, 16 free inodes, 0 directories, 16 unused inodes
> Free blocks: 196608-229375
> Free inodes: 97-112
> -Group 7: (Blocks 229376-262143) [INODE_UNINIT, BLOCK_UNINIT]
> +Group 7: (Blocks 229376-262143) [INODE_UNINIT]
> Backup superblock at 229376, Group descriptors at 229377-229377
> Reserved GDT blocks at 229378-230048
> Block bitmap at 680 (bg #0 + 680), Inode bitmap at 764 (bg #0 + 764)
> @@ -114,7 +114,7 @@ Group 8: (Blocks 262144-294911) [INODE_UNINIT, BLOCK_UNINIT]
> 32768 free blocks, 16 free inodes, 0 directories, 16 unused inodes
> Free blocks: 262144-294911
> Free inodes: 129-144
> -Group 9: (Blocks 294912-327679) [INODE_UNINIT, BLOCK_UNINIT]
> +Group 9: (Blocks 294912-327679) [INODE_UNINIT]
> Backup superblock at 294912, Group descriptors at 294913-294913
> Reserved GDT blocks at 294914-295584
> Block bitmap at 682 (bg #0 + 682), Inode bitmap at 766 (bg #0 + 766)
> @@ -212,7 +212,7 @@ Group 24: (Blocks 786432-819199) [INODE_UNINIT, BLOCK_UNINIT]
> 32768 free blocks, 16 free inodes, 0 directories, 16 unused inodes
> Free blocks: 786432-819199
> Free inodes: 385-400
> -Group 25: (Blocks 819200-851967) [INODE_UNINIT, BLOCK_UNINIT]
> +Group 25: (Blocks 819200-851967) [INODE_UNINIT]
> Backup superblock at 819200, Group descriptors at 819201-819201
> Reserved GDT blocks at 819202-819872
> Block bitmap at 698 (bg #0 + 698), Inode bitmap at 782 (bg #0 + 782)
> @@ -226,7 +226,7 @@ Group 26: (Blocks 851968-884735) [INODE_UNINIT, BLOCK_UNINIT]
> 32768 free blocks, 16 free inodes, 0 directories, 16 unused inodes
> Free blocks: 851968-884735
> Free inodes: 417-432
> -Group 27: (Blocks 884736-917503) [INODE_UNINIT, BLOCK_UNINIT]
> +Group 27: (Blocks 884736-917503) [INODE_UNINIT]
> Backup superblock at 884736, Group descriptors at 884737-884737
> Reserved GDT blocks at 884738-885408
> Block bitmap at 700 (bg #0 + 700), Inode bitmap at 784 (bg #0 + 784)
> @@ -554,7 +554,7 @@ Group 80: (Blocks 2621440-2654207) [INODE_UNINIT, BLOCK_UNINIT]
> 32768 free blocks, 16 free inodes, 0 directories, 16 unused inodes
> Free blocks: 2621440-2654207
> Free inodes: 1281-1296
> -Group 81: (Blocks 2654208-2686975) [INODE_UNINIT, BLOCK_UNINIT]
> +Group 81: (Blocks 2654208-2686975) [INODE_UNINIT]
> Backup superblock at 2654208, Group descriptors at 2654209-2654209
> Reserved GDT blocks at 2654210-2654880
> Block bitmap at 754 (bg #0 + 754), Inode bitmap at 838 (bg #0 + 838)
> diff --git a/tests/m_resize_inode_meta_bg/expect.1 b/tests/m_resize_inode_meta_bg/expect.1
> index 7feaed9d8..83c7bc57c 100644
> --- a/tests/m_resize_inode_meta_bg/expect.1
> +++ b/tests/m_resize_inode_meta_bg/expect.1
> @@ -67,7 +67,7 @@ Group 0: (Blocks 0-255) [ITABLE_ZEROED]
> 159 free blocks, 53 free inodes, 2 directories, 53 unused inodes
> Free blocks: 97-255
> Free inodes: 12-64
> -Group 1: (Blocks 256-511) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> +Group 1: (Blocks 256-511) [INODE_UNINIT, ITABLE_ZEROED]
> Backup superblock at 256, Group descriptor at 257
> Block bitmap at 3 (bg #0 + 3)
> Inode bitmap at 18 (bg #0 + 18)
> @@ -82,7 +82,7 @@ Group 2: (Blocks 512-767) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> 256 free blocks, 64 free inodes, 0 directories, 64 unused inodes
> Free blocks: 512-767
> Free inodes: 129-192
> -Group 3: (Blocks 768-1023) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> +Group 3: (Blocks 768-1023) [INODE_UNINIT, ITABLE_ZEROED]
> Backup superblock at 768
> Block bitmap at 5 (bg #0 + 5)
> Inode bitmap at 20 (bg #0 + 20)
> @@ -97,7 +97,7 @@ Group 4: (Blocks 1024-1279) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> 256 free blocks, 64 free inodes, 0 directories, 64 unused inodes
> Free blocks: 1024-1279
> Free inodes: 257-320
> -Group 5: (Blocks 1280-1535) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> +Group 5: (Blocks 1280-1535) [INODE_UNINIT, ITABLE_ZEROED]
> Backup superblock at 1280
> Block bitmap at 7 (bg #0 + 7)
> Inode bitmap at 22 (bg #0 + 22)
> diff --git a/tests/m_uninit/expect.1 b/tests/m_uninit/expect.1
> index 3c2875527..2058de7a9 100644
> --- a/tests/m_uninit/expect.1
> +++ b/tests/m_uninit/expect.1
> @@ -56,7 +56,7 @@ Group 0: (Blocks 1-8192) [ITABLE_ZEROED]
> 7406 free blocks, 2037 free inodes, 2 directories, 2037 unused inodes
> Free blocks: 787-8192
> Free inodes: 12-2048
> -Group 1: (Blocks 8193-16384) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> +Group 1: (Blocks 8193-16384) [INODE_UNINIT, ITABLE_ZEROED]
> Backup superblock at 8193, Group descriptors at 8194-8194
> Reserved GDT blocks at 8195-8450
> Block bitmap at 8451 (+258), Inode bitmap at 8452 (+259)
> @@ -70,7 +70,7 @@ Group 2: (Blocks 16385-24576) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> 7678 free blocks, 2048 free inodes, 0 directories, 2048 unused inodes
> Free blocks: 16899-24576
> Free inodes: 4097-6144
> -Group 3: (Blocks 24577-32768) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> +Group 3: (Blocks 24577-32768) [INODE_UNINIT, ITABLE_ZEROED]
> Backup superblock at 24577, Group descriptors at 24578-24578
> Reserved GDT blocks at 24579-24834
> Block bitmap at 24835 (+258), Inode bitmap at 24836 (+259)
> @@ -84,7 +84,7 @@ Group 4: (Blocks 32769-40960) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> 7678 free blocks, 2048 free inodes, 0 directories, 2048 unused inodes
> Free blocks: 33283-40960
> Free inodes: 8193-10240
> -Group 5: (Blocks 40961-49152) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> +Group 5: (Blocks 40961-49152) [INODE_UNINIT, ITABLE_ZEROED]
> Backup superblock at 40961, Group descriptors at 40962-40962
> Reserved GDT blocks at 40963-41218
> Block bitmap at 41219 (+258), Inode bitmap at 41220 (+259)
> @@ -98,7 +98,7 @@ Group 6: (Blocks 49153-57344) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> 7678 free blocks, 2048 free inodes, 0 directories, 2048 unused inodes
> Free blocks: 49667-57344
> Free inodes: 12289-14336
> -Group 7: (Blocks 57345-65536) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> +Group 7: (Blocks 57345-65536) [INODE_UNINIT, ITABLE_ZEROED]
> Backup superblock at 57345, Group descriptors at 57346-57346
> Reserved GDT blocks at 57347-57602
> Block bitmap at 57603 (+258), Inode bitmap at 57604 (+259)
> @@ -112,7 +112,7 @@ Group 8: (Blocks 65537-73728) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> 7678 free blocks, 2048 free inodes, 0 directories, 2048 unused inodes
> Free blocks: 66051-73728
> Free inodes: 16385-18432
> -Group 9: (Blocks 73729-81920) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
> +Group 9: (Blocks 73729-81920) [INODE_UNINIT, ITABLE_ZEROED]
> Backup superblock at 73729, Group descriptors at 73730-73730
> Reserved GDT blocks at 73731-73986
> Block bitmap at 73987 (+258), Inode bitmap at 73988 (+259)
> --
> 2.41.0
>
Cheers, Andreas
Attachment:
signature.asc
Description: Message signed with OpenPGP
