[PATCH 2/4] large-fs: improve space diversification for ext4

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Currently we allocated several giant files one by one until limit,
so empty space is located as one chunk which limit code-path coverage.
This patch consume all space with NUM_SPACE_FILES files (by default 1024)
each has same size, and when truncate each one by required delta.
As result we have $NUM_SPACE_FILES chunks of free blocks distributed
across whole filesystem.
This should help us to avoid regressions similar to e7c9e3e99adf6c49

Signed-off-by: Dmitry Monakhov <dmonakhov@xxxxxxxxxx>
---
 common/rc |   40 ++++++++++++++++------------------------
 1 files changed, 16 insertions(+), 24 deletions(-)

diff --git a/common/rc b/common/rc
index c44acea..902fc19 100644
--- a/common/rc
+++ b/common/rc
@@ -440,12 +440,17 @@ _setup_large_ext4_fs()
 	fs_empty_space=$((50*1024*1024*1024))
 
 	[ "$LARGE_SCRATCH_DEV" != yes ] && return 0
+	[ -z "$NUM_SPACE_FILES" ] && export NUM_SPACE_FILES=1024
 	[ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0	
 	fs_empty_space=$((fs_empty_space + $SCRATCH_DEV_EMPTY_SPACE))
 	[ $fs_empty_space -ge $fs_size ] && return 0
 
 	# calculate the size of the file we need to allocate.
+
 	space_to_consume=$(($fs_size - $fs_empty_space))
+	file_size_falloc=$(($fs_size/$NUM_SPACE_FILES))
+	file_size_final=$(($space_to_consume/$NUM_SPACE_FILES))
+
 	# mount the filesystem and create 16TB - 4KB files until we consume
 	# all the necessary space.
 	_scratch_mount 2>&1 >$tmp_dir/mnt.err
@@ -457,33 +462,20 @@ _setup_large_ext4_fs()
 		return $status
 	fi
 	rm -f $tmp_dir/mnt.err
-
-	file_size=$((16*1024*1024*1024*1024 - 4096))
-	nfiles=0
-	while [ $space_to_consume -gt $file_size ]; do
-
+	mkdir $SCRATCH_MNT/.use_space
+	# Consume all space on filesytem
+	for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do 
 		xfs_io -F -f \
-			-c "truncate $file_size" \
-			-c "falloc -k 0 $file_size" \
-			$SCRATCH_MNT/.use_space.$nfiles 2>&1
-		status=$?
-		if [ $status -ne 0 ]; then
-			break;
-		fi
-
-		space_to_consume=$(( $space_to_consume - $file_size ))
-		nfiles=$(($nfiles + 1))
+			-c "truncate $file_size_falloc" \
+			-c "falloc -k 0 $file_size_falloc" \
+			$SCRATCH_MNT/.use_space/use_space.$nfiles 2>&1
 	done
-
-	# consume the remaining space.
-	if [ $space_to_consume -gt 0 ]; then
+	# Truncate files to smaller size, will free chunks of space
+	for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do 
 		xfs_io -F -f \
-			-c "truncate $space_to_consume" \
-			-c "falloc -k 0 $space_to_consume" \
-			$SCRATCH_MNT/.use_space.$nfiles 2>&1
-		status=$?
-	fi
-	export NUM_SPACE_FILES=$nfiles
+			-c "truncate $file_size_final" \
+			$SCRATCH_MNT/.use_space/use_space.$nfiles 2>&1
+	done
 
 	umount $SCRATCH_MNT
 	if [ $status -ne 0 ]; then
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Reiser Filesystem Development]     [Ceph FS]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Linux FS]     [Yosemite National Park]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Device Mapper]     [Linux Media]

  Powered by Linux