Re: [PATCH] Add test 248: Check filesystem FITRIM implementation

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 11/26/10 8:12 AM, Lukas Czerner wrote:
> FITRIM ioctl  is used on a mounted filesystem to discard (or "trim")
> blocks which are not in use by the filesystem.  This is useful for
> solid-state drives (SSDs) and thinly-provi-sioned storage. This test
> helps to verify filesystem FITRIM implementation to assure that it
> does not corrupts data.
> 
> This test creates checksums of all files in /usr/share/doc directory and
> run several processes which clear its working directory on SCRATCH_MNT,
> then copy everything from /usr/share/doc into its working directory, create
> list of files in working directory and its checksums and compare it with the
> original list of checksums. Every process works in the loop so it repeat
> remove->copy->check, while fstrim tool is running simultaneously.
> 
> Fstrim is just a helper tool which uses FITRIM ioctl to actually do the
> filesystem discard.
> 
> I found this very useful because when the FITRIM is really buggy (thus
> data-destroying) the 248 test will notice, because checksums will most
> likely change.
> 
> Signed-off-by: Lukas Czerner <lczerner@xxxxxxxxxx>
> ---
>  248          |  163 +++++++++++++++++++++++++++++++++++++
>  248.out      |    3 +
>  group        |    1 +
>  src/Makefile |    2 +-
>  src/fstrim.c |  257 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  5 files changed, 425 insertions(+), 1 deletions(-)
>  create mode 100755 248
>  create mode 100644 248.out
>  create mode 100644 src/fstrim.c
> 
> diff --git a/248 b/248
> new file mode 100755
> index 0000000..0d2f17f
> --- /dev/null
> +++ b/248
> @@ -0,0 +1,163 @@
> +#!/bin/bash
> +# FS QA Test No. 248
> +#
> +# This test was created in order to verify filesystem FITRIM implementation.
> +# By many concurrent copy and remove operations and checking that files
> +# does not change after copied into SCRATCH_MNT test if FITIM implementation

typo up there in "FITIM", just FWIW.

> +# corrupts the filesystem (data/metadata).
> +#
> +#-----------------------------------------------------------------------
> +# Copyright 2010 (C) Red Hat, Inc., Lukas Czerner <lczerner@xxxxxxxxxx>
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#-----------------------------------------------------------------------
> +
> +owner=lczerner@xxxxxxxxxx
> +
> +seq=`basename $0`
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=`mktemp -d`
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 3
> +trap "_destroy; exit \$status" 2 15
> +chpid=0
> +mypid=$$
> +
> +# get standard environment, filters and checks
> +. ./common.rc
> +. ./common.filter
> +
> +# real QA test starts here
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +_scratch_mkfs >/dev/null 2>&1
> +_scratch_mount
> +
> +_cleanup()
> +{
> +	rm -rf $tmp
> +}
> +
> +_destroy()
> +{
> +	kill $pids $fstrim_pid
> +	wait $pids $fstrim_pid
> +	rm -rf $tmp
> +}
> +
> +_destroy_fstrim()
> +{
> +	kill $fpid
> +	wait $fpid
> +}
> +
> +fstrim_loop()
> +{
> +	trap "_destroy_fstrim; exit \$status" 2 15
> +	fsize=$(df | grep $SCRATCH_MNT | grep $SCRATCH_DEV  | awk '{print $2}')
> +
> +	while true ; do
> +		step=1048576
> +		start=0
> +		$here/src/fstrim $SCRATCH_MNT &
> +		fpid=$!
> +		wait $fpid
> +		while [ $start -lt $fsize ] ; do
> +			$here/src/fstrim -s ${start}k -l ${step}k $SCRATCH_MNT &
> +			fpid=$!
> +			wait $fpid
> +			start=$(( $start + $step ))
> +		done

I may be dense here but 

a) why do you background and then immediately wait?
b) does this start a whole-device trim followed by several
smaller range-trims?

I could do with some comments, I suppose.

> +	done
> +}
> +
> +function check_sums() {
> +	dir=$1
> +
> +	(
> +	cd $SCRATCH_MNT/$p
> +	find -P . -xdev -type f -print0 | xargs -0 md5sum | sort -o $tmp/stress.$$.$p
> +	)
> +
> +	diff $tmp/content.sums $tmp/stress.$$.$p
> +	if [ $? -ne 0 ]; then
> +		echo "!!!Checksums has changed - Filesystem possibly corrupted!!!\n"
> +		kill $mypid

what is $mypid?  Oh right $$ ...  why not:

_fail "!!!Checksums has changed - Filesystem possibly corrupted!!!"

> +	fi
> +	rm -f $tmp/stress.$$.$p
> +}
> +
> +function run_process() {
> +	local p=$1
> +	repeat=10
> +
> +	sleep $((5*$p))s &
> +	export chpid=$! && wait $chpid &> /dev/null

I guess I don't sight-read bash very well.  What's going
on with all the backgrounding/waiting here?

> +	chpid=0
> +
> +	while [ $repeat -gt 0 ]; do
> +
> +		# Remove old directories.
> +		rm -rf $SCRATCH_MNT/$p
> +		export chpid=$! && wait $chpid &> /dev/null

and here?

> +		# Copy content -> partition.
> +		mkdir $SCRATCH_MNT/$p
> +		cp -ax $content/* $SCRATCH_MNT/$p
> +		export chpid=$! && wait $chpid &> /dev/null
> +
> +		check_sums
> +		repeat=$(( $repeat - 1 ))
> +	done
> +}
> +
> +nproc=20
> +content=/usr/share/doc
> +
> +# Check for FITRIM support
> +echo -n "Checking FITRIM support: "
> +$here/src/fstrim -l 10M $SCRATCH_MNT
> +[ $? -ne 0 ] && exit
> +echo "done."
> +
> +mkdir -p $tmp
> +
> +(
> +cd $content
> +find -P . -xdev -type f -print0 | xargs -0 md5sum | sort -o $tmp/content.sums
> +)
> +
> +echo -n "Running the test: "
> +pids=""
> +fstrim_loop &
> +fstrim_pid=$!
> +p=1
> +while [ $p -le $nproc ]; do
> +	run_process $p &
> +	pids="$pids $!"
> +	p=$(($p+1))
> +done
> +echo "done."
> +
> +wait $pids
> +kill $fstrim_pid
> +wait $fstrim_pid
> +
> +status=0
> +_check_scratch_fs

Scratch fs should get checked automatically after the test I think?
I guess other tests do this, but I'm not sure it's necessary
unless filesystems are made, remounted, etc in a loop during
the test

> +exit
> diff --git a/248.out b/248.out
> new file mode 100644
> index 0000000..880d9c7
> --- /dev/null
> +++ b/248.out
> @@ -0,0 +1,3 @@
> +QA output created by 248
> +Checking FITRIM support: done.
> +Running the test: done.
> diff --git a/group b/group
> index 0f94dd9..fea84ed 100644
> --- a/group
> +++ b/group
> @@ -361,3 +361,4 @@ deprecated
>  245 auto quick dir
>  246 auto quick rw
>  247 auto quick rw
> +248 ioctl

I suppose maybe a trim group would be worthwhile at some point ...

> diff --git a/src/Makefile b/src/Makefile
> index b827bd0..885fd65 100644
> --- a/src/Makefile
> +++ b/src/Makefile
> @@ -17,7 +17,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
>  	preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest \
>  	locktest unwritten_mmap bulkstat_unlink_test t_stripealign \
>  	bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable \
> -	stale_handle
> +	stale_handle fstrim
>  
>  SUBDIRS =
>  
> diff --git a/src/fstrim.c b/src/fstrim.c
> new file mode 100644
> index 0000000..6686c99
> --- /dev/null
> +++ b/src/fstrim.c
> @@ -0,0 +1,257 @@
> +/*
> + * fstrim.c -- discard the part (or whole) of mounted filesystem.
> + *
> + * Copyright (C) 2009 Red Hat, Inc., Lukas Czerner <lczerner@xxxxxxxxxx>

2010 I think :)

> + *
> + * This program is free software: you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation, either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + *
> + * This program uses FITRIM ioctl to discard parts or the whole filesystem
> + * online (mounted). You can specify range (start and lenght) to be
> + * discarded, or simply discard while filesystem.
> + *
> + * Usage: fstrim [options] <mount point>
> + */
> +
> +#include <string.h>
> +#include <unistd.h>
> +#include <stdlib.h>
> +#include <errno.h>
> +#include <stdio.h>
> +#include <stdint.h>
> +#include <fcntl.h>
> +#include <limits.h>
> +#include <stdarg.h>
> +#include <getopt.h>
> +
> +#include <sys/ioctl.h>
> +#include <sys/stat.h>
> +#include <linux/fs.h>
> +
> +#ifndef FITRIM
> +struct fstrim_range {
> +	uint64_t start;
> +	uint64_t len;
> +	uint64_t minlen;
> +};
> +#define FITRIM		_IOWR('X', 121, struct fstrim_range)
> +#endif
> +
> +const char *program_name = "fstrim";
> +
> +struct options {
> +	struct fstrim_range *range;
> +	char mpoint[PATH_MAX];
> +	char verbose;
> +};
> +
> +static void usage(void)
> +{
> +	fprintf(stderr,
> +		"Usage: %s [-s start] [-l length] [-m minimum-extent]"
> +		" [-v] {mountpoint}\n\t"
> +		"-s Starting Byte to discard from\n\t"
> +		"-l Number of Bytes to discard from the start\n\t"
> +		"-m Minimum extent length to discard\n\t"
> +		"-v Verbose - number of discarded bytes\n",
> +		program_name);
> +}
> +
> +static void err_exit(const char *fmt, ...)
> +{
> +	va_list pvar;
> +	va_start(pvar, fmt);
> +	vfprintf(stderr, fmt, pvar);
> +	va_end(pvar);
> +	usage();
> +	exit(EXIT_FAILURE);
> +}
> +
> +/**
> + * Get the number from argument. It can be number followed by
> + * units: k|K, m|M, g|G, t|T
> + */
> +static unsigned long long get_number(char **optarg)
> +{
> +	char *opt, *end;
> +	unsigned long long number, max;
> +
> +	/* get the max to avoid overflow */
> +	max = ULLONG_MAX / 1024;
> +	number = 0;
> +	opt = *optarg;
> +
> +	if (*opt == '-') {
> +		err_exit("%s: %s (%s)\n", program_name,
> +			 strerror(ERANGE), *optarg);
> +	}
> +
> +	errno = 0;
> +	number = strtoul(opt, &end , 0);
> +	if (errno)
> +		err_exit("%s: %s (%s)\n", program_name,
> +			 strerror(errno), *optarg);
> +
> +	/*
> +	 * Convert units to numbers. Fall-through stack is used for units
> +	 * so absence of breaks is intentional.
> +	 */

wish we had a library for this sort of thing :(

> +	switch (*end) {
> +	case 'T': /* terabytes */
> +	case 't':
> +		if (number > max)
> +			err_exit("%s: %s (%s)\n", program_name,
> +				 strerror(ERANGE), *optarg);
> +		number *= 1024;
> +	case 'G': /* gigabytes */
> +	case 'g':
> +		if (number > max)
> +			err_exit("%s: %s (%s)\n", program_name,
> +				 strerror(ERANGE), *optarg);
> +		number *= 1024;
> +	case 'M': /* megabytes */
> +	case 'm':
> +		if (number > max)
> +			err_exit("%s: %s (%s)\n", program_name,
> +				 strerror(ERANGE), *optarg);
> +		number *= 1024;
> +	case 'K': /* kilobytes */
> +	case 'k':
> +		if (number > max)
> +			err_exit("%s: %s (%s)\n", program_name,
> +				 strerror(ERANGE), *optarg);
> +		number *= 1024;
> +		end++;
> +	case '\0': /* end of the string */
> +		break;
> +	default:
> +		err_exit("%s: %s (%s)\n", program_name,
> +			 strerror(EINVAL), *optarg);
> +		return 0;
> +	}
> +
> +	if (*end != '\0') {
> +		err_exit("%s: %s (%s)\n", program_name,
> +			 strerror(EINVAL), *optarg);
> +	}
> +
> +	return number;
> +}
> +
> +static int parse_opts(int argc, char **argv, struct options *opts)
> +{
> +	int c;
> +
> +	while ((c = getopt(argc, argv, "s:l:m:v")) != EOF) {
> +		switch (c) {
> +		case 's': /* starting point */
> +			opts->range->start = get_number(&optarg);
> +			break;
> +		case 'l': /* length */
> +			opts->range->len = get_number(&optarg);
> +			break;
> +		case 'm': /* minlen */
> +			opts->range->minlen = get_number(&optarg);
> +			break;
> +		case 'v': /* verbose */
> +			opts->verbose = 1;
> +			break;
> +		default:
> +			return EXIT_FAILURE;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +int main(int argc, char **argv)
> +{
> +	struct options *opts;
> +	struct stat sb;
> +	int fd, err = 0, ret = EXIT_FAILURE;
> +
> +	opts = malloc(sizeof(struct options));
> +	if (!opts)
> +		err_exit("%s: malloc(): %s\n", program_name, strerror(errno));
> +
> +	opts->range = NULL;
> +	opts->verbose = 0;
> +
> +	if (argc > 1)
> +		strncpy(opts->mpoint, argv[argc - 1], sizeof(opts->mpoint));
> +
> +	opts->range = calloc(1, sizeof(struct fstrim_range));
> +	if (!opts->range) {
> +		fprintf(stderr, "%s: calloc(): %s\n", program_name,
> +			strerror(errno));
> +		goto free_opts;
> +	}
> +
> +	opts->range->len = ULLONG_MAX;
> +
> +	if (argc > 2)
> +		err = parse_opts(argc, argv, opts);
> +
> +	if (err) {
> +		usage();
> +		goto free_opts;
> +	}
> +
> +	if (strnlen(opts->mpoint, 1) < 1) {
> +		fprintf(stderr, "%s: You have to specify mount point.\n",
> +			program_name);
> +		usage();
> +		goto free_opts;
> +	}
> +
> +	if (stat(opts->mpoint, &sb) == -1) {
> +		fprintf(stderr, "%s: %s: %s\n", program_name,
> +			opts->mpoint, strerror(errno));
> +		usage();
> +		goto free_opts;
> +	}
> +
> +	if (!S_ISDIR(sb.st_mode)) {
> +		fprintf(stderr, "%s: %s: (%s)\n", program_name,
> +			opts->mpoint, strerror(ENOTDIR));
> +		usage();
> +		goto free_opts;
> +	}
> +
> +	fd = open(opts->mpoint, O_RDONLY);
> +	if (fd < 0) {
> +		fprintf(stderr, "%s: open(%s): %s\n", program_name,
> +			opts->mpoint, strerror(errno));
> +		goto free_opts;
> +	}
> +
> +	if (ioctl(fd, FITRIM, opts->range)) {
> +		fprintf(stderr, "%s: FSTRIM: %s\n", program_name,
> +			strerror(errno));
> +		goto free_opts;
> +	}
> +
> +	if ((opts->verbose) && (opts->range))
> +		fprintf(stdout, "%llu Bytes was trimmed\n", opts->range->len);

"Bytes were trimmed," just FWIW

> +
> +	ret = EXIT_SUCCESS;
> +
> +free_opts:
> +	if (opts) {
> +		if (opts->range)
> +			free(opts->range);
> +		free(opts);
> +	}
> +
> +	return ret;
> +}

_______________________________________________
xfs mailing list
xfs@xxxxxxxxxxx
http://oss.sgi.com/mailman/listinfo/xfs


[Index of Archives]     [Linux XFS Devel]     [Linux Filesystem Development]     [Filesystem Testing]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux