On Sun, Apr 12, 2020 at 10:44:19PM -0700, ira.weiny@xxxxxxxxx wrote: > From: Ira Weiny <ira.weiny@xxxxxxxxx> > > Add XXX to test per file DAX operations. > > The following is tested[*] > > 1. There exists an in-kernel access mode flag S_DAX that is set when > file accesses go directly to persistent memory, bypassing the page > cache. Applications must call statx to discover the current S_DAX > state (STATX_ATTR_DAX). > > 2. There exists an advisory file inode flag FS_XFLAG_DAX that is > inherited from the parent directory FS_XFLAG_DAX inode flag at file > creation time. This advisory flag can be set or cleared at any > time, but doing so does not immediately affect the S_DAX state. > > Unless overridden by mount options (see (3)), if FS_XFLAG_DAX is set > and the fs is on pmem then it will enable S_DAX at inode load time; > if FS_XFLAG_DAX is not set, it will not enable S_DAX. > > 3. There exists a dax= mount option. > > "-o dax=never" means "never set S_DAX, ignore FS_XFLAG_DAX." > > "-o dax=always" means "always set S_DAX (at least on pmem), > and ignore FS_XFLAG_DAX." > > "-o dax" is an alias for "dax=always". > > "-o dax=inode" means "follow FS_XFLAG_DAX" and is the default. > > 4. There exists an advisory directory inode flag FS_XFLAG_DAX that can > be set or cleared at any time. The flag state is copied into any > files or subdirectories when they are created within that directory. > > 5. Programs that require a specific file access mode (DAX or not DAX) > can do one of the following: > > (a) Create files in directories that the FS_XFLAG_DAX flag set as > needed; or > > (b) Have the administrator set an override via mount option; or > > (c) Set or clear the file's FS_XFLAG_DAX flag as needed. Programs > must then cause the kernel to evict the inode from memory. This > can be done by: > > i> Closing the file and re-opening the file and using statx to > see if the fs has changed the S_DAX flag; and > > ii> If the file still does not have the desired S_DAX access > mode, either unmount and remount the filesystem, or close > the file and use drop_caches. > > 6. It's not unreasonable that users who want to squeeze every last bit > of performance out of the particular rough and tumble bits of their > storage also be exposed to the difficulties of what happens when the > operating system can't totally virtualize those hardware > capabilities. Your high performance sports car is not a Toyota > minivan, as it were. > > [*] https://lore.kernel.org/lkml/20200409165927.GD6741@magnolia/ > > Signed-off-by: Ira Weiny <ira.weiny@xxxxxxxxx> > > --- > Changes from v6 (kernel patch set): > Start versions tracking the kernel patch set. > Update for new requirements > > Changes from V1 (xfstests patch): > Add test to ensure moved files preserve their flag > Check chattr of non-dax flags (check bug found by Darrick) > --- > tests/xfs/999 | 272 ++++++++++++++++++++++++++++++++++++++++++++++ > tests/xfs/999.out | 22 ++++ > tests/xfs/group | 1 + > 3 files changed, 295 insertions(+) > create mode 100755 tests/xfs/999 > create mode 100644 tests/xfs/999.out > > diff --git a/tests/xfs/999 b/tests/xfs/999 > new file mode 100755 > index 000000000000..3ca04b98c517 > --- /dev/null > +++ b/tests/xfs/999 > @@ -0,0 +1,272 @@ > +#! /bin/bash > +# SPDX-License-Identifier: GPL-2.0 > +# Copyright (c) 2019 Intel, Corp. All Rights Reserved. > +# > +# FSQA Test No. 999 (temporary) > +# > +# Test setting of DAX flag > +# > +seq=`basename $0` > +seqres=$RESULT_DIR/$seq > +echo "QA output created by $seq" > + > +here=`pwd` > +status=1 # failure is the default! > + > +dax_dir=$TEST_DIR/dax-dir > +dax_sub_dir=$TEST_DIR/dax-dir/dax-sub-dir > +dax_inh_file=$dax_dir/dax-inh-file > +dax_non_inh_file=$dax_dir/dax-non-inh-file > +non_dax=$TEST_DIR/non-dax > +dax_file=$TEST_DIR/dax-dir/dax-file > +dax_file_copy=$TEST_DIR/dax-file-copy > +dax_file_move=$TEST_DIR/dax-file-move > +data_file=$TEST_DIR/non-dax-dir/data-file > + > +_cleanup() { > + rm -rf $TEST_DIR/* > +} > + > +trap "_cleanup ; exit \$status" 0 1 2 3 15 > + > +# get standard environment, filters and checks > +. ./common/rc > + > +# real QA test starts here > +_supported_os Linux > +_require_xfs_io_command "lsattr" > +_require_xfs_io_command "statx" > +_require_test This might be a good time to introduce a few new helpers: _require_scratch_dax ("Does $SCRATCH_DEV support DAX?") _require_scratch_dax_mountopt ("Does the fs support the DAX mount options?") _require_scratch_daX_iflag ("Does the fs support FS_XFLAG_DAX?") Also, I think this should be split into two tests. One test to check the correct operation of FS_XFLAG_DAX, because the flag has behaviors that always apply even if the test/scratch devices are not pmem; and a second test to check the S_DAX operation if the system under test actually has pmem. > + > +# > +# mnt_opt's we expect > +# '' > +# '-o dax=off' > +# '-o dax=inode' > +# '-o dax' > +# '-o dax=always' > +function remount_w_option { > + mnt_opt=$1 > + export MOUNT_OPTIONS="" > + export TEST_FS_MOUNT_OPTS="" > + _test_unmount &> /dev/null > + _test_mount $mnt_opt &> /dev/null > +} > + > +function check_dax_mount_option { > + mnt_opt=$1 > + _fs_options $TEST_DEV | grep -qw '$mnt_opt' > + if [ "$?" == "0" ]; then > + echo "FAILED: to mount FS with option '$mnt_opt'" > + fi > +} > + > +function check_xflag_dax { > + xfs_io -c 'lsattr' $1 | awk -e '{ print $1 }' | grep 'x' &> /dev/null s/xfs_io/$XFS_IO_PROG/g > + if [ "$?" != "0" ]; then > + echo "FAILED: Did NOT find FS_XFLAG_DAX on $1" > + fi > +} > + > +function check_s_dax { > + attr=`xfs_io -c 'statx -r' $1 | grep 'stat.attributes' | awk -e '{ print $3 }'` Also, the statx bits mean this test needs: _require_xfs_io_command statx -r So that we skip the test on old userspace. --D > + masked=$(( $attr & 0x2000 )) > + if [ "$masked" != "8192" ]; then > + echo "FAILED: Did NOT find S_DAX flag on $1" > + fi > +} > + > +function check_no_xflag_dax { > + xfs_io -c 'lsattr' $1 | awk -e '{ print $1 }' | grep 'x' &> /dev/null > + if [ "$?" == "0" ]; then > + echo "FAILED: Found FS_XFLAG_DAX on $1" > + fi > +} > + > +function check_no_s_dax { > + attr=`xfs_io -c 'statx -r' $1 | grep 'stat.attributes' | awk -e '{ print $3 }'` > + masked=$(( $attr & 0x2000 )) > + if [ "$masked" == "8192" ]; then > + echo "FAILED: Found S_DAX flag on $1" > + fi > +} > + > +echo "running tests..." > + > +remount_w_option "" > +check_dax_mount_option "dax=inode" > + > +echo " *** mark dax-dir as dax enabled" > +mkdir $dax_dir > +xfs_io -c 'chattr +x' $dax_dir > +check_xflag_dax $dax_dir > + > +echo " *** check file inheritance" > +touch $dax_inh_file > +check_xflag_dax $dax_inh_file > +check_s_dax $dax_inh_file > + > +echo " *** check directory inheritance" > +mkdir $dax_sub_dir > +check_xflag_dax $dax_sub_dir > + > +echo " *** check changing directory" > +xfs_io -c 'chattr -x' $dax_dir > +check_no_xflag_dax $dax_dir > +check_no_s_dax $dax_dir > + > +echo " *** check non file inheritance" > +touch $dax_non_inh_file > +check_no_xflag_dax $dax_non_inh_file > +check_no_s_dax $dax_non_inh_file > + > +echo " *** check that previous file stays enabled" > +check_xflag_dax $dax_inh_file > +check_s_dax $dax_inh_file > + > +echo " *** Reset the directory" > +xfs_io -c 'chattr +x' $dax_dir > +check_xflag_dax $dax_dir > + > +# Set up for next test > +touch $dax_file > +touch $non_dax > + > +# > +# inode flag > +# ./ > +# + dax-dir/ X > +# + dax-sub-dir/ X > +# + dax-inh-file X > +# + dax-non-inh-file > +# + dax-file X > +# + non-dax > +# > + > +# check mount overrides > +# ===================== > + > +echo " *** Check '-o dax'" > +remount_w_option "-o dax" > +check_dax_mount_option "dax=always" > + > +echo " *** non-dax inode but overrides to be effective" > +check_no_xflag_dax $non_dax > +check_s_dax $non_dax > + > +echo " *** Check for non-dax inode to be dax with mount option" > +check_no_xflag_dax $dax_non_inh_file > +check_s_dax $dax_non_inh_file > + > + > +echo " *** Check '-o dax=never'" > +remount_w_option "-o dax=never" > +check_dax_mount_option "dax=never" > + > +check_xflag_dax $dax_dir > +check_xflag_dax $dax_sub_dir > +check_xflag_dax $dax_inh_file > +check_no_s_dax $dax_inh_file > +check_no_xflag_dax $dax_non_inh_file > +check_no_s_dax $dax_non_inh_file > +check_no_xflag_dax $non_dax > +check_no_s_dax $non_dax > +check_xflag_dax $dax_file > +check_no_s_dax $dax_file > + > + > +echo " *** Check '-o dax=inode'" > +remount_w_option "-o dax=inode" > +check_dax_mount_option "dax=inode" > + > +check_xflag_dax $dax_dir > +check_xflag_dax $dax_sub_dir > +check_xflag_dax $dax_inh_file > +check_s_dax $dax_inh_file > +check_no_xflag_dax $dax_non_inh_file > +check_no_s_dax $dax_non_inh_file > +check_no_xflag_dax $non_dax > +check_no_s_dax $non_dax > +check_xflag_dax $dax_file > +check_s_dax $dax_file > + > + > +# Check non-zero file operations > +# ============================== > + > +echo " *** Verify changing FS_XFLAG_DAX flag" > + > +mkdir $TEST_DIR/non-dax-dir > +$here/ltp/fsx $options -N 20000 $data_file > $tmp.log 2>&1 & > +pid=$! > +check_no_xflag_dax $data_file > +check_no_s_dax $data_file > + > +if [ ! -n "$pid" ]; then > + echo "FAILED to start fsx" > +else > + # toggle inode flag back and forth. > + # s_dax should not change while fsx is using file. > + xfs_io -c 'chattr +x' $data_file &> /dev/null > + check_xflag_dax $data_file > + check_no_s_dax $data_file > + xfs_io -c 'chattr -x' $data_file &> /dev/null > + check_no_xflag_dax $data_file > + check_no_s_dax $data_file > + xfs_io -c 'chattr +x' $data_file &> /dev/null > + check_xflag_dax $data_file > + check_no_s_dax $data_file > +fi > + > +wait $pid > +status=$? > +if [ "$status" != "0" ]; then > + echo "FAILED: fsx exited with status : $status" > + head $dax_file.fsxlog > +fi > +pid="" > + > +echo 2 > /proc/sys/vm/drop_caches > + > +echo " *** Verify S_DAX change on drop_caches" > +# s_dax should change after drop_caches > +check_xflag_dax $data_file > +check_s_dax $data_file > + > + > +# Check inheritance on cp, mv > +# =========================== > + > +echo " *** check making 'data' dax with cp" > +cp $non_dax $dax_dir/conv-dax > +check_xflag_dax $dax_dir/conv-dax > +check_s_dax $dax_dir/conv-dax > + > +echo " *** check making 'data' non-dax with cp" > +rm -f $data_file > +cp $dax_dir/conv-dax $data_file > +check_no_xflag_dax $data_file > +check_no_s_dax $data_file > + > +echo " *** Moved files 'don't inherit'" > +mv $non_dax $dax_dir/move-dax > +check_no_xflag_dax $dax_dir/move-dax > +check_no_s_dax $dax_dir/move-dax > + > +echo " *** Moved files preserve their flag" > +mv $dax_file $TEST_DIR/move-dax > +check_xflag_dax $TEST_DIR/move-dax > +check_s_dax $TEST_DIR/move-dax > + > +echo " *** Check file chattr of non-dax flags" > +xfs_io -c 'chattr +s' $dax_inh_file > +xfs_io -c 'chattr -s' $dax_inh_file > + > +echo " *** Check '-o dax=garbage'" > +remount_w_option "-o dax=garbage" > +_check_mounted_on TEST_DEV $TEST_DEV TEST_DIR $TEST_DIR > +if [ "$?" == "0" ]; then > + echo "ERROR: fs mounted with '-o dax=garbage'" > +fi > + > +status=0 ; exit > diff --git a/tests/xfs/999.out b/tests/xfs/999.out > new file mode 100644 > index 000000000000..3a4f970a5073 > --- /dev/null > +++ b/tests/xfs/999.out > @@ -0,0 +1,22 @@ > +QA output created by 999 > +running tests... > + *** mark dax-dir as dax enabled > + *** check file inheritance > + *** check directory inheritance > + *** check changing directory > + *** check non file inheritance > + *** check that previous file stays enabled > + *** Reset the directory > + *** Check '-o dax' > + *** non-dax inode but overrides to be effective > + *** Check for non-dax inode to be dax with mount option > + *** Check '-o dax=never' > + *** Check '-o dax=inode' > + *** Verify changing FS_XFLAG_DAX flag > + *** Verify S_DAX change on drop_caches > + *** check making 'data' dax with cp > + *** check making 'data' non-dax with cp > + *** Moved files 'don't inherit' > + *** Moved files preserve their flag > + *** Check file chattr of non-dax flags > + *** Check '-o dax=garbage' > diff --git a/tests/xfs/group b/tests/xfs/group > index 522d4bc44d1f..816883a268bf 100644 > --- a/tests/xfs/group > +++ b/tests/xfs/group > @@ -511,3 +511,4 @@ > 511 auto quick quota > 512 auto quick acl attr > 513 auto mount > +999 auto > -- > 2.25.1 >
