On Sat, Mar 21, 2015 at 08:02:54AM -0400, Jan Tulak wrote:
> Allowing the dot in test name has one important consequence - how to know what is a test and what is a test output?
> Now, I see it is a bit inconsistent, because it is possible to create such test but not run it, but that's a bug.
>
> If we don't want to touch existing tests, then I see two possibilities regarding this:
> - dot "." must not be part of a test name at all
> - string ".out" must not be part of test name (including .out.something, because there are files like generic/088.out.linux)
>
> I would say forbidding dot is cleaner approach. What do you think?
I agree, no dot in test name is easier and cleaner.
But you may also want to hear what do others think :)
>
>
> And I will look on sorting the non-numbered tests.
Thanks!
Eryu
>
> Cheers,
> Jan
>
>
> ----- Original Message -----
> > From: "Eryu Guan" <eguan@xxxxxxxxxx>
> > To: "Jan Ťulák" <jtulak@xxxxxxxxxx>
> > Cc: fstests@xxxxxxxxxxxxxxx
> > Sent: Saturday, 21 March, 2015 5:49:32 AM
> > Subject: Re: [PATCH] fstests: tests can use any name now, not 3 digits only.
> >
> > On Fri, Mar 20, 2015 at 04:03:25PM +0100, Jan Ťulák wrote:
> > > Tests can use any name now, not 3 digits only.
> > > (e.g. a test can be named "tests/generic/some-name")
> > >
> > > Signed-off-by: Jan Ťulák <jtulak@xxxxxxxxxx>
> > > ---
> > > README | 2 +-
> > > check | 11 ++++++-----
> > > new | 27 +++++++++++++++++++++++++--
> > > 3 files changed, 32 insertions(+), 8 deletions(-)
> > >
> > > diff --git a/README b/README
> > > index 0c9449a..2376674 100644
> > > --- a/README
> > > +++ b/README
> > > @@ -205,7 +205,7 @@ Test script environment:
> > >
> > > Verified output:
> > >
> > > - Each test script has a numerical name, e.g. 007, and an associated
> > > + Each test script has a name, e.g. 007, and an associated
> > > verified output, e.g. 007.out.
> > >
> > > It is important that the verified output is deterministic, and
> > > diff --git a/check b/check
> > > index 0830e0c..394fae4 100755
> > > --- a/check
> > > +++ b/check
> > > @@ -58,7 +58,7 @@ then
> > > exit 1
> > > fi
> > >
> > > -SUPPORTED_TESTS="[0-9][0-9][0-9] [0-9][0-9][0-9][0-9]"
> > > +SUPPORTED_TESTS="\S\+"
> > > SRC_GROUPS="generic shared"
> > > export SRC_DIR="tests"
> > >
> > > @@ -96,21 +96,22 @@ get_group_list()
> > > l=$(sed -n < $SRC_DIR/$d/group \
> > > -e 's/#.*//' \
> > > -e 's/$/ /' \
> > > - -e "s;\(^[0-9][0-9][0-9]\).* $grp .*;$SRC_DIR/$d/\1;p")
> > > + -e "s;^\($SUPPORTED_TESTS\).* $grp .*;$SRC_DIR/$d/\1;p")
> > > grpl="$grpl $l"
> > > done
> > > echo $grpl
> > > }
> > >
> > > -# find all tests, excluding files that are test metadata such as group
> > > files.
> > > -# This assumes that tests are defined purely by alphanumeric filenames
> > > with no
> > > -# ".xyz" extensions in the name.
> > > +# Find all tests, excluding files that are test metadata such as group
> > > files.
> > > +# It matches test names against $SUPPORTED_TESTS defined at the top of
> > > this
> >
> > Trailing whitespace in above line
> >
> > > +# file.
> > > get_all_tests()
> > > {
> > > touch $tmp.list
> > > for d in $SRC_GROUPS $FSTYP; do
> > > ls $SRC_DIR/$d/* | \
> > > grep -v "\..*" | \
> > > + grep "^$SRC_DIR/$d/$SUPPORTED_TESTS"| \
> >
> > I just found that this will skip all tests with dot in the name, if
> > there's a test has dot in the name(e.g. some-name.test) it won't be run
> > when ./check with no other arguments. "./check -n" could show all tests
> > to be run, and "some-name.test" is not in the list.
> >
> > Or "." should not be allowed in test name?
> >
> > > grep -v "group\|Makefile" >> $tmp.list 2>/dev/null
> > > done
> > > }
> > > diff --git a/new b/new
> > > index d1f8939..60b898a 100755
> > > --- a/new
> > > +++ b/new
> > > @@ -84,8 +84,11 @@ eof=1
> > > for found in `cat $tdir/group | $AWK_PROG '{ print $1 }'`
> > > do
> > > line=$((line+1))
> > > - if [ -z "$found" ] || [ "$found" == "#" ];then
> > > - continue
> > > + if [ -z "$found" ] || [ "$found" == "#" ]; then
> > > + continue
> > > + elif ! echo "$found" | grep "^[0-9][0-9][0-9]$" > /dev/null; then
> >
> > grep -q to be quiet
> >
> > > + # this one is for tests not named by a number
> > > + continue
> > > fi
> > > i=$((i+1))
> > > id=`printf "%03d" $i`
> > > @@ -102,6 +105,26 @@ fi
> > >
> > > echo "Next test is $id"
> > >
> > > +read -p "Do you want to use ANOTHER name? y,[n]: " -r
> > > +if [[ "$REPLY" =~ ^[Yy]$ ]]; then
> > > + id=""
> > > + while [ "$id" = "" ]; do
> > > + read -p "Enter the new name: "
> > > + if [ "$REPLY" = "" ]; then
> > > + echo "Can't use empty name. For canceling, use ctrl+c."
> > > + elif [ -e "$tdir/$REPLY" ]; then
> > > + echo "File '$REPLY' already exists, use another one."
> > > + echo #
> > > + elif echo "$REPLY" | grep "^\\S\+$" > /dev/null; then
> >
> > here too, grep -q
> >
> > > + id="$REPLY"
> > > + else
> > > + echo "Filename must not contain whitespaces!"
> > > + echo
> > > + fi
> > > + done
> > > +fi
> > > +echo "Using '$id'."
> > > +
> >
> > Now new case with customized name is inserted in numbered tests, as
> >
> > eguan@dhcp-13-216:~/workspace/src/xfstests$ git diff
> > diff --git a/tests/generic/group b/tests/generic/group
> > index 8154401..e409035 100644
> > --- a/tests/generic/group
> > +++ b/tests/generic/group
> > @@ -69,6 +69,10 @@
> > 064 auto quick prealloc
> > 065 metadata auto quick
> > 066 metadata auto quick
> > +some-test.001 other
> > +some-test-002 other
> > +some-test-003 other
> > +hello! other
> > 068 other auto freeze dangerous stress
> > 069 rw udf auto quick
> > 070 attr udf auto quick stress
> >
> > I'd like to see all customized test names are at the end(or beginning)
> > of group file, sorted if possible.
> >
> > Maybe I'm asking too much, but this is what I'd like to see :)
> >
> > And the test case template in "new" can be updated too, I think, we
> > don't need "No." anymore
> >
> > # FS QA Test No. $id
> >
> > Thanks,
> > Eryu
> > > if [ -f $tdir/$id ]
> > > then
> > > echo "Error: test $id already exists!"
> > > --
> > > 2.1.0
> > >
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe fstests" in
> > > the body of a message to majordomo@xxxxxxxxxxxxxxx
> > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > --
> > To unsubscribe from this list: send the line "unsubscribe fstests" in
> > the body of a message to majordomo@xxxxxxxxxxxxxxx
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> >
--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
