> On Feb 23, 2023, at 10:19 AM, J. Bruce Fields <bfields@xxxxxxxxxxxx> wrote:
>
> On Wed, Feb 22, 2023 at 01:20:43PM -0500, Jeff Layton wrote:
>> The READMEs for v4.0 and v4.1 are inconsistent here. For v4.0, the "all"
>> flag is supposed to run all of the "standard" tests. For v4.1 "all" is
>> documented to run all of the tests, but it actually doesn't since not
>> every tests has "all" in its FLAGS: field.
>>
>> I move that we change this. If I say that I want to run "all", then I
>> really do want to run _all_ of the tests. Ensure that every test has the
>> "all" flag set.
>
> In some (all?) cases where the "all" flag was left off, it was
> intentional.
>
> We try not to flag spec-compliant servers as failing, because people are
> sometimes a little careless about "fixing" failures that in their
> particular case really shouldn't be fixed. But sometimes it's still
> useful to have a test that goes somewhat beyond the spec.
>
> There might be other ways to handle that kind of test, but it would need
> some more thought.
We could use a different name for "all" since it doesn't actually
run /all/ tests. Jeff suggested "standard", which seems sensible.
Also, we could add test categories specifically for particular server
implementations, if that's interesting to folks.
> --b.
>
>> Signed-off-by: Jeff Layton <jlayton@xxxxxxxxxx>
>> ---
>> nfs4.1/testmod.py | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> If this is unacceptable, then an alternative could be to add a new
>> (similarly special-cased) "everything" flag.
>>
>> diff --git a/nfs4.1/testmod.py b/nfs4.1/testmod.py
>> index 11e759d673fd..7b3bac543084 100644
>> --- a/nfs4.1/testmod.py
>> +++ b/nfs4.1/testmod.py
>> @@ -386,6 +386,8 @@ def createtests(testdir):
>> for t in tests:
>> ## if not t.flags_list:
>> ## raise RuntimeError("%s has no flags" % t.fullname)
>> + if "all" not in t.flags_list:
>> + t.flags_list.append("all")
>> for f in t.flags_list:
>> if f not in flag_dict:
>> flag_dict[f] = bit
>> --
>> 2.39.2
--
Chuck Lever
