Dave,
On Tue, Nov 24, 2015 at 12:08 AM, Dave Chinner <david@xxxxxxxxxxxxx> wrote:
> On Wed, Nov 18, 2015 at 03:17:48PM +0100, Andreas Gruenbacher wrote:
[...]
>> +ncheck "touch x"
>> +ncheck "setrichacl --set 'owner@:rwp::allow group@:rwp::allow everyone@:r::allow' x"
>> +check "getrichacl x" <<EOF
>> +x:
>> + owner@:rwp----------::allow
>> + group@:rwp----------::allow
>> + everyone@:r------------::allow
>> +EOF
>
> Ok, let's break this down, because most of the tests are similar.
>
> ncheck is not needed, as golden output matching will catch an
> unexpected error.
>
> check is not needed, because the output of the command should
> match what is in the golden output file. i.e, this test should be
> simply:
>
> touch x
> setrichacl --set 'owner@:rwp::allow group@:rwp::allow everyone@:r::allow' x
> getrichacl x
>
> And the golden output file should contain:
>
> x:
> owner@:rwp----------::allow
> group@:rwp----------::allow
> everyone@:r------------::allow
>
> The fact that you do golden output matching directly in the script
> to calculate "checks_succeeded" and "checks_failed" and then check
> those at the end of the test to set the exit status is completely
> unnecessary. The xfstests tsts harness does all this pattern
> matching for you via the golden output file. Using output files
> correctly make patches 3 and 4 in this series go away.
This is really not where I want to go with those tests. I'm not using
golden output files on full purpose; I really do want the output and
exit status of each command under test to be checked individually. I
also do want proper logging and error reporting --- Which commands
from where in the test script were executed with which parameters?
Were they successful and did they produce the expected output? If not,
what's the difference between the expected and actual output?
With xfstest output matching, I get none of that reliably. I don't
even get to know which exact commands are executed; parameters which
differ from run to run are masked on purpose (grepping for "sed" in
xfstests shows some examples). When a test fails, I have to start
guessing which command might have caused the failure; the tests are
peppered with echo "Here I am" statements to make that possible at
all.
For example, the above test produces the following log:
[8] touch x -- ok
[9] setrichacl --set 'owner@:rwp::allow group@:rwp::allow
everyone@:r::allow' x -- ok
[10] getrichacl x -- ok
[...]
46 tests (46 passed, 0 failed)
A failure would come out as follows (with exit status 1):
[8] touch x -- ok
[9] setrichacl --set 'owner@:rwp::allow group@:rwp::allow
everyone@:r::allow' x -- ok
[10] getrichacl x -- FAILED
--- expected
+++ got
@@ -1,4 +1,4 @@
x:
- owner@:rwp----------::allow
- group@:rwp----------::allow
+ owner@:r------------::allow
+ group@:r------------::allow
everyone@:r------------::allow
[...]
46 tests (45 passed, 1 failed)
I also do want these tests to remain functional inside the richacl
package, where they are coming from.
Thanks,
Andreas
_______________________________________________
xfs mailing list
xfs@xxxxxxxxxxx
http://oss.sgi.com/mailman/listinfo/xfs
