Re: [PATCH] test-lib: save test counts across invocations

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, Sep 01, 2011 at 02:38:54PM -0400, Alex Vandiver wrote:

> On Thu, 2011-09-01 at 12:38 -0400, Jeff King wrote:
> > Hmm. What happens when we're wrong? Does our eye-candy just print
> > something non-sensical like "13/12", or does prove actually care that we
> > run the right number of tests?
> 
> prove very much does care -- having a mismatch between the number of
> tests planned and the number of tests run is an error in the testfile,
> and is reported as such in big red text.  This is because stating how
> many tests you plan to run gives prove a way (in addition to the exit
> status) to know if the test stopped prematurely, so all mismatches
> between plan and actual test counts are reported as testfile failures.

Thanks. I suspected something like that, but was too lazy to look. :)

Given that our methods for automatically determining the number of tests
are so flaky, and that prove will treat it so seriously, it doesn't seem
worth pursuing to me.

We already handle the premature abort case by trapping exit from the
shell before the script calls test_done. So I don't think that is a
feature of prove that we particularly care about.

> As far as I know prove doesn't have a way to print the estimated time
> remaining, though using the contents of the .prove file (if you ran
> prove --state=save) to guess it wouldn't be all that hard of a change.

That would be a neat feature. In practice, I know about how many tests
there are total (~7500), and how long it takes to run on my system (~60
seconds), so I can do the math myself. Still, a little more eye candy
couldn't hurt. ;)

If I underestand the code correctly, we could even write our own custom
"formatter" for git and use it via "prove --formatter".

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]