Hi Ralf,
Ralf Wildenhues wrote:
Hi Chris,
* Chris Pickett wrote on Mon, Mar 13, 2006 at 06:13:07PM CET:
I'd like to do unit testing for test-driven development of C programs.
I'd also like to use up-to-date autotools. I was wondering if anyone
had any experience with unit testing in their projects and had
recommendations, or even if autoconf was considering it for the future.
By unit testing I mean writing multiple small tests for each function in
an interface. The prescribed development methodology is to write
failing tests before you write code that will make them pass.
Autotest would be good as a wrapper around a unit-testing framework, but
not good for driving the actual low-level tests. The same goes for
DejaGNU.
Out of curiosity: why exactly do Autotest and DejaGNU not qualify for
this job? Which parts are missing?
I'm sorry, I should have explained it better. But first a disclaimer: I
only started thinking about unit testing on Friday, and I only came to
understand the basic autotools setup for starting a project from scratch
over the weekend, although in the past I have used autotools and m4 a
reasonable amount, but starting from other's work. So I may completely
wrong here.
Autotest and DejaGNU are great for "system testing" -- running compiled
programs and checking exit success, failure, the correctness of stdout
and stderr, handling of command line options, stuff like that.
However, there isn't any support for checking the correctness of
individual functions in a module. Ideally for each function in your
interface, you have several test functions. Then if you want to
refactor your interface implementation at a later date, you already have
a bunch of test functions to help reassure you that the refactoring
works. You have client code for each function from the get go.
The way a unit testing framework handles this is by making it easy to
create small test functions that exercise your real functions.
DejaGNU provides an API to such frameworks in dejagnu.h
The archetypal unit testing framework, xUnit, is described pretty
clearly here:
http://www.oreilly.com/catalog/unitest/chapter/ch03.pdf
It describes the following simple set of classes:
-- TestCase, a test class with several individual test methods
-- TestFixture, providing setUp() and tearDown() functions for tests
-- TestRunner, for running individual tests and reporting
speed, success
-- TestSuite, which can contain TestCases or even other TestSuites
-- TestResult, for accumulating test outcomes
Some of the additional features that I like in check, the C unit testing
framework I'm considering:
-- Tests can optionally be forked in their own address space
-- you can catch signals, expected or unexpected
-- you can stop early exits
-- you can prevent tests using the same setup data from
interfering with each other
-- You can have test timeouts
-- You can loop through a table of values with a looping test
-- Integration with gcov, to report code coverage (percentage of code
that your tests exercise)
So, why did I write here?
1) to find out if other people had gotten existing unit testing
frameworks to play nicely with autotools. Check actually was designed
around the 2.13 series, and although development has dwindled, it
probably could work pretty well. I was just hoping to hear about
other's experiences before committing to it.
-- Braden says he uses Boost Test
-- Ed says he doesn't bother with a heavyweight framework and
just writes his own API test code by hand
2) to find out if people had managed to do it cleanly using just
Autotest/DejaGNU
3) (failing 2) to suggest that making it easy to unit test your code
might make a good addition to autotools. I'm not sure about this,
writing something language-agnostic could be tricky.
I'm also interested in more detailed or "best practice" autotest
tutorials but the advice in the archives seems to be, "look at other
projects and existing tutorials and cobble something together."
FWIW, CVS M4 and CVS Libtool use Autotest.
I checked out M4 and Libtool and looked around. I didn't see anything
testing individual C functions (not really applicable for M4), but maybe
I missed it. For example, there's nothing that I saw testing the
functions in libltdl/ in Libtool, and nothing testing the .c files in
src/ in M4. Of course, they do get tested indirectly.
Cheers,
Chris
_______________________________________________
Autoconf mailing list
Autoconf@xxxxxxx
http://lists.gnu.org/mailman/listinfo/autoconf
