Am Tue, 14 Mar 2006 11:32:31 -0500 schrieb Chris Pickett <chris.pickett@xxxxxxxxxxxxxx>: > [[ Sorry, I originally sent this from an unsubscribed account ]] > > -------- Original Message -------- > Subject: Re: unit testing with autotools > Date: Tue, 14 Mar 2006 11:25:06 -0500 > From: Chris Pickett <cpicke@xxxxxxxxxxxx> > Reply-To: chris.pickett@xxxxxxxxxxxxxx > To: Ralf Wildenhues <Ralf.Wildenhues@xxxxxx> > CC: autoconf@xxxxxxx > References: <4415A823.2000307@xxxxxxxxxxxxxx> > <20060314125150.GE28722@xxxxxxxxxxxxxxx> > > Hi Ralf, > > Ralf Wildenhues wrote: > > Hi Chris, > > > > * Chris Pickett wrote on Mon, Mar 13, 2006 at 06:13:07PM CET: > > > >>I'd like to do unit testing for test-driven development of C > >>programs. I'd also like to use up-to-date autotools. I was > >>wondering if anyone had any experience with unit testing in their > >>projects and had recommendations, or even if autoconf was > >>considering it for the future. > >> > >>By unit testing I mean writing multiple small tests for each > >>function in an interface. The prescribed development methodology > >>is to write failing tests before you write code that will make them > >>pass. > >> > >>Autotest would be good as a wrapper around a unit-testing > >>framework, but not good for driving the actual low-level tests. > >>The same goes for DejaGNU. > > > > > > Out of curiosity: why exactly do Autotest and DejaGNU not qualify > > for this job? Which parts are missing? > > I'm sorry, I should have explained it better. But first a > disclaimer: I only started thinking about unit testing on Friday, and > I only came to understand the basic autotools setup for starting a > project from scratch over the weekend, although in the past I have > used autotools and m4 a reasonable amount, but starting from other's > work. So I may completely wrong here. > > Autotest and DejaGNU are great for "system testing" -- running > compiled programs and checking exit success, failure, the correctness > of stdout and stderr, handling of command line options, stuff like > that. > > However, there isn't any support for checking the correctness of > individual functions in a module. Ideally for each function in your > interface, you have several test functions. Then if you want to > refactor your interface implementation at a later date, you already > have a bunch of test functions to help reassure you that the > refactoring works. You have client code for each function from the > get go. > > The way a unit testing framework handles this is by making it easy to > create small test functions that exercise your real functions. > DejaGNU provides an API to such frameworks in dejagnu.h > > The archetypal unit testing framework, xUnit, is described pretty > clearly here: > > http://www.oreilly.com/catalog/unitest/chapter/ch03.pdf > > It describes the following simple set of classes: > > -- TestCase, a test class with several individual test methods > -- TestFixture, providing setUp() and tearDown() functions for > tests -- TestRunner, for running individual tests and reporting > speed, success > -- TestSuite, which can contain TestCases or even other TestSuites > -- TestResult, for accumulating test outcomes > > Some of the additional features that I like in check, the C unit > testing framework I'm considering: > > -- Tests can optionally be forked in their own address space > -- you can catch signals, expected or unexpected > -- you can stop early exits > -- you can prevent tests using the same setup data from > interfering with each other > -- You can have test timeouts > -- You can loop through a table of values with a looping test > -- Integration with gcov, to report code coverage (percentage of > code that your tests exercise) > > So, why did I write here? > > 1) to find out if other people had gotten existing unit testing > frameworks to play nicely with autotools. Check actually was designed > around the 2.13 series, and although development has dwindled, it > probably could work pretty well. I was just hoping to hear about > other's experiences before committing to it. > -- Braden says he uses Boost Test > -- Ed says he doesn't bother with a heavyweight framework and > just writes his own API test code by hand > > 2) to find out if people had managed to do it cleanly using just > Autotest/DejaGNU > > 3) (failing 2) to suggest that making it easy to unit test your code > might make a good addition to autotools. I'm not sure about this, > writing something language-agnostic could be tricky. > > >>I'm also interested in more detailed or "best practice" autotest > >>tutorials but the advice in the archives seems to be, "look at > >>other projects and existing tutorials and cobble something > >>together." > > > > > > FWIW, CVS M4 and CVS Libtool use Autotest. > > I checked out M4 and Libtool and looked around. I didn't see anything > testing individual C functions (not really applicable for M4), but > maybe I missed it. For example, there's nothing that I saw testing > the functions in libltdl/ in Libtool, and nothing testing the .c > files in src/ in M4. Of course, they do get tested indirectly. > We use cppunit (http://cppunit.sourceforge.net/cppunit-wiki/FrontPage) together with autotools. Thomas -- _______________________________________________ Autoconf mailing list Autoconf@xxxxxxx http://lists.gnu.org/mailman/listinfo/autoconf