On Wed, Jul 13, 2005 at 09:58:56AM -0700, Mary Edie Meredith wrote: > On Wed, 2005-07-13 at 09:38 -0700, Bryce Harrington wrote: > > On Wed, Jul 13, 2005 at 09:13:52AM -0700, Mark Delcambre wrote: > > > On Wednesday 13 July 2005 08:47 am, Mary Edie Meredith wrote: > > > > To make these test cases easy to implement and use, could you > > > > please make sure the scripts have comments that state the goal > > > > of the test case, if you haven't already? Sorry, I should have > > > > thought of this when we went over the Psuedo code. > > > > Yes, we should have them print out a couple sentences explaining the > > purpose. Do you think we can just take the description that you wrote > > on your test case summary page? > > Yes, the purpose from the test case summary page is good. If that > does'nt match somehow, we need to correct the web page. > > If you are exporting the number of loops, then you can print > it once at the beginning right? Otherwise, that's a lot of > extra unnecessary output. Yes, that's a good approach. I think this should fit in well with how Mark's implemented the looping. > Speaking of output, maybe we need a flag that lets you only report > failures? I'm torn on this one, because you might want to see where > the failure fits in to other successes and if you don't capture it > on a long run, you have to rerun. Of course you can always > turn the flag off... This isn't a big thing for someone to add, but > it would be nice if it was consistent across the scripts. > Your thoughts? Well, remember we can always just `grep FAIL` from the output. ;-) In fact, that's standard procedure for processing the output from these tests. ;-) This is why we've taken care to make sure the tests print out PASS/FAIL in a standardized way, so we can use grep and get the info needed to debug it (the name of the failed test, plus the error message). In general I feel that anything that we could do easily enough with standard unix tools, we should leave out of the test case scripts, and let them focus on just doing their specific test as well as they can. > > I think the wrapper script should be pretty straightforward; it should > > just define the number of loops to run, then invoke all of the > > hotplug*.sh tests in the directory. The little for loop I wrote > > yesterday could be used as a starting point. > Easy, I know, but an accompanying loop that has reasonable > defaults lets someone run this quickly without very much > investigation. That's one goal. Second is to run it by adding > repetitions and background activity, but I doubt we can > package that -- document how to, certainly. Agreed, sounds good. Have you given more thought to test case 5 and the workload to use for it? Maybe next week you and Mark can put your heads together on it. Bryce
