On Wed, Jul 20, 2016 at 03:40:29PM +0800, Qu Wenruo wrote: > At 07/20/2016 03:01 PM, Eryu Guan wrote: > >On Tue, Jul 19, 2016 at 01:42:03PM +0800, Qu Wenruo wrote: > >>> > >>>This test uses $LOAD_FACTOR, so it should be in 'stress' group. And it > >>>hangs the latest kernel, stop other tests from running, I think we can > >>>add it to 'dangerous' group as well. > >>> > >> > >>Thanks for this info. > >>I'm completely OK to add this group to 'stress' and 'dangerous'. > >> > >> > >>However I'm a little curious about the meaning/standard of these groups. > >> > >>Does 'dangerous' conflicts with 'auto'? > >>Since under most case, tester would just execute './check -g auto' and the > >>system hangs at the test case. > >>So I'm a little confused with the 'auto' group. > > > >I quote my previous email here to explain the 'auto' group > >http://www.spinics.net/lists/fstests/msg03262.html > > > >" > >I searched for Dave's explainations on 'auto' group in his reviews, and > >got the following definitions: > > > >- it should be a valid & reliable test (it's finished and have > > deterministic output) [1] > >- it passes on current upstream kernels, if it fails, it's likely to be > > resolved in forseeable future [2] > >- it should take no longer than 5 minutes to finish [3] > >" > > > >And "The only difference between quick and auto group criteria is the > >test runtime." Usually 'quick' tests finish within 30s. > > > >For the 'dangerous' group, it was added in commit 3f28d55c3954 ("add > >freeze and dangerous groups"), and seems that it didn't have a very > >clear definition[*]. But I think any test that could hang/crash recent > >kernels is considered as dangerous. > > > >* http://oss.sgi.com/archives/xfs/2012-03/msg00073.html > > Thanks for all the info, really helps a lot. > > Especially for quick and auto difference. > > Would you mind me applying this standard to current btrfs test cases? It shoul dbe applied to all test cases, regardless of the filesystem type. > BTW, does the standard apply to *ALL* possible mount options or just > *deafult* mount option? Generally it applies to the default case. > For example, btrfs/011 can finish in about 5min with default mount > option, but for 'nodatasum' it can take up to 2 hours. > So should it belong to 'auto'? Yes. Also, keep in mind that runtime is dependent on the type of storage you are testing on. The general idea is that the "< 30s quick, < 5m auto" rule is based on how long the test takes to run on a local single spindle SATA drive, as that is the basic hardware we'd expect people to be testing against. This means that a test that takes 20s on your SSD might not be a "quick" test because it takes 5 minutes on spinning rust.... Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
