On Fri, Apr 24, 2020 at 1:24 AM Amir Goldstein <amir73il@xxxxxxxxx> wrote: > > On Fri, Apr 24, 2020 at 12:53 AM Arvind Raghavan > <raghavan.arvind@xxxxxxxxx> wrote: > > > > Hi Amir! > > > > We have 9 tests each consisting of several tests grouped around a > > single operation (rename, falloc, write, dwrite, mmapwrite, > > unlink, fsetxattr, removexattr, and truncate). The number of > > tests and runtime for each can be varied by changing how many > > flags and combinations of files we want to test. Some tests, such > > as the falloc test, have so many options that they could contain > > >100 tests and take several minutes. Is there an upper time bound > > that we should shoot for when building these tests? > > No there is no upper time bound. > It is just important to tag the tests correctly. > Tests in group quick usually run well under 1 minute (depending on > system and fs). > > Running all the tests in auto group could take several hours and each non > quick test could take several minutes, so it is not uncommon to add tests > that run several minutes. > > However, you should know that more testers run the quick tests and more often > that testers running non-quick tests. > > So the quick tests that you write will end up being run more often on more > filesystems by more testers Ah, got it! > > For example, the rename test contains 37 tests and takes around > > 12-15 seconds to run, which is around the same amount of time as > > Jayashree's initial link test took. I can send out a patch for > > that test if that runtime seems reasonable. > > That would be a good quick test. > > I think you should also consider tagging your tests with a group > more specific to crash consistency test. Great! It seems that most tests that use _flakey_drop_and_remount() are tagged with 'log' and 'metadata', and it seems like both of those tags fit here. We can go ahead and send the patch with the tags 'auto', 'quick', 'log', and 'metadata', and if we can agree on a crash consistency group (see below), we could retroactively tag those tests in a different patch. > I myself did not tag the _log_writes_replay_log tests I submitted > for a specific crash consistency group. I used the 'log' group > same as Jayashree's test. I think we can come up with a better > group name and tag those existing tests as well, probably some > others, but I have no suggestion for a good name, sorry. When Jayashree submitted the first Crashmonkey test, Felipe mentioned here [1] that the test "should be in the 'quick' and 'log' groups", and that he wasn't sure if it should be added to the new "crash" group. However, I can't seem to find any tests labelled with this 'crash' group. Was there a specific reason why this group would've been removed? [1] https://www.spinics.net/lists/fstests/msg10772.html > Thanks, > Amir.
