> How long does each test case take to run? All the tests would touch 4 files at most or write 32-64KB of data to a file, starting from a empty file system image (hence very minimal running time). We are not going to bring CrashMonkey in the loop - we will port the generated tests to xfstest using dm-flakey (like generic/498 [1], which was submitted in response to a bug found by CrashMonkey in btrfs). Hence, each test should take the same time as that of current crash-consistency tests in the xfstest suite (for example, similar to generic/034 which takes about a second to run). > And note, by the way, that > by default we automatically run fsck on the test device after each > test. So number one, if you use the test device, you don't need to > worry about running fsck explicitly; the xfstests check script will do > that, and fail the test if the file system is corrupted --- and number > two, this will influence whether which groups each test should be > assigned. Noted. Since we will be writing the out-file (checker) manually, will ensure that checks only the content/metadata. > See the file xfstests-dev/tests/generic/group to see how groups get > assigned to tests. I suppose all of the crashmonkey tests should be > assigned to a new group, say, "crashmonkey". Whether or not they > should get assigned to the "auto" or "quick" group is a different > question. Note that if running these tests will signicantly increase > the test run time of smoke tests and even the full "automatic" > regression tests, there may be some resistence in adding all of these > tests to the "auto" or "quick" groups. Or even if you do, many file > system developers may choose to exclude all tests from the > "crashmonkey" group because if a 15 minute smoke test suddenly gets > extended to take 6 hours, developers are wont to get.... cranky. :-) It makes sense to add it to a new group as you suggest, and considering a second to run each test, it should take around 5 minutes to run this batch of CrashMonkey tests. Once the tests cases are ready, we can give you a better estimate of total time spent on the newly added tests. [1] https://patchwork.kernel.org/patch/10458749/ Thanks, Jayashree
