Hi! > There is a pretty large amount of overlap between LTP and xfstests, > and xfstests are what most of the file system developers are using, > and we have developed a lot of automated test automation which means > running xfstests is very easy and convenient. For example: > > https://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/README > > The ability for me to build a kernel and then with a single command, > "kvm-xfstests smoke", do a quick verification in about 30 minutes, is > very convenient. LTP is automated to a degree where you run single script and get a file with list of failed testcases later. We do not have any kind of kvm integration though. > As I recall, ltp was integrated with autotest, and my experience with > autotest at multiple companies is if anything, worse than ltp's > reputation. (I considered ltp to be mostly harmless, albeit not > particularly useful, whereas I considered autotest to be activetly > harmful to engineer productivity.) The autotest integration is not a part of the LTP at all. I remember seeing it somewhere but I've never used it/looked at the code. LTP has it's own script and testdriver to run testcases, but given that LTP tests are just binaries that are mostly self-contained it's not doing much more than starting a test, writing logfiles and killing lefover processes (if the tests fails to collect them itself). I will not pretend that it's clean and well designed code but at least it works fine (as a matter of fact I've started to work on redesigning/rewriting it from scratch some time ago). > Anyway, it's already the case that most of the useful file-system > specific bits of LTP has been cherry picked into xfstests, and I > suspect it will be a lot easier to get a few additional LTP test cases > added into xfstests, than it will be to convince a large number of > file system developers that they should (a) try to figure out how to > integrate LTP into their test harnesses, and (b) how to avoid > duplicating tests which xfstests are already running. Well I can personally help with (a). The test in question here (mmap_11-4) is a part of the Open Posix Testsuite that continues to live in LTP. The whole testsuite runs in about 30 minutes and covers most of the POSIX interfaces in ~1600 testcases. Then there is a syscalls testsuite which covers, in addition to the POSIX specs, some of the Linux specific interfaces too. The runtime is about 15 minutes for ~1030 testcases. I guess that we can filter filesystem related syscalls quite easily. The overlap would take more work though. In LTP we have mostly conformance testcases and some stress testcases. I'm not much familiar with xfstests and its coverage. And we have a more tests that may be interesting to fs maintaners, there are aio testcases (which are likely covered by xfstests allready), some fs stress tests, ... -- Cyril Hrubis chrubis@xxxxxxx -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
