On 5/21/14, 4:20 PM, chrubis@xxxxxxx wrote: > Hi! >> There is a pretty large amount of overlap between LTP and xfstests, >> and xfstests are what most of the file system developers are using, >> and we have developed a lot of automated test automation which means >> running xfstests is very easy and convenient. For example: >> >> https://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/README >> >> The ability for me to build a kernel and then with a single command, >> "kvm-xfstests smoke", do a quick verification in about 30 minutes, is >> very convenient. > > LTP is automated to a degree where you run single script and get a file > with list of failed testcases later. We do not have any kind of kvm > integration though. > >> As I recall, ltp was integrated with autotest, and my experience with >> autotest at multiple companies is if anything, worse than ltp's >> reputation. (I considered ltp to be mostly harmless, albeit not >> particularly useful, whereas I considered autotest to be activetly >> harmful to engineer productivity.) > > The autotest integration is not a part of the LTP at all. I remember > seeing it somewhere but I've never used it/looked at the code. > > LTP has it's own script and testdriver to run testcases, but given that > LTP tests are just binaries that are mostly self-contained it's not > doing much more than starting a test, writing logfiles and killing > lefover processes (if the tests fails to collect them itself). I will > not pretend that it's clean and well designed code but at least it works > fine (as a matter of fact I've started to work on redesigning/rewriting > it from scratch some time ago). > >> Anyway, it's already the case that most of the useful file-system >> specific bits of LTP has been cherry picked into xfstests, and I >> suspect it will be a lot easier to get a few additional LTP test cases >> added into xfstests, than it will be to convince a large number of >> file system developers that they should (a) try to figure out how to >> integrate LTP into their test harnesses, and (b) how to avoid >> duplicating tests which xfstests are already running. > > Well I can personally help with (a). > > The test in question here (mmap_11-4) is a part of the Open Posix > Testsuite that continues to live in LTP. The whole testsuite runs in > about 30 minutes and covers most of the POSIX interfaces in ~1600 > testcases. > > Then there is a syscalls testsuite which covers, in addition to the > POSIX specs, some of the Linux specific interfaces too. The runtime is > about 15 minutes for ~1030 testcases. > > I guess that we can filter filesystem related syscalls quite easily. The > overlap would take more work though. In LTP we have mostly conformance > testcases and some stress testcases. I'm not much familiar with xfstests > and its coverage. > > And we have a more tests that may be interesting to fs maintaners, there > are aio testcases (which are likely covered by xfstests allready), some > fs stress tests, ... > FWIW, I don't think there's any real compelling reason to migrate existing LTP tests into xfstests. Maybe LTP folks just need to do a better job of pitching LTP to filesystem developers, as we did with xfstests. :) -Eric -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
