@vincentfu I'm in the same boat, not familiar with the features and caveats of the different options to make an informed recommendation at present. @sitsofe I agree, I think a `make unittest` that is incorporated into CI would be wonderful. Even better - if it generates a code/branch coverage report. Other projects I'm involved with require unit testing to provide code & branch coverage for any modified code before being merged. I think a similar strategy would be beneficial, however, I'm the least likely to be impacted by such a change in a process. At Nutanix different groups use different tools for performance testing and validation, and it depends on who you talk to. I've been a heavy Vdbench user (great tool), but am at a crossroads where I'm reevaluating tools for a future project. fio's open development process is very appealing to me. If time allows, I'll test drive a few unit testing frameworks and respond on thread w/ my branch. Jens, do you have any thoughts you can share? -bw C. Brantley West, III cbwest@xxxxxxxxx On Sat, Jun 3, 2017 at 9:29 AM, Tomohiro Kusumi <kusumi.tomohiro@xxxxxxxxx> wrote: > Hi > > 2017-06-03 16:07 GMT+03:00 Sitsofe Wheeler <sitsofe@xxxxxxxxx>: >> On 2 June 2017 at 20:30, Brantley West <cbwest@xxxxxxxxx> wrote: >>> >>> I'm looking to adopt fio for some storage testing in the near future >>> and have noticed a few issues (#377, #378) that seem to be >>> math-related. Assuming these issues get fixed, it seems like unit >>> testing would help ensure they stay fixed. Additionally, fio has a >>> lot of useful configuration options, and unit tests seem like a good >>> way to test for expected operation when multiple configuration >>> parameters are present. >>> >>> Is a unit test framework something that is worth considering for fio? >>> If so, are there any preferences? It seems there are multiple options >>> (check, cunit, unity, etc). >> >> I think it would be useful if there was an explicit make selftest >> target for fio which could do more rigorous checks than my "does it >> run?" make test. fio has the odd adhoc test (e.g. >> https://github.com/axboe/fio/tree/master/unit_tests ) and now I look I >> spy the odd test for checking certain job files for regressions too >> (see https://github.com/axboe/fio/tree/master/t/jobs ). It would be >> nice to run these and more via CI (travis/appveyor) on commit though >> (assuming they finish quickly)... >> >> Some pieces of fio have been broken out into standalone programs (see >> some of them in https://github.com/axboe/fio/tree/master/t/ ) which >> might make testing for correctness easier. Perhaps Tomohiro Kusumi >> (who has made pieces of fio more standalone) will be able to make >> suggestions? > > Yes. > Things under os/ arch/ lib/ oslib/ are mostly stand-alone utility > functions now (i.e. you can include them or link to them via your > main()), so those are probably easy to do unit testing using cunit or > variants. > > Others more or less have dependencies on fio itself (and there isn't > anything wrong with that too), so those are not as easy, or maybe > depends on what you mean by unit testing. > >> >> Personally I've used cunit (via the libiscsi project) but I've never >> compared all C unit test frameworks. Further, the fio build doesn't >> seem to add many dependencies on other libraries (other than for I/O >> engines)... >> >> Way back in 2008 there was a proposal to add a "make check" to fio >> (http://maillist.kernel.dk/fio-devel/0584.html ) which looked like it >> was readily accepted but it's unclear whether something was actually >> produced... >> >> If you've got any pre-existing checks please post/link them! >> >> PS: Do you find fio useful at Nutanix and are there any other public >> I/O exercisers you folks use? >> >> -- >> Sitsofe | http://sucs.org/~sits/ -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
