Resending thread explicitly to Jens (thread may be difficult to read because we have both top and bottom posting)... On 7 June 2017 at 21:52, Brantley West <cbwest@xxxxxxxxx> wrote: > @vincentfu I'm in the same boat, not familiar with the features and > caveats of the different options to make an informed recommendation at > present. > > @sitsofe I agree, I think a `make unittest` that is incorporated into > CI would be wonderful. Even better - if it generates a code/branch > coverage report. Other projects I'm involved with require unit > testing to provide code & branch coverage for any modified code before > being merged. I think a similar strategy would be beneficial, > however, I'm the least likely to be impacted by such a change in a > process. At Nutanix different groups use different tools for > performance testing and validation, and it depends on who you talk to. > I've been a heavy Vdbench user (great tool), but am at a crossroads > where I'm reevaluating tools for a future project. fio's open > development process is very appealing to me. > > If time allows, I'll test drive a few unit testing frameworks and > respond on thread w/ my branch. > > Jens, do you have any thoughts you can share? > > -bw > C. Brantley West, III > cbwest@xxxxxxxxx > > > On Sat, Jun 3, 2017 at 9:29 AM, Tomohiro Kusumi > <kusumi.tomohiro@xxxxxxxxx> wrote: >> Hi >> >> 2017-06-03 16:07 GMT+03:00 Sitsofe Wheeler <sitsofe@xxxxxxxxx>: >>> On 2 June 2017 at 20:30, Brantley West <cbwest@xxxxxxxxx> wrote: >>>> >>>> I'm looking to adopt fio for some storage testing in the near future >>>> and have noticed a few issues (#377, #378) that seem to be >>>> math-related. Assuming these issues get fixed, it seems like unit >>>> testing would help ensure they stay fixed. Additionally, fio has a >>>> lot of useful configuration options, and unit tests seem like a good >>>> way to test for expected operation when multiple configuration >>>> parameters are present. >>>> >>>> Is a unit test framework something that is worth considering for fio? >>>> If so, are there any preferences? It seems there are multiple options >>>> (check, cunit, unity, etc). >>> >>> I think it would be useful if there was an explicit make selftest >>> target for fio which could do more rigorous checks than my "does it >>> run?" make test. fio has the odd adhoc test (e.g. >>> https://github.com/axboe/fio/tree/master/unit_tests ) and now I look I >>> spy the odd test for checking certain job files for regressions too >>> (see https://github.com/axboe/fio/tree/master/t/jobs ). It would be >>> nice to run these and more via CI (travis/appveyor) on commit though >>> (assuming they finish quickly)... >>> >>> Some pieces of fio have been broken out into standalone programs (see >>> some of them in https://github.com/axboe/fio/tree/master/t/ ) which >>> might make testing for correctness easier. Perhaps Tomohiro Kusumi >>> (who has made pieces of fio more standalone) will be able to make >>> suggestions? >> >> Yes. >> Things under os/ arch/ lib/ oslib/ are mostly stand-alone utility >> functions now (i.e. you can include them or link to them via your >> main()), so those are probably easy to do unit testing using cunit or >> variants. >> >> Others more or less have dependencies on fio itself (and there isn't >> anything wrong with that too), so those are not as easy, or maybe >> depends on what you mean by unit testing. >> >>> >>> Personally I've used cunit (via the libiscsi project) but I've never >>> compared all C unit test frameworks. Further, the fio build doesn't >>> seem to add many dependencies on other libraries (other than for I/O >>> engines)... >>> >>> Way back in 2008 there was a proposal to add a "make check" to fio >>> (http://maillist.kernel.dk/fio-devel/0584.html ) which looked like it >>> was readily accepted but it's unclear whether something was actually >>> produced... >>> >>> If you've got any pre-existing checks please post/link them! >>> >>> PS: Do you find fio useful at Nutanix and are there any other public >>> I/O exercisers you folks use? -- Sitsofe | http://sucs.org/~sits/ -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
