On 12/11/19 8:54 PM, Jens Axboe wrote: > On 12/11/19 1:10 PM, Vincent Fu wrote: >> On 12/11/19 2:57 PM, Jens Axboe wrote: >>> On 12/11/19 12:53 PM, Vincent Fu wrote: >>>> On 12/10/19 4:32 PM, Sitsofe Wheeler wrote: >>>>> On Tue, 10 Dec 2019 at 17:56, <vincentfu@xxxxxxxxx> wrote: >>>>>> >>>>>> From: Vincent Fu <vincent.fu@xxxxxxx> >>>>>> >>>>>> Jens, please consider this series of patches related to testing. >>>>>> >>>>>> The patches improve t/run-fio-tests.py in various ways, most prominently >>>>>> adding support for Windows and macOS. >>>>>> >>>>>> Also included are travis and appveyor patches that add run-fio-tests.py >>>>>> as a step. Currently both the travis and appveyor build processes >>>>>> complete in less than four minutes. Adding run-fio-tests.py increases >>>>>> this to about 20 minutes for travis and 14 minutes for appveyor. >>>>> >>>>> In general I think this work is fantastic and much needed (you can >>>>> search through the fio commit logs using "git log --grep 'size='" to >>>>> find jobs files that have caused issues in the past and may be worth >>>>> turning into tests at some point). However, I think making the builds >>>>> so slow may be a disadvantage rather than a benefit. I agree with >>>>> nearly all the patch set bar running this by default with >>>>> travis/appveyor... >>>>> >>>> >>>> Many thanks for the feedback, Sitsofe. I agree that the build times are >>>> uncomfortably long, but since I had done the work I thought I would >>>> offer the patches to Jens. >>>> >>>> Jens, what do you think? Would you like me to re-send the patch series >>>> without the appveyor and travis changes? >>> >>> Looks like it's about 20 min, I'm not so sure we need faster turn-around >>> than that. I would personally much rather see a pull request with 20 min >>> delay on whether it passes or not, rather than have to run it manually. >>> That's how things get missed. >>> >>> So I'd be leaning towards just making it run it by default. Is 20 min >>> really that big of an issue? It's not like it's holding anyone up. >>> >> >> I think the main difference is that at 4 minutes, it's easy enough to >> wait around to check that the build has succeeded, whereas at 20 minutes >> one has moved onto another task and will have to make a context switch >> to come back to check on the build. Also, if people add more tests the >> build times will become even longer. >> >> All that said, it's easy enough to revert the patch if adding the tests >> hurts the workflow too much. > > I don't disagree that it's a long time. But the idea is that you push > it out to a branch, and then you get a notification some time later > if it worked or failed. As long as the notification works for both > cases, then I don't think it's much of an issue. Even 4 minutes will not > have people sitting around waiting for it to finish, at least not me. > I'd be on to other stuff the second I push it out. > > Let's try this out and see how it goes. If it doesn't work so well, then > we always have the option of slimming down the default run, or whatever > else we can do to make it faster. It seems to fail, though: https://travis-ci.org/axboe/fio/jobs/625901116?utm_medium=notification&utm_source=email -- Jens Axboe
