On Fri, Jan 29, 2021 at 02:07:31PM +0100, Michal Privoznik wrote: > On 1/29/21 1:45 PM, Pavel Hrdina wrote: > > On Fri, Jan 29, 2021 at 09:30:24AM -0300, Daniel Henrique Barboza wrote: > > > > > > > > > On 1/27/21 2:59 PM, Michal Privoznik wrote: > > > > Since we've switched to meson our tests run with a timeout (meson > > > > uses 30 seconds as the default). However, not every machine that > > > > builds libvirt is fast enough to run every test under 30 seconds > > > > (each test binary has its own timeout, but still). For instance > > > > when building a package for distro on a farm that's under load. > > > > Or on a generally slow ARM hardware. While each developer can > > > > tune their command line for building by adding > > > > --timeout-multiplier=10, this is hard to do for aforementioned > > > > build farms. > > > > > > > > It's time to admit that not everybody has the latest, top shelf > > > > CPU and increase the timeout. > > > > > > > > Signed-off-by: Michal Privoznik <mprivozn@xxxxxxxxxx> > > > > --- > > > > > > This sure will help these build farms environments, but what about the cases > > > where an actual timeout means that there is something wrong with the code? > > > E.g. commits 46d88d8dba56 and 2ba0b7497ce7 were only possible because I was > > > seeing tests timing out in Power hosts when the 30 sec timeout was being > > > enforced. > > > > > > A 120 second default timeout for the majority of the test cases is a long time. > > > virschematest in this laptop I use takes 2.5 sec to complete. If I do something > > > wrong in the code and now the test is now 4 times slower (10 sec) I will not be > > > able to detect it (I'll need to start keeping track or something). You'll have to > > > run the test suit on your RasPi 2B to see that something went wrong because the > > > timeout is better tuned to your RasPI than this laptop, but then the code is already > > > upstream. > > > > > > And the tests will get more complex and will naturally take longer to complete. > > > Eventually this timeout might no be enough. Increase the timeout again? > > > > > > Meson 0.57 has support for disabling test timeout entirely with --timeout-multiplier=0, > > > disabling test timeout entirely. Can't we bump the meson version to 0.57 and > > > then tell the distros to use timeout-multiplier=0? That will solve the problems > > > for them, I keep the short timeouts for development, and we won't need to keep > > > bumping the test timeout once every 2 years or something because the s390x > > > TCG enviroment in Fedora COPR is timing out again. > > > > Agreed. The timeout is useful and we can always find a machine where the > > current timeout will not be good enough. I don't think it is a good idea > > to increase the timeout like this for only few specific use-cases. > > So we agree that timeouts are evil :-) Back in the autotools days we did not No we don't :) timeouts are useful in test suites to reveal issues that something takes a long time when it shouldn't. Pavel > use any timeout at all and I don't remember that causing any trouble. But > maybe it was so painful that I blocked it in my mind :-) > > Michal
Attachment:
signature.asc
Description: PGP signature
