On Fri, 2017-03-17 at 23:00 -0400, Dusty Mabe wrote: > > > I've just verified that this is actually working: when openQA tests the > > 'two-week Atomic' composes and a failure occurs, the mail does get > > sent. However, there hasn't been a failure of the test since - AFAICS - > > 2016-10-01. So that's why no such mails have been sent out recently. I > > got Mike to check, and he actually did get a mail on 20167-10-01, when > > the test last failed. > > woot for passing tests! Adam, do you know if there is any way to send > a weekly report that basically states how many tests ran and how many > passed? That would at least let us know that they were there and still > running and passing. > > I imagine now that the tests results are in resultsdb the answer is > going to be something related to that. The thing that sends the per-compose reports is just a script that I wrote, called check-compose: https://pagure.io/fedora-qa/check-compose there's a fedmsg consumer which notices when the last test for a compose is run, and runs check-compose for that compose with a configuration that results in the mails being sent. check-compose currently has no ability to do a weekly summary report or anything like it, it can only generate reports for exactly one compose at a time. It wouldn't be too difficult to write such a thing (either as an extension to check-compose or as a separate project), but someone would have to do it. check-compose queries openQA directly, rather than resultsdb; I wrote it before we were forwarding results to resultsdb. It can't be ported to only use resultsdb as it would still have to query openQA directly for some stuff that isn't forwarded to resultsdb and really can't be (like the logs it uses to report on system resource usage and stuff). At some point I'd like to adjust it to use rdb as well as direct openQA queries so it can include info on results from other test systems, but it's not at the top of my priority list. You could write a simple 'weekly summary' kinda script purely from the info in resultsdb, and that way you could consider the results from openQA and autocloud (currently) / taskotron (future?) together. I do believe that's the general idea for this kind of status reporting indeed - if you want that kind of info, build something that uses resultsdb, whether it's some kinda web dashboard or script or whatever you like. > Yes! we would love a UEFI test and I think it would actually be good > to run the atomic-host-tests against these images assuming openQA is > the right tool for that job. I thought openQA was more for interactive > install testing, so please let me know where I'm wrong. roshi knows > openQA and atomic-host-tests so he might be able to comment here. You can do just about anything with openQA, really; we *could* do all the testing we're currently doing in other systems in it. But it wouldn't necessarily be the *best* way to do every kind of testing. :) Interactive install testing is the classic example of a thing openQA can do quite well that few other things can do at all. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ cloud mailing list -- cloud@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to cloud-leave@xxxxxxxxxxxxxxxxxxxxxxx
