Hi folks! So, since we've been getting Workstation ostree installer images in the last few Rawhide composes, I've been setting up openQA to test it. This is not at present deployed to production openQA, but it *is* deployed to staging openQA. Unfortunately the last few Rawhide composes have contained a broken anaconda so the image has not been built, but the 20170615.n.0 compose *did* feature a working image, and you can see the test results there: https://openqa.stg.fedoraproject.org/tests/overview?distri=fedora&version=Rawhide&build=Fedora-Rawhide-20170615.n.0&groupid=1 (note that *right now* there's an infra outage going on so as I write this the page is inaccessible, but all should be back to normal shortly). Look for the Workstation dvd-ostree 'Flavor'. As you can see, for now I just have it running a subset of the regular Workstation tests, with the update-related tests - which assume an RPM-based system - left out, for now. Everything mostly appears to work well, except for the known bug https://bugzilla.redhat.com/show_bug.cgi?id=1193590 (that's the cause of most of the tests showing up as 'soft failures', the 'orange' state, rather than 'passes' / green). So I have two purposes here: firstly just to let you know that this testing is now happening (they will be run for any Rawhide or Branched compose which contains a Workstation ostree installer image), and secondly to ask about any other testing that would be useful. Is there any practical way we can actually test the ostree update process, for instance? For testing RPM-based updates, what we do is install a 'dummy' python3-kickstart package, with a NEVR known to be lower than any 'real' version of the package, so we know for sure that an update will be available from the official repositories, and then we go ahead and run an update in the usual fashion. Would there be any similar kind of possibility for testing ostree updates? The goals of openQA testing are to test as 'realistically' as possible, so we only test unmodified images, and want to have the test behave as similarly as possible to how the 'real' operation would behave on an end user's system. And besides updates, can anyone think of any other test that would be particularly useful to run? Of course, to be clear, this is mostly 'advisory' testing for now: this image is still not a release-blocking image, so failures of the tests will not automatically trigger any kind of special attention or even bugs being filed, it will require people to be interested in looking at them and filing bugs. I will try and do so, of course. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ desktop mailing list -- desktop@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to desktop-leave@xxxxxxxxxxxxxxxxxxxxxxx
