On Mon, 2016-10-24 at 16:49 -0700, Adam Williamson wrote: > On Mon, 2016-10-24 at 20:19 +0000, Fedora compose checker wrote: > > No missing expected images. > > > > Failed openQA tests: 82/101 (x86_64), 3/17 (i386), 1/2 (arm) > > Executive summary: I broke it, yo. Slightly more detailed version of what I think happened: We had a bit of an issue with how openQA registers and then cleans up 'assets' (which are things like ISOs and hard disk images) that I fixed last week. There was a bit of remaining cleanup work to do which I was going to do after these tests ran (because of boring details, it wasn't possible to fully complete the cleanup until some tests ran a full two days after I installed the fixed patches). Unfortunately the mess resulted in the asset cleanup code deleting the assets that were actually needed for these tests before most of them had even run: it decided it needed to free up space, but it couldn't find older assets that were downloaded during the buggy period, so it could only find the newer assets to delete. There also turned out to be a bug in one of the fixes I submitted which meant openQA wouldn't properly log the name of the asset it was looking for but couldn't find, which confused the issue a bit. I've now cleaned up the consequences of the initial asset registration bugs, so hopefully openQA should no longer decide it needs to delete assets that we haven't finished using yet, and sent a fix for the logging bug. I've re-sent the 20161024.n.0 Rawhide tests and will send a new compose check report manually when they complete (assuming they work properly this time). We possibly could improve the asset cleanup code to check for pending jobs that use the assets it's about to delete and not delete them in that case, that might be a useful safety precaution in case of weird issues like this in future... -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx
