On Tue, 2021-11-02 at 10:32 -0700, Adam Williamson wrote: > > 3. I'm not sure why Bodhi is still not allowing the update to be > submitted for stable even though the tests have been waived, this is > odd. I ran the greenwave query manually and it returns (in part): > > "policies_satisfied": true, > "unsatisfied_requirements": [] > > which should always satisfy Bodhi. If I've figured this out before (as > Kalev implied), then I've forgotten it now. :P But whenever the new > Bodhi version actually does get released and deployed, it tweaks > several things in this area, so whatever the problem is may get fixed. > Hopefully, once I re-run the openQA tests and they actually pass, Bodhi > will be happy. Further to this: I did re-run the tests, they did pass, it did make Bodhi happy, and I successfully submitted the update to stable. It should get pushed in the next push, I hope. It looks like what happened is that Bodhi didn't update its recorded gating status for the update when the waivers were submitted. Note there's two different calculations of the gating status in Bodhi, which can confuse things. The status you see on the right-hand side of the web UI - "All required tests passed" or whatever - is actually generated *by the front end code* whenever your browser loads the page. The JS front end code runs a (verbose) Greenwave query when you view the page, and uses the results to generate that status and also the Automated Tests results list itself. So that status will always be 'up to date'. When you try and push the update, though - whether by clicking on the button in the web UI, or using the CLI client - Bodhi doesn't use that status, because the Bodhi back end code doesn't *know* about that status at all. Instead it checks a property of the update object, which gets updated...sometimes. The "This update's test gating status has been changed to XXX" messages that get posted on the update periodically are actually telling you about *that* status check. Basically, unless the most recent message was "This update's test gating status has been changed to passed" or "This update's test gating status has been changed to ignored", you will not be able to push it. I think probably the bit that broke down here is that when you submitted your waivers, Bodhi didn't get told. The way this is *currently* supposed to work is that Greenwave listens out for new waivers, decides whether they change an update push decision, and publishes a greenwave.decision.update message if so. Bodhi listens for the greenwave.decision.update messages and either just accepts what they say or re-calculates the decision (it depends on some other stuff that doesn't matter rn). Looking through datagrepper logs, I see waiverdb.waiver.new messages from waiverdb when you created your waivers, but I *don't* see greenwave.decision.update messages in response to them. So I think Greenwave messed up and didn't think the waivers changed the decision. So Bodhi didn't get a message telling it to re-calculate the gating status, and it stayed at 'failed'. There is, I believe, a cron job that runs every few hours or every day or something that re-calculates the gating status of *every* active update, so it would probably have got caught up at some point. But it should really get recalculated as soon as a relevant waiver is submitted. I *hope* this should be fixed in the next Bodhi release, whenever it gets done and deployed. I actually rewrote how that works completely: https://github.com/fedora-infra/bodhi/pull/4230 mainly because I considered the existing design to be flat out wrong (for reasons I won't go into unless you really care :>), but also because I did, a few months ago, look through the code and find that greenwave could get this wrong for openQA tests/waivers. I forget the details, but there's some point at which it makes some assumptions which are only true for CI tests/waivers. I started out aiming to fix that, but instead decided the design was wrong and it made more sense to have Bodhi just listen out for new result / new waiver messages directly. I believe I wrote that properly so it will work for waivers/results related to both CI and openQA tests, but we'll find out for sure when it's deployed, I guess. :D Sorry again for the trouble! -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
