On 11. 01. 21 5:26, Adam Williamson wrote:
On Mon, 2021-01-11 at 00:46 +0100, Miro Hrončok wrote:
On 10. 01. 21 23:25, Matthew Miller wrote:
On Sun, Jan 10, 2021 at 02:20:04PM -0800, Adam Williamson wrote:
All this does is making it again harder to issue bug fixes for the very
packages where it matters the most.
But...if the tests pass it doesn't, and I already said that the tests
pretty much always pass and I actively work to resolve any case where a
test fails when it shouldn't.
You're not going to be waiving results for every update, here. It would
be a pretty rare occurrence.
And, also hopefully also a rare occasion, but if this were enabled (and the
definitions up to date), problems like
https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx/thread/CAS6KHTZLR6LUNWEVK3BOIO6HVNQDETZ/#N5HJDKMTGOTL44BT2HZ43LE6Q23345IQ
would be caught before they hit users.
I believe we should gate on installability first.
See https://pagure.io/fesco/issue/2343
If the package is actually part of the test scenario, openQA
effectively does this. There is a check wired into all the update tests
at the end. It checks whether any package that's in one of the updates
is installed, but is *not* the version from the update, and fails if so
- because this usually means the package was not installable, so when
the test updated the system with the packages from the update
available, dnf did not update it.
That's great!
openQA won't catch a package not being installable if it is not
actually part of the system-under-test in any of the tests, but most
critical path packages ought to be.
Cool.
Is there any reason you think these need to go in a particular order?
Technically, we can roll those in any order. I just think that a "simple"
instability test might be easier to grasp by packagers, especially those who
argue this is complicated already. OpenQA test failures require a certain skill
in order to be able to see what's wrong. So do the rpmdeplint tests, but
arguably packagers are more likely to have such skill already. Once everybody
gets the idea of such test and learns how to work with this, we could extend the
test set.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
