On Sat, 2016-07-16 at 14:35 +0000, Zbigniew Jędrzejewski-Szmek wrote: > On Wed, Jul 13, 2016 at 09:44:36AM -0700, Adam Williamson wrote: > > > > On Wed, 2016-07-13 at 11:41 +0200, Vít Ondruch wrote: > > > > > > If I should provide "how to test" information, it should be probably > > > script which should be run by AutoQA or something. But TBH, I have no > > > idea how to provide such script. There is precisely zero information > > > about this stuff on places I would expect (this one [1] as an example). > > > There is also zero support in tools such as fedpkg and dist-git. > > > > If you're willing to do this, that's great, and we would definitely > > like to enable it. I know the Taskotron (it hasn't been AutoQA for a > > while) team is working on enabling generic dist-git integration (which > > is basically what you're asking for), but I don't know the current > > status of that...CCing tflink and kparal who may have more information. > > > > > > > > I am pretty sure this would be more valuable then some "how to test" > > > document. > > > > I think it varies substantially by package, but I don't think we're > > realistically going to get sufficient automated testing for all > > packages, including ones that are important to users. > > I don't think koschei has been mentioned in this thread yet. > For library type packages that's the testing we want to do, and if > I'm not mistaken, koschei compilations already are started for dependent > packages when an rpm hits updates-testing, if enabled for those packages. > This kind of testing complements the usability testing that is > suitable for leaf packages. > > So adding testing instructions might help for some types of packages, > for other types we should be working on integrating koschei more > closely, in particular flagging the update that breaks builds. Well, there can be functional aspects to library testing too; take GTK+ as an interesting case, you can have a GTK+ update which passes all its internal tests and all or most packages rebuild fine against it, but it changes some bit of CSS or something and makes text go white in Firefox, or something like that. It's happened before. ;) But yeah, there are definitely cases - as I said - where the style of human testing that Bodhi is kinda designed around doesn't really fit, and we could definitely improve things. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://lists.fedoraproject.org/admin/lists/devel@xxxxxxxxxxxxxxxxxxxxxxx
