On Fri, 2019-09-06 at 09:38 -0400, Danny Lee wrote: > Hello all, > > I have a question related to regression and testing updates that I was > hoping somebody might have the time to answer. Thankfully, @kalev was > kind enough to give me an overview of whats going on this test in Bodhi > https://bodhi.fedoraproject.org/updates/FEDORA-2019-18f7fc4f08 but I > have further questions that I thought I'd bring to the mailing list. > > Please treat me like a long-experienced (mostly windows) computer > user/troubleshooter, who has never studied CS and only knows fragments > of programming languages at this point in time. I used unixes back in > the 90s, but never became overly skilled/knowledgeable. I've started > watching/listening to videos on regression testing and I think I > understand the overall concept. > > That all being said, I thank you for your patience, and the point of my > email is that I have questions related to the practical next steps for > this type of test - > > Question 1: > > Part A: I installed the .iso at > https://kojipkgs.fedoraproject.org/compose/branched/latest-Fedora-31/compose/Workstation/x86_64/iso/ > into a virtual machine (vmware) and started running tests. I ran > into some errors, so would I now go into > https://kojipkgs.fedoraproject.org/compose/branched/ and download > the latest branch prior to the 'latest-Fedora-31'? > > Part B: Could I switch my repository inside the installed system to > /etc/yum/repos.d/fedora.repo and uninstall Gnome Desktop (GD) and > then reinstall the GD from that repo and test to see if it worked > previously? This is a bit tricky if you're thinking in terms of regression testing, because there isn't exactly a 'known good' base for Fedora 31, or anything. The way Fedora development works is that we have a constantly rolling development distribution, Rawhide, which has no real formal quality requirements so it can be (and sometimes is) completely broken. New releases are branched from Rawhide periodically and then 'refined' to meet the quality requirements for stable releases. So a given release starts out in a nearly-unknown, potentially completely-broken state and, on average, gets 'better' until it's good enough to release. So there's no reason that any of the earlier snapshots you find there will necessarily be better than the current one, though one may *happen* to be. For testing an update in updates-testing, if you're thinking in 'regression' terms, the key question is "do things work better with this update installed than without it". If installing an update makes things *worse* than before, then it should probably get a -1. Testing updates for the in-development release (which we call 'Branched' - Fedora 31 is the current 'Branched' release) can 'feel' a bit different than testing updates for a stable release. If you're testing an update for a stable release, then it's usually the case that without the update installed, things are generally working fine. For Branched this is not always the case, the baseline might be...bad. :) What I'd generally say, for doing Branched testing, is just install the system and keep it up to date regularly, and when testing an update, compare how the packages in that update behave with the update installed to how they behaved before you installed it. I wouldn't suggest 'going backwards' to try and find some sort of clean baseline to compare the update to. > Question 2: > > Part A: In filing a bug reports for issues like these, what is > standard operating procedure for where to send the bug (wiki/web > links welcome here). Do I file upstream and at redhat bugzilla > first and then upstream after some period of time? Or do I file > upstream first or both? The simplest thing to do is always file at Red Hat Bugzilla, against the Fedora product, against the component that is the source RPM for the package with the bug ('rpm -qi package' will show you the source package name). Sometimes it's a good idea to file upstream as well - usually if the bug appears in a package that provides a new upstream release, for instance, that's an indication the bug is likely in the upstream release not the package. But you can't really go too far wrong by just filing in RHBZ. If you find a problem in an update, you should always put a comment *on the update* to explain this, and if it's a significant problem that's new - i.e. the previous version of the package didn't have the problem - you should probably give the package -1 karma (click the -1 button for 'Is the update generally functional? (karma)'). > Part B: I received the command 'journalctl -a -b' as something to > comb through, and adding screenshots, but is there anything else > that you (testers and bug-fixers) might want in your bug reports to > help make the issue clearer from the start? This is highly contextual, to be honest - it depends a lot on the bug. There are several 'how to debug' pages on the wiki which provide more specific info for certain types of bug: https://fedoraproject.org/wiki/Category:Debugging these can help if your problem falls into one of those categories; otherwise, journal output is indeed usually never a bad idea, and beyond that the bug assignee should be able to tell you what else they need. It's a good idea to specifically mention in your report that you're happy to provide additional information, so the assignee knows it's not just a 'drive-by' report and the effort spent explaining what additional information is required won't be wasted. There are some typical things, like, if the failing app has its own log files...attach those. If the failing app is something you can run at a console, run it at a console and see if you get any messages on the console, if so, paste (if short) or attach (if long) those. If the app has a man page or other docs, check it to see if there's any debugging tips there, and look for arguments you can use to get more info like '- -debug' or '--verbose' or '-v' or something. > Once again, thank you for your patience and insights, and thanks to > @kalev for your help. It is all much appreciated in trying to get over > this learning curve :) No problem, thanks for writing and testing! -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
