Greetings Testers! With the F22 switch from yum to dnf as a package manager, there was a need to figure out what the scope would be for QA testing. Naturally, to define the scope for a subset, I had to figure out (at least in a general sense) how large the complete set is. There's a multitude of complication vectors with a switch like this - especially since dnf is specifically designed to not be a drop in replacement for yum. First question: Who/what all uses yum? -------------------------------------- Swapping out yum is the equivalent of someone coming up with a new form of hemoglobin, then trying to figure out what all could go wrong when you make the swap (tl;dr: Everything). The first to come to mind is anyone who installs packages from the CLI. But then you have all the GUI front-ends (PackageKit and Apper) and all the build tools - not to mention the installer. Some places yum touches: - Oz/ImageFactory (Building Cloud images) - koji (RPM-based build system) - pungi (Build installation trees and isos) - ABRT (Automatic Bug Reporting Tool) - Anaconda (Fedora installer) - liveimage-creator (tool for creating live images) - cloud-init (tool for configuring freshly launched cloud images) - FedUp (tool for updating Fedora between releases) - Software Center and Apper (Default software installers) - rolekit (tool to easily deploy roles to a Server installation) - virt-builder and friends (tools for manipulating virtualized environments) All of those make up a large portion of the Fedora ecosystem - everything from composing images to users being able to update their software packages, and this doesn't even touch the different ways each of the above actually *uses* yum functionality - just that yum somehow impacts these tools. Second question: What's QA responsible for? ------------------------------------------- According to the QA wiki page: "Fedora QA is the project which covers all testing of the software that makes up Fedora. It's our goal to continually improve the quality of Fedora releases and updates." A more literal translation of that passage would indicate that QA is responsible for all the testing, but a more nuanced interpretation says that QA is responsible for releases and updates to those releases. Luckily for us in QA, we seem to operate from the more nuanced interpretation. So, out of that list above, which things would QA need to test? - Anaconda (which we already test heavily) - ABRT - FedUp - Software Center/Apper - rolekit While those are the things easily placed directly under QA for testing, other teams would likely appreciate help in testing their tools. This doesn't necessarily equate to more work for QA, since a lot of the feedback the other teams (Release Engineering, Infra) get is from QA testing the output their tools create - releases and updates. Also, those 5 things aren't trivial to test either. Here's the partial breakdown: Anaconda: - GUI installation - Do all the offered mechanisms work? - Package selection (installing all the packages selected) - Remote repositories - text based installation - kickstart installation - This is non-trivial to test ABRT: This tool uses a bunch of yum tricks and workarounds to get the needed information (AIUI) - We have 36 testcases for this (some will need updated) FedUp: - F20-F22 upgrades - How to handle migrating existing systems from yum to dnf. Each will have different installed package sets due to utilizing different depsolvers. - This will require more testing during the F23 cycle than during the F22 cycle Software Center and Apper: These tools aren't just a front-end for yum/dnf, but tie into the libraries dnf uses. For the switch to dnf, this is hawkey and libsolv. It could be argued that this doesn't need to be tested as part of the migration, but they for sure fall under the Default application functionality Final criterion. rolekit: We get a small pass for this one, since there's currently only two supported roles: Domain Controller and Database. rolekit relies on yum for installing package sets for the role, so it'll have to be ported and tested. Hopefully this requires minimal extra testing. Third question: What do we currently have to work with? --------------------------------------- Our wiki has a plethora of testcases and documentation we can utilize throughout this transition. We should be able to use/re-purpose large portions of our existing testcases to get good test coverage of the change. We might also have to take a look at the Release criteria going forward to make sure there aren't any yum-isms left over. All this means is that we don't have to start from scratch when it comes to testing this. Testcases: I count 54 testcases currently available to test ABRT, FedUp, Yum, Installation Repos, and kickstarts. Many of these will likely need revised and more will likely need added to get the coverage we need. Matrix Changes: For the image sanity checks, we'll have to make sure a successful run with dnf is within the bounds required of yum. We'll also have to go through and make sure that existing instructions within testcases use dnf (and still work). There might be some F22 specific criteria we need to add in order to set a baseline for what has to work with dnf for the release. Kickstarts: We already use kickstarts for most of the heavy lifting on the releng side of things, so we have access to a known set of things that have to work with kickstarts. Testing these should be pretty simple and easy to grow/use going forward. Conclusion ========== Migrating from yum to dnf isn't going to be an easy task. On the upside, it will help us identify all the places we use yum without thinking about it and may not actively test for. So while it's going to be a lot of work, it lets us lay the groundwork for any other hemoglobin-esque changes Fedora sees in the future. We've already had an Anaconda/DNF test day (more are likely to come) and will probably need to schedule some test days for other large projects as well. There's probably stuff that I missed, or overestimated the difficulty on (I'm not terribly familiar with all the areas this encompasses), so I don't think this is a complete list. If you noticed something that should be here please reply! It'll really help us to get as complete a list of things to test that we can so we don't end up hitting a bunch of snags when we get to release. I've put most of this email up on the wiki [0] - so feel free to add things there as well. Thanks! [0] https://fedoraproject.org/wiki/QA:DNF_Migration_Test_Overview -- // Mike -- Fedora QA freenode: roshi http://roshi.fedorapeople.org -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
