On Fri, Mar 7, 2014 at 6:37 PM, Vitaly Kuznetsov <vitty@xxxxxxxxxx> wrote: > "Sandro \"red\" Mathys" <red@xxxxxxxxxxxxxxxxx> writes: > >> Heads-up: I've taken ownership for the external need "Automatic >> Smoketests on Image Build" [0] >> >> Testing an image takes time and resources, and with several images, it >> takes several times that. Since that simply doesn't scale - well, >> doesn't even work well without scaling - we want to automate the >> smoketests using Taskotron. Unfortunately, Taskotron is still in a >> very early development phase and having mentioned only just our (most) >> basic needs, Tim estimates it won't be good enough for us in the >> target timeframe (i.e. by F21 Alpha). There's a slight chance it will >> be ready, but even then we still need time to implement the >> smoketests. > > I just want to point out the fact that for RHEL/Fedora/etc image testing > we have the following tool: > > https://github.com/RedHatQE/valid > > We were using it to test Fedora images since F18. > > To be honest I'm not that familiar with Taskotron but maybe we can > collaborate somehow. > >> >> So if we want to have Taskotron ready for us in time, we need to find >> some developer(s) committing some of their resources. Any takers? If >> you're interested, head to [1] to learn more, there's also a >> contribution guide linked from there. I'm sure tflink in #fedora-qa is >> happy to talk to people still interested after reading through the >> wiki if there's further questions. >> >> Also, we need to compile our actual needs. What kind of tests do we >> want to run? Most basic tests (things that we manually tested in the >> past) are probably: >> - Was the IP correctly set up? >> - Is SSH running and reachable? >> - Does the fedora user exist? >> - Is the ssh pubkey installed for the fedora user? >> - Is the fedora user able to use sudo? >> - Is the root account locked? >> - Is package installation working? >> - Is the firewall disabled? >> - Checking cloud-init and systemctl for bootup errors - does that make >> sense or is there too much noise? >> >> What else can / should we test? Obviously the tailored images should >> be tested for their specific stuff but lets first focus on the common >> ground. >> > > In addition to that current image validation has the following: > - testing different instace types > - empty bash history > - selinux contexts test > - running services > - package set > - EBS test > - Ephemeral test > - available cores/memory test > - userdata handling (cloud-init) > - several bugs test (regressions) > - ... (full test list is here: https://github.com/RedHatQE/valid/tree/master/valid/testing_modules) > >> Thanks, >> Sandro >> >> [0] https://fedoraproject.org/wiki/Cloud_Changelist#External_Need:_Automatic_Smoketests_on_Image_Build >> [1] https://fedoraproject.org/wiki/Taskotron So we have the RedHatQE tests, Taskotron and CentOS's CI. Can anyone of the people involved (at the Red Hat side, I guess) well me why we have 3 systems for 1 task? When I took ownership of this "external need" (for the Fedora cloud product) I was under the impression we only just (are going to) have Taskotron and everyone knows it's THE way to go. -- Sandro _______________________________________________ cloud mailing list cloud@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
