"Sandro \"red\" Mathys" <red@xxxxxxxxxxxxxxxxx> writes: > On Fri, Mar 7, 2014 at 6:37 PM, Vitaly Kuznetsov <vitty@xxxxxxxxxx> wrote: >> "Sandro \"red\" Mathys" <red@xxxxxxxxxxxxxxxxx> writes: >> >>> Heads-up: I've taken ownership for the external need "Automatic >>> Smoketests on Image Build" [0] >>> >>> Testing an image takes time and resources, and with several images, it >>> takes several times that. Since that simply doesn't scale - well, >>> doesn't even work well without scaling - we want to automate the >>> smoketests using Taskotron. Unfortunately, Taskotron is still in a >>> very early development phase and having mentioned only just our (most) >>> basic needs, Tim estimates it won't be good enough for us in the >>> target timeframe (i.e. by F21 Alpha). There's a slight chance it will >>> be ready, but even then we still need time to implement the >>> smoketests. >> >> I just want to point out the fact that for RHEL/Fedora/etc image testing >> we have the following tool: >> >> https://github.com/RedHatQE/valid >> >> We were using it to test Fedora images since F18. >> >> To be honest I'm not that familiar with Taskotron but maybe we can >> collaborate somehow. >> >>> >>> So if we want to have Taskotron ready for us in time, we need to find >>> some developer(s) committing some of their resources. Any takers? If >>> you're interested, head to [1] to learn more, there's also a >>> contribution guide linked from there. I'm sure tflink in #fedora-qa is >>> happy to talk to people still interested after reading through the >>> wiki if there's further questions. >>> >>> Also, we need to compile our actual needs. What kind of tests do we >>> want to run? Most basic tests (things that we manually tested in the >>> past) are probably: >>> - Was the IP correctly set up? >>> - Is SSH running and reachable? >>> - Does the fedora user exist? >>> - Is the ssh pubkey installed for the fedora user? >>> - Is the fedora user able to use sudo? >>> - Is the root account locked? >>> - Is package installation working? >>> - Is the firewall disabled? >>> - Checking cloud-init and systemctl for bootup errors - does that make >>> sense or is there too much noise? >>> >>> What else can / should we test? Obviously the tailored images should >>> be tested for their specific stuff but lets first focus on the common >>> ground. >>> >> >> In addition to that current image validation has the following: >> - testing different instace types >> - empty bash history >> - selinux contexts test >> - running services >> - package set >> - EBS test >> - Ephemeral test >> - available cores/memory test >> - userdata handling (cloud-init) >> - several bugs test (regressions) >> - ... (full test list is here: https://github.com/RedHatQE/valid/tree/master/valid/testing_modules) >> >>> Thanks, >>> Sandro >>> >>> [0] https://fedoraproject.org/wiki/Cloud_Changelist#External_Need:_Automatic_Smoketests_on_Image_Build >>> [1] https://fedoraproject.org/wiki/Taskotron > > So we have the RedHatQE tests, Taskotron and CentOS's CI. Can anyone > of the people involved (at the Red Hat side, I guess) well me why we > have 3 systems for 1 task? (my personal opinion) I think we rather have plenty of tasks, not one. Afaict (after 5 min. of reading Taskotron's development plan https://fedoraproject.org/wiki/User:Tflink/taskotron_development_plan) Taskotron is designed to replace AutoQA in the first place. RHEL's Cloud Image Validation was developed several years ago when the following task was on the table: we have many AWS regions, many images, different architectures, we need to try different hardware types and AWS-specific features (e.g. attach EBS on the fly or test AWS-specific content delivery) and finally we need to aggregate the result. Existing test infrastructure was built around Beaker which is not that well suited for the job and creating a separate tool was considered a reasonable trade-off. > When I took ownership of this "external > need" (for the Fedora cloud product) I was under the impression we > only just (are going to) have Taskotron and everyone knows it's THE > way to go. I personally love collaboration. It would be awesome if we could avoid spreading resources on '3 systems for 1 task'. I definitely want to know more about Taskotron and its movement towards cloud image testing. -- Vitaly Kuznetsov _______________________________________________ cloud mailing list cloud@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
