On Wed, 5 Mar 2014 20:58:32 +0900 "Sandro \"red\" Mathys" <red@xxxxxxxxxxxxxxxxx> wrote: > Heads-up: I've taken ownership for the external need "Automatic > Smoketests on Image Build" [0] > > Testing an image takes time and resources, and with several images, it > takes several times that. Since that simply doesn't scale - well, > doesn't even work well without scaling - we want to automate the > smoketests using Taskotron. Unfortunately, Taskotron is still in a > very early development phase and having mentioned only just our (most) > basic needs, Tim estimates it won't be good enough for us in the > target timeframe (i.e. by F21 Alpha). There's a slight chance it will > be ready, but even then we still need time to implement the > smoketests. > > So if we want to have Taskotron ready for us in time, we need to find > some developer(s) committing some of their resources. Any takers? If > you're interested, head to [1] to learn more, there's also a > contribution guide linked from there. I'm sure tflink in #fedora-qa is > happy to talk to people still interested after reading through the > wiki if there's further questions. To go into a bit more detail, the taskotron bits which would be required would depend a little bit on what comes out of the cloud image generation. I'm assuming the following: - the image creation process will generate a fedmsg upon completion that's relatively simple to differentiate from other fedmsgs. - this fedmsg contains some unique information which can be used to obtain a path to the generated image - the image is not already being loaded into a cloud-ish system - some cloud-ish system (ie openstack) is available for testing If my assumptions are correct, the scheduling component would be a relatively simple change. Taskotron would need one or two new modules to do the following: - take the fedmsg information, upload the desired image into the cloud-ish system and return the image id - launch an instance with a specified image id, flavor and key id. return the instance's IP address There would be a complication here in that Taskotron doesn't have a concept of shared secrets for the ssh private key or but I'm sure we could figure something out. > Also, we need to compile our actual needs. What kind of tests do we > want to run? Most basic tests (things that we manually tested in the > past) are probably: > - Was the IP correctly set up? > - Is SSH running and reachable? > - Does the fedora user exist? > - Is the ssh pubkey installed for the fedora user? > - Is the fedora user able to use sudo? > - Is the root account locked? > - Is package installation working? > - Is the firewall disabled? > - Checking cloud-init and systemctl for bootup errors - does that make > sense or is there too much noise? The easiest way to write a task to check stuff like this would be to write it in python. Input would be a key id (see the shared secret comment above) and a user id. The task would check the items that you're interested in, collate and return the results. Taskotron would then report results and emit a fedmsg indicating said result (fedmsg code is yet to be completed but is slated for our first deliverable). I realize that the documentation for how to do this is non-existent at the moment. We've been focusing on getting the details figured out and a staging system up and running before getting howtos written. We have a trivial example written now (rpmlint) and will have a more complicated example (depcheck) mostly complete before the end of next week. While those are indeed package-level checks, they will have examples of the task definition format (YAML) and the interface needed for tasks written in python. I hope this is enough detail to at least outline what would need to be done. I'd love to see it done but I'm not sure what our (qa-devel) priorities are going to be once the base Taskotron system is in place and thus, not sure if it would otherwise be done before F21-alpha. Please let me know if you have any questions, I'm happy to help where I can. Tim
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ cloud mailing list cloud@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
