On 07/19/2011 09:49 AM, Sergio Rubio wrote: > Greetings! > > Posted this to the wrong mailing list (David Cantrell suggested using > this one instead). > > Linking to avoid re-typing :), my apologies: > > https://www.redhat.com/archives/anaconda-devel-list/2011-July/msg00090.html > > In a nutshell, we are looking for better ways to test our Anaconda > based installer and I thought that speaking to the real guys and > sharing would be a good way to start. > > Any feedback is greatly appreciated. Cool, we're always looking for ways to do better testing. As James said, honqing and twu are more actively working on the installer automation but I have a couple of design questions. Why virtualbox? - I was under the impression that VB didn't always play nice with the fedora kernels and was a reason to use KVM/qemu instead - Then there's the whole licensing issue ... How well does it adapt to small changes in visual layout? - The scripting looks very rigid to me which is fine for things that don't change very often - Do the scripts have to be completely re-done for small visual changes or is it possible to just change a portion? How often are you seeing false positives/negatives? - ImageMagick's 'compare' seems awfully sensitive to me, have you seen many problems with the image comparison logic? - If I'm reading the code correctly, it seems to be looking for an exact match How are you defining pass/fail? - From my very quick scan of the code, it looks like PASS -> no crash during installation FAIL -> image comparison failures, unexpected events, crash - Is there any verification that the VM boots after install? How do you handle crashes? - Are you collecting anaconda stack traces on crash? - Otherwise, is the VM in a state that allows for data collection if a crash occurs? Another thing that you may be interested in is OpenSuSE's openqa project [1] and specifically, os-autoinst [2]. Thanks, Tim [1] http://openqa.opensuse.org/ [2] http://www.os-autoinst.org/
Attachment:
signature.asc
Description: OpenPGP digital signature
-- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
